To interject with my own babble, why does a composited desktop require OpenGL?
So that the people who are good at writing accelerated video code can concentrate on doing that, and the people who are good at writing GUI code can concentrate on that, and neither have to make a half-arsed job of doing the other.
Same reason you (I'm guessing) go down to the baker to buy bread, even though it's trivial to make your own.
(Substitute in some other foodstuff, or clothing, or anything in your life you don't manufacture yourself if you do happen to bake your own bread.)
I had already thrown out OCFS2 and GFS2 as possible candidates, but that was irrelevant to my reply. Also currently I am unaware of any non-proprietary hardware or software RAID (mdadm in particular) that supports active/active or active/passive on a shared backplane at any RAID level other than 1 or 0 (i.e. DRBD) and rather expensive and not yet released Areca external RAID controllers [areca.com.tw]. Also I'm looking for whitebox OSS solutions.
This helps to explain more about what you want to do, but doesn't really help to explain why you want to do it (ie: is it the right solution).
I can tell you about a solution we used to use at a previous job for bulk, cheap storage. This scaled from ~12T at the start of its life through to ~192T at the end, and was built using Linux cLVM and Promise vTrak disk shelves. It briefly also featured GFS2.
The primary objective was lots of cheap, redundant space. Availability, and especially performance, were not high priorities. With that said, it was a (mostly) highly-available solution.
At the base storage layer, were FC-attached Promise vTrak disk shelves. These were attached via two FC fabrics to two "controller nodes". The disks in the shelves were configured in single RAID6s of 15 drives + 1HS and presented to the two controller nodes as a single (multipathed) LUN.
These LUNs were aggregated into a single logical storage pool using Clustered LVM, then redivided into LVs and controlled by the automounter.
This system was used as an archival storage target for image files. It originally ran some proprietary software which would "receive" collections of image files, then derive some metadata about that collection and store them together. At the end of each day a consolidation process would create an appropriately-sized LV and move the last 24 hours' worth of images onto it. Initially, since the "receiving" process was done on the controller nodes, GFS2 was used so the load could be balanced. As the load grew, however, we decided to split off the receiving function onto dedicated machines, and with that the need for load-balancing and a clustered FS disappeared. So, we switched to ext3, reconfigured the controller nodes in an active/passive failover using heartbeat, and shared the storage to the new receiving nodes via NFS (with some SSH scripts to handle the daily LV creation on the controller nodes). Whenever a disk shelf would fill up, we'd simply buy another one, stuff it full of COTS drives, configure it up as a big RAID6, present the LUN to the controllers units and add it to the VG. With that system we went from a single shelf and 12T all the way up to eight shelves and ~192T, with something like 800 LVs averaging around 200G each.
The weakness in this design is, obviously, the single set of disks on the back end that represent a SPOF. However, since the images were regularly rolled off to tape, and both the RTO and RPO were relatively high, it was considered a reasonable tradeoff.
When originally conceived, Linux RAID in the CentOS of the day (4.something) was relatively immature, and in particular it lacked the bitmap capability present now. As such, the decision was made to go with the hardware RAID in the vTraks. Were I to do it again, I would use Linux software RAID, though this would necessitate some slightly more involved failover scripting to re-assemble the RAID arrays on the second host as part of a failover event. Obviously the array couldn't be active on both hosts simultaneously (though there was no real need for that to be possible anyway).
Additionally, the storage bus in our case was FC, which makes sharing physical devices to multiple servers trivial. However, anything capable of having multiple hosts (eg: SAS) would work just as well.
A second, similar, project was to have a more highly-available image store, with no SPOFs. For that we used pairs of servers, Linux software RAID and LVM on SAS shelves (for sufficient spindles), each shelf dedi
I've worked with VMs on all different kinds of storage (fiber channel SAN, local disk, iSCSI SAN (over 1Gb and 10Gb ethernet), Local hardware raid, NFS file shares, GFS2 (as in the RedHat cluster file system), and MooseFS and GlusterFS) All of these have been either in large test labs or in production cloud deployments. I've never had a cluster file system get close to passing muster as a storage medium for VM usage. IO is the number 1 bottleneck in virtualized environments, and these schemes just add completely unacceptable latency and bandwidth restrictions.
Not to put too fine a point on it, but VMFS is a cluster filesystem and runs VMs exceptionally well.
Also, NetApp (and EMC, and others) have demonstrated multiple times that the performance difference between FC, NFS and iSCSI is negligible (single-digit percentages). Certainly not enough to be a dealbreaker when it comes to choosing which one to use.
And in terms of convenience, cost and manageability, NFS destroys them all.
The only way to really run VMs is fiber channel SAN, local disk (or hardward raid), or iSCSI with 10GbE (on the storage server side). Even iSCSI with 2GbE (2x1GbE bonded) is not speedy enough to support more than 5-10 VMs running concurrently.
Maybe if your VMs have high IO bandwidth needs, but most do not. In the same benchmarks I mentioned above, so long as there isn't a bandwidth constriction (ie: you're interested in IOPS) 1GbE iSCSI/NFS is just as fast as 4/8Gb FC or 10GbE iSCSI/NFS (and probably faster than your backing storage). 100MB/sec is a *lot* of data to sustain and there are few needs for it even in commercial environments, let alone the home server scenario being discussed here.
I'll have to look more deeply in to ZFS as I keep hearing it thrown out there. I should have probably qualified my statement as "thereby supporting live guest migrations".
Well, you still don't need a clustered filesystem for that. Or are you using file-backed virtual disks and using your data storage servers as virtualisation hosts as well ?
If you are, my advise would be to split out your data storage to a separate set of machines. So:
Two data storage servers, software RAID6 (or RAIDZ2 if ZFS), replicating, serving NFS, CIFS, iSCSI, etc. You can set them up as a failover pair as well if you really want HA, but you'll need to either a) be prepared for slow NFS performance (sync mount) or b) get some hardware that will let you make a caching SSD visibile to both hosts.
X virtualisation hosts, mounting the data store via NFS. You will be able to live migrate VMs between them regardless of whether the NFS sever is a pair of HA machines, standalone, or requires manual intervention in case of a failure. Obviously in the latter two cases there is the potential for data loss within the VMs for the data they keep on virtual disks, should the storage server suffer a hard failure - but your need to store important, up-to-the-minute data on those VMs should be fairly low, if it exists at all.
This is basically the system I have at home, and also nearly identical to the production system at my last employer (albeit with a much higher-end, fully redundant NetApp NAS rather than DIYed).
One as the primary, sharing space via NFS for your VMs and whatever else. Throw a couple of SSDs in there for caching.
The second replicating from the first (via ZFS send/receive, or just simple rsync) with snapshotting for backups and regular syncs to some off-site data store for truly irreplaceable data.
This is the setup I use at home, and it sits behind a 3-node VMware cluster, several desktop PCs (one of which boots from the main server over iSCSI), and couple of media PCs.
Other than that, your requirements seem a bit confused. "Cluster filesystem" looks to be a buzzword being thrown out there without any actual need for same. "the cluster filesystem should work well with hosting virtual machines in a high-available fashion thereby supporting guest migrations" is a non-sequitur as neither a cluster filesystem, nor high-availability are a necessity for "guest migration".
What are your key requirements here ? Data reliability is a lot easier (=cheaper) to achieve than high availability, and it's a struggle to see how real high availability could be any sort of requirement in a home server scenario.
...prevents this type of wildcat strike. Airlines are governed under the RLA (Railway Labor Act) which makes wildcatting illegal. The parties must negotiate in good faith before being released to self-help.
Just a point that "negotiations" on this issue have been ongoing for a year, if not longer, and there have been several previous strikes (all with the appropriate notice periods).
This is an apparently spontaneous action by QANTAS _management_. The Unions have been quite responsible about their actions.
It was Joe Sixpack who watched the stocks in his retirement portfolio boom. It was the Jpe Sixpack stockholders and employees of the banks, real estate brokers, home improvement stores, etc... etc...
And it was Joe Sixpack who didn't have the insider knowledge to see what was coming and now owes $400k on a house value at $150k.
Some alternative schools of economics have NOT failed, and in fact have made better predictions of economic events than our current, "mainstream" view of economics that is self-servingly spread by our government and its friends, who always seem to benefit.
And the best thing about economics, is that you can say this no matter *what* happens.
Or, as I prefer: "for every economist, there is an equal and opposite economist".
Hmmm, yeah, because NetWare was only around (or popular) for a week.
It was more the "it was really popular, then nearly disappeared practically overnight" aspect I was focusing on, which is basically what happened to Netware in the late '90s.
The problem with OS/2 in his example is that it was never particularly big to start with, let alone "the biggest thing".
We've had multi-user operating systems on Windows home computers for what, 10 years, and this idiot still does not understand the distinction between program and user data.
From the perspective of simple data separation, it's more like 15 years.
Trying to get a Windows app to jump through a ragged hoop is not a very good example. If you can leave Wine out of your equation, then your point might have meaning.
Given the benchmark was:
I don't have the source for Solaris-x86 Oracle but I CAN run that under Linux (different OS and no source) just fine. Have been able to for years.
Which is clearly going to require some sort of API or ABI translation layer, I'd say the example of IE under WINE is both reasonable, and appropriate.
Slashdot Mirror
So that the people who are good at writing accelerated video code can concentrate on doing that, and the people who are good at writing GUI code can concentrate on that, and neither have to make a half-arsed job of doing the other.
Same reason you (I'm guessing) go down to the baker to buy bread, even though it's trivial to make your own.
(Substitute in some other foodstuff, or clothing, or anything in your life you don't manufacture yourself if you do happen to bake your own bread.)
Given the OP's talk of "both sides", and a game that was for the Apple ][, my guess is he was talking about single-sided, 140k 5.25" floppies.
So still a bit bigger than just the text on this page, but not that far off.
Programmers have lost the _need_ to optimise for space in the general case. There's a difference.
You'll notice people generally no longer keep weeks or months worth of food in (or near) their houses, either. Same principle.
This helps to explain more about what you want to do, but doesn't really help to explain why you want to do it (ie: is it the right solution).
I can tell you about a solution we used to use at a previous job for bulk, cheap storage. This scaled from ~12T at the start of its life through to ~192T at the end, and was built using Linux cLVM and Promise vTrak disk shelves. It briefly also featured GFS2.
The primary objective was lots of cheap, redundant space. Availability, and especially performance, were not high priorities. With that said, it was a (mostly) highly-available solution.
At the base storage layer, were FC-attached Promise vTrak disk shelves. These were attached via two FC fabrics to two "controller nodes". The disks in the shelves were configured in single RAID6s of 15 drives + 1HS and presented to the two controller nodes as a single (multipathed) LUN.
These LUNs were aggregated into a single logical storage pool using Clustered LVM, then redivided into LVs and controlled by the automounter.
This system was used as an archival storage target for image files. It originally ran some proprietary software which would "receive" collections of image files, then derive some metadata about that collection and store them together. At the end of each day a consolidation process would create an appropriately-sized LV and move the last 24 hours' worth of images onto it. Initially, since the "receiving" process was done on the controller nodes, GFS2 was used so the load could be balanced. As the load grew, however, we decided to split off the receiving function onto dedicated machines, and with that the need for load-balancing and a clustered FS disappeared. So, we switched to ext3, reconfigured the controller nodes in an active/passive failover using heartbeat, and shared the storage to the new receiving nodes via NFS (with some SSH scripts to handle the daily LV creation on the controller nodes). Whenever a disk shelf would fill up, we'd simply buy another one, stuff it full of COTS drives, configure it up as a big RAID6, present the LUN to the controllers units and add it to the VG. With that system we went from a single shelf and 12T all the way up to eight shelves and ~192T, with something like 800 LVs averaging around 200G each.
The weakness in this design is, obviously, the single set of disks on the back end that represent a SPOF. However, since the images were regularly rolled off to tape, and both the RTO and RPO were relatively high, it was considered a reasonable tradeoff.
When originally conceived, Linux RAID in the CentOS of the day (4.something) was relatively immature, and in particular it lacked the bitmap capability present now. As such, the decision was made to go with the hardware RAID in the vTraks. Were I to do it again, I would use Linux software RAID, though this would necessitate some slightly more involved failover scripting to re-assemble the RAID arrays on the second host as part of a failover event. Obviously the array couldn't be active on both hosts simultaneously (though there was no real need for that to be possible anyway).
Additionally, the storage bus in our case was FC, which makes sharing physical devices to multiple servers trivial. However, anything capable of having multiple hosts (eg: SAS) would work just as well.
A second, similar, project was to have a more highly-available image store, with no SPOFs. For that we used pairs of servers, Linux software RAID and LVM on SAS shelves (for sufficient spindles), each shelf dedi
Not to put too fine a point on it, but VMFS is a cluster filesystem and runs VMs exceptionally well.
Also, NetApp (and EMC, and others) have demonstrated multiple times that the performance difference between FC, NFS and iSCSI is negligible (single-digit percentages). Certainly not enough to be a dealbreaker when it comes to choosing which one to use.
And in terms of convenience, cost and manageability, NFS destroys them all.
Maybe if your VMs have high IO bandwidth needs, but most do not. In the same benchmarks I mentioned above, so long as there isn't a bandwidth constriction (ie: you're interested in IOPS) 1GbE iSCSI/NFS is just as fast as 4/8Gb FC or 10GbE iSCSI/NFS (and probably faster than your backing storage). 100MB/sec is a *lot* of data to sustain and there are few needs for it even in commercial environments, let alone the home server scenario being discussed here.
Well, you still don't need a clustered filesystem for that. Or are you using file-backed virtual disks and using your data storage servers as virtualisation hosts as well ?
If you are, my advise would be to split out your data storage to a separate set of machines. So:
Two data storage servers, software RAID6 (or RAIDZ2 if ZFS), replicating, serving NFS, CIFS, iSCSI, etc. You can set them up as a failover pair as well if you really want HA, but you'll need to either a) be prepared for slow NFS performance (sync mount) or b) get some hardware that will let you make a caching SSD visibile to both hosts.
X virtualisation hosts, mounting the data store via NFS. You will be able to live migrate VMs between them regardless of whether the NFS sever is a pair of HA machines, standalone, or requires manual intervention in case of a failure. Obviously in the latter two cases there is the potential for data loss within the VMs for the data they keep on virtual disks, should the storage server suffer a hard failure - but your need to store important, up-to-the-minute data on those VMs should be fairly low, if it exists at all.
This is basically the system I have at home, and also nearly identical to the production system at my last employer (albeit with a much higher-end, fully redundant NetApp NAS rather than DIYed).
One as the primary, sharing space via NFS for your VMs and whatever else. Throw a couple of SSDs in there for caching.
The second replicating from the first (via ZFS send/receive, or just simple rsync) with snapshotting for backups and regular syncs to some off-site data store for truly irreplaceable data.
This is the setup I use at home, and it sits behind a 3-node VMware cluster, several desktop PCs (one of which boots from the main server over iSCSI), and couple of media PCs.
Other than that, your requirements seem a bit confused. "Cluster filesystem" looks to be a buzzword being thrown out there without any actual need for same. "the cluster filesystem should work well with hosting virtual machines in a high-available fashion thereby supporting guest migrations" is a non-sequitur as neither a cluster filesystem, nor high-availability are a necessity for "guest migration".
What are your key requirements here ? Data reliability is a lot easier (=cheaper) to achieve than high availability, and it's a struggle to see how real high availability could be any sort of requirement in a home server scenario.
Because they live in the same world you do, and in many cases are responsible for making it work.
And they deserve not to have people like you spit on them every time they drive past.
At which point America might end up with a healthcare system as capable and accessible as every other civilised country. The horror !
How are the military and police funded in your world ?
This is a standard behaviour for any delays and disruptions to QANTAS operations, regardless of whether it's industrial action or a broken down plane.
Just a point that "negotiations" on this issue have been ongoing for a year, if not longer, and there have been several previous strikes (all with the appropriate notice periods).
This is an apparently spontaneous action by QANTAS _management_. The Unions have been quite responsible about their actions.
On the one hand, current (and immediately previous) QANTAS management has been woeful, and are now merely reaping what they have sown.
One the other, the employees in question are already on a pretty sweet deal, and asking for more is just raw greed.
The difficulty lies not in predicting that a crash is coming, but when it will happen.
Not to mention it's a gimme. All booms eventually bust, it's called the business cycle.
And it was Joe Sixpack who didn't have the insider knowledge to see what was coming and now owes $400k on a house value at $150k.
And the best thing about economics, is that you can say this no matter *what* happens.
Or, as I prefer: "for every economist, there is an equal and opposite economist".
It was more the "it was really popular, then nearly disappeared practically overnight" aspect I was focusing on, which is basically what happened to Netware in the late '90s.
The problem with OS/2 in his example is that it was never particularly big to start with, let alone "the biggest thing".
By all means, please identify the parts of IE that run in kernel space, and that the kernel depends on.
From the perspective of simple data separation, it's more like 15 years.
Given the benchmark was:
Which is clearly going to require some sort of API or ABI translation layer, I'd say the example of IE under WINE is both reasonable, and appropriate.
You need to look long and hard to find unix systems without /bin/sh.
It likely will, as probably would Windows.
The difficulty of writing such a drop-in replacement, however, would be immense (and of highly questionable benefit).
Rubbish. IE is userspace code. Nothing in the Windows kernel has a dependency on IE.
They did. In 1997. With Internet Explorer 4.
Every version of IE since has had the same architecture.
The vast majority of IE *is* a "system library", and has been since IE4.
It was changing IE to a system library that got them into trouble in the first place.