Doing any single RAID level over 12 disks is just foolhardy, including RAID6. Multiple smaller arrays striped together is the way to go (RAID10, RAID50, RAID60,. Or is that multiple stripesets RAID'd together? I always get the terms backwards.
You'll also get better throughput, at the cost of a bit of storage space.
You don't need smartmontools with 3Ware cards. The controllers monitor SMART health themselves, and can do auto-verify in the background, on a schedule that you choose. And the cards will send e-mail alerts whenever they repair a bad sector, lose cache coherency, lose a drive, etc.
The web-based 3dm2 management tool has to be one of the easiest RAID management tools to use (of the ones I've used, anyway, which include LSI, 3Ware, IBM, and Dell).
No OS support required, it's all handled at the hardware level.
3Ware cards are infinitely better than LSI MegaRAID crap. Especially when it comes to the management features. The web-based 3dm2 may have its shortcomings, but at least it's usable, understandable, and simple without skimping on features. The megacli/megamon stuff from LSI most definitely is not.
3Ware also had native SATA chipsets long before LSI. Most of the LSI SATA RAID controllers used PATA bridge chips, so you didn't actually get any of the good SATA features like over 100 MBps throughput, hot-plug/hot-swap, NCQ, etc.
3Ware also employs FreeBSD and Linux developers, so there's proper driver support, with techs/support that know how the hardware works and how the open-source drivers work.
We started with LSI MegaRAID controllers, and quickly returned them all for 3Ware. We were originally thinking Areca, but our local suppliers couldn't order them in quick enough, so we went with 3Ware. Haven't looked back.
The truly unfortunate (ironic) thing: LSI just bought AMCC, who currently owns 3Ware.:( Here's hoping that the 3Ware engineering team takes over from LSI... otherwise...
Don't bother using ZFS on Linux, as it goes through the FUSE framework, absolutely killing throughput. It's okay for prototyping and testing, but I wouldn't use it in production.
If you know Linux, then give FreeBSD a try. It'll be less of a culture shock than Solaris.
As one of the techs behind the solution linked to on FreeBSD forums, I just wanted to chime in with a "definitely give ZFS a try". Whether you run it on FreeBSD or Solaris (or even Linux via FUSE if you don't really care about the throughput) doesn't really matter.
You don't even need to use RAID controllers like we did (although the individual drives are configured as "Single Drive" "arrays" so non of the actual RAID hardware was used). Just throw in some good SATA controllers into PCIe or PCI-X slots and you're set. (We used RAID controllers for the management features, and extra level of cache.)
ZFS takes care of the RAID setup (RAID1, RAID5, RAID6, with built-in striping across arrays/vdevs), detects data corruption via end-to-end checksumming, can alert you to when a drive has issues (and tell you which one), gives you in-filesystems snapshots, filesystem compression, and a whole bunch more.
Add in rsync for network transfers (the built-in snapshot send/receive feature still needs a bit of work) and you have a very nice backup setup, even across redundant servers.
Add iSCSI and you have a very nice SAN setup.
Add Samba or NFS and you have a very nice NAS setup.
There's even support for thin-provisioning (create a volume that's 500 GB in size, but only give it 100 GB of actual disk space) making it ideal for virtualisation setups.
And you can "stack" storage boxes to create a virtually infinite storage setup (create a pair of storage servers using disks and ZFS, export a single iSCSI volume -- then use those iSCSI exports on a third server to create a storage pool -- when you need more storage, just add another pair of storage servers).
You can also replace the drives with larger drives and get (almost) instant access to the extra space.
Finally, since it's a copy-on-write, transactional filesystem, you don't lose any write speed, since you always write out new files; which also eliminates the "RAID5/6 write hole".
Once you start using ZFS and pooled storage, you'll find the whole Linux storage stack (disks -> md -> lvm pv -> lvm vg -> lvm lv -> filesystem) to be unbelievable unwieldy and wonder how you ever managed TBs of disk before.
They're not. Of course, it depends on the game--many games don't translate well to 3D, and the retro charm of 8bit is always nice--but let's not kid ourselves that "immersion" (yeah, yeah) is a part of game enjoyment. You can't make an 8bit WoW, now would you want to try, but a 3D pacman isn't going to translate well either.
Hey! 3D Pac-man was a hell of a lot of fun in the arcade. Especially being able to jump over ghosts and some walls! Really added a nice twist to an old favourite. Don't knock it until you've tried it.:)
Also if they want to ask "We Rent Movies, So Why Not Textbooks?" then why not also have public libraries of movies, as its worked for hundreds of years for books. The libraries buy the books and our taxes pay for the libraries so they can buy movies (and music) the same way. After all books don't earn their living from libraries as books are still also sold to fans of the books, so its not as if libraries are the only source of income for books.
This is already being done. Our local library has a large selection of VHS and DVD movies available for signing out, along with an even larger selection of CDs. And these aren't just ancient, crappy releases either, around half of the items are current releases.
It would be so much nicer if the consumer industry would abandon optical disks for something like a flash-based "cartridge". SD cards are a bit too small, physically, for this purpose, but the technology behind them is pretty much perfect.
Come out with a physical form-factor around the size of a floppy diskette or maybe a GameBoy game: big enough to not lose too easily, small enough to not hog a lot of shelf space, lots of room internally to store flash chips. Put a controller onto the diskette/cartridge, and use a standardised physical interface.
Voila! A future-proof storage medium. Need more space? Just put more (and/or higher-density) flash chips inside. The hardware interface doesn't have to change. The reader hardware doesn't have to change. And you don't have to worry about wear-levelling and what-not, as it's a read-only setup.
You could even reserve a little section of the storage space to include the media codec, so you could "upgrade" the player when you play the media. No more format wars!! Turns the player into a "dumb" device, with a general-purpose CPU/DSP/whatever, where all the smarts needed to play the media is included with the media. (Isn't that how Blu-Ray works?)
Learn from the harddrive market: a single IDE connector/controller can be used for drives as small as 10 MB and as large as 500 GB (I think that was the largest IDE drive). A single SATA connector/controller can be used for drives as small as 80 GB and as large as 2 TB.
Standardise the physical interface... and you can change the innards as needed.
This could be used for pretty much any kind of media: audio/music, video, applications, you name it.
Of course, the RIAA/MPAA would have a coronary if this ever happened. No more forced re-purchasing of your entire library (Beta -> VHS -> DVD -> HD-DVD -> Blu-Ray -> whatever; vinyl -> 8-track -> cassette -> MD -> CD -> DVD -> whatever). No more forced upgrades of your home theatre equipment to read the latest optical format. Etc.
Or, use a zvol instead of a full-fledged zfs filesystem. You get all the benefits of the storage pool and easy snapshotting, as well as all the benefits of a "bare blockdevice".
ZFS is layered. It's just layered differently. People need to stop thinking of ZFS as *just* a filesystem. It's a full-fledged storage management system. Personally, I think they chose the wrong name -- it should have been ZSMS (Zettabyte Storage Management System), where ZFS (the filesystem) is just the top layer.
In a nutshell:
At the bottom is the vdev. Those are joined together into the pool.
Then there's the volume manager.
Then there's the filesystem (ZFS) itself.
Instead of a massive stack with tonnes of layers that don't understand each other, there's a shorter stack where the layers know about each other, and work together to provide end-to-end reliability.
Think of the difference between the OSI network stack and the TCP/IP stack. One could also call the entire TCP/IP stack a "massive layering violation".
And when your house burns down, turning that nice little NAS box into charcoal, what then?;)
Or, if someone breaks into the house and takes the NAS box along with the computer?
There's a reason for having off-site backups, whether it be a box discs in the safe deposit box, a removable drive at a friend's, stuck into your gmail account, or somewhere "in the cloud". Just so long as it's not right next to the device it's backing up.:)
$25,000 for 10 TB of disk storage? That's outrageously expensive. Considering it's "in the cloud", you'll be limited to whatever your Internet connection is for throughput. Unless you have a couple of massive fibre links, with very few hops between you and "the cloud", performance will suck.
For $10,000 CDN ($8,000 US) you can put together a 10 TB storage box using multiple 3Ware RAID controllers (or even just plain SATA controllers), 24x 500 GB SATA drives, a 5U rackmount case, redundant power supplies, and quad-port NIC, using FreeBSD or OpenSolaris with ZFS.
Export zvols using iSCSI to use it as a SAN, or export zfs filesystems using CIFS or NFS to use it as NAS.
Need more space, then just replace the 500 GB drives with larger ones as needed, and the zpool expands out to use the extra space.
Need off-site storage? Build two servers, and configure rsync to run everyday. If they ever fix the performance issues, you could even use the zfs snapshot send/recv to keep the servers in sync. It'd still be cheaper than their $25,000.
And the performance will be a hell of a lot better.
BTW, what OSes with BSD support also have a package manager? I take it from your comment that FreeBSD does, but do any of the others? I've used both NetBSD and OpenBSD, and neither had a package manager, just a "ports" automatic source-compiling system.
FreeBSD has pkg_add, including remote fetching, dependency tracking, and installation.
OpenBSD has a package manager, as well, and they prefer it if you install using packages. The ports tree is only for when you need something not offered by default in the packages. I think it's also called pkg_add, but don't know for sure.
I've never used NetBSD, so I don't know if there's a package installation tool or not.
Lack of more cutting edge virtualization software. At this point, Qemu is the only real option. Right now, you have to jump through hoops to get a FreeBSD *guest* under Zen, so being a Zen host is probably out of the question.
It's Xen, and you can run FreeBSD in an HVM just fine, no hoop-jumping required. Works for 32-bit and 64-bit guests, all the way back to 6.2 (earliest version I used in my testing).
It's true that PV support for FreeBSD guests is lacking, but that's not to say you can't run FreeBSD on Xen.
It all depends on what you do with it, and how much time you put into it. If you are willing to put in the effort up front, to plan out how the pool is created, to tune the OS and ZFS, and to test things... then you can get a very performant, very stable system.
We have a pair of storage boxes running FreeBSD 7.1-RELEASE with / and/usr on CompactFlash on one and USB sticks on the other (2x 2GB mirrored using geom_mirror), with/var,/tmp,/usr/local,/usr/src,/usr/obj,/usr/ports,/usr/ports/distfiles, and/storage/backup as ZFS filesystems. Some have compression enabled (gzip-9 or lzjb), some have atime enabled, some have snapshots created daily.
They're 5U rackmounts with 2x dual-core Opterons, 8 GB of ECC SDRAM, a 3Ware 9550SXU PCI-X RAID controller, a 3Ware 96650SE PCIe RAID controller, a 1350-watt 4-way redundant power supply, 4-port Intel gigabit NIC, as 24x 500 GB SATA harddrives.
The zpool is configured using 3x 8-drive raidz2 vdevs, giving a total usable space of just over 10 TB.
The primary server does an rsync-based backup of 90 FreeBSD and Linux servers every night. Takes just under 5 hours. According to MRTG, which polls the 32-bit storage counters every 60 seconds, it sustains 80 MBytes/sec for the bulk of the 5 hours. We can't get the 64-bit storage counters to work, so it's possible the counters are looping. We're also limited by the remote site bandwidth, as most sites are ADSL with ~768 Kbps uploads.
Other than some problems with the initial configuration of the server (don't use more than 9 drives in a raidz2 vdev), and with the initial "trial and error" for the kmem, ARC, and network tuning, it's been running very smoothly. We have daily backups going back four months, and only used 2 TB so far.
At this rate, we'll be able to keep a full year of daily backups without running out of space. And if we start to get low on space, we just swap out the drives in one of the raidz2 vdevs for larger ones, and continue one with the extra space.
We're really looking forward to the changes that FreeBSD 8.0 will bring, as it includes ZFS v13, removes the 2 GB kmem barrier, and a bunch of other unrelated performance enhancements.
Just because some people have run into some problems with ZFS, doesn't mean it's crap for everyone. You just need to run the 64-bit version, with lots of RAM, or be willing to test and tune to find the right settings for your 32-bit setup.
Once you've used a pooled storage system, or any filesystem with inline snapshots, you won't be able to use LVM anymore. LVM is okay, and has it's uses, but the snapshots system for LVM is crap, more crap, and worse crap.
Can't remember the exact name (Pac-3D or Pac-World or Pac-Maze, something like that), but there was a pseudo-3D version of Pac-Man where you could jump over the ghosts and some of the walls of the maze. Was quite fun, although slow to play in the arcades.
Don't think a first-person viewpoint would work, but an updated 3D engine and higher framerates could make it a lot more fun.:)
Don't know which "Canada" you live in, but you're full of shit.
Rogers doesn't charge for incoming text messages, never has (at least in the 8 years I've had a phone with them). Incoming text messages are included in all phone plans, even the $10/mth ones. You only pay for outgoing text messages, with prices ranging from $0.10/ea (no plan) to $7 for 2500 (and can be included in an Essentials pack dropping it even further).
Can't comment on their pay-as-you-go service, as I don't know anyone with a Rogers pay-as-you-go. Seems all the p-a-y-g users here in BC are Telus.
Don't know about Bell. Don't know anyone who has a Bell plan/phone.
Telus was the first to charge for incoming text messages, and that was only on the pay-as-you-go plans. Regular phone plans (contracts) include free incoming text messages.
Koodo (part of Telus) doesn't charge for incoming test messages either.
Don't know about Fido (part of Rogers).
Nor about Virgin (don't know which of the Big Three they piggy-back on).
Generally I don't get all the fuss about ZFS... Okay, it's cool that we'll need more energy to reach it's limits than needed to boil this oceans...
smb/nfs/iscsi support integrated, Volume AND partition manager.
Yes, that's cool... But why does it need to be *in* the filesystem???
Snapshots and pooled storage.
LVM snapshots are okay for doing online backups, as you only have to stop access for a second or two (freeze filesystem, create snapshot, unfreeze filesystem, do backup using snapshot, destroy snapshot). But there's a whole lot of administrative overhead involved (you have to leave unallocated space in the VG for the snapshots, you have to figure out how much data will change during the backup in order to size the snapshot).
And replacing drives in a VG is a headache, especially if you are near capacity in the VG (where do you move the data to, in order to replace the drive?).
Then there's all the extra administrative pain of trying to plan out any kind of software RAID setup, and layering LVM on top, especially when it comes time to replace a failed drive.
I used to think LVM was the neatest thing ever... until I had to replace a drive. Suddenly, it wasn't all that great.
I'm not expert but AFAIK we've got LVM for volume management, and we can run any filesystem ontop of LVM... I think the idea of having volume management separate from filesystem sounds like a good idea, as it would enable you to used different filesystems for different purposes...
With ZFS, the volume management isn't "in" the filesystem. There's several layers to the ZFS system, just not the same layering that currently exists in the hardware/md/lvm/fs stack. The filesystem is just the top layer.
For example, you can create zvols and use UFS instead of ZFS. You can even create zvols to use for swap space. You can probably put other FS on top of zvols, although I've never tried. You still get all the volume management features and pooled storage capabilities, everything you can with the zpool(8) command is still available.
All the sports games now are just league simulators. All the racing games are just driving sims (Gran Tourismo is the worst for this). All the flying games are just flight sims with machines guns.
Oh, for the days when you could race without ever taking your thumb off the accelerator, when you could strategically bounce around corners, and you could drive your opponents into walls.
Or for the days when you could play sports games where the players had super-powers, or could be knocked down from 10' away, or could steam roll through defenders like a hot knife through butter?
Or you could fly a fighter jet without worrying about which position the ailerons were in, or whether or not the fuselage could withstand the Gs while doing nose-dives and pull-ups to get away? Or a space game where you don't have to worry about weightless drift, and firing the retro-rockets to slow down, just whip the joystick around and fire?
Why all the demand for "realism"? Doesn't anyone have an imagination anymore? Can't people "suspend their disbelief" for a few hours?
Give me realistic graphics, sure. But give me a GAME, damnit!
Then it must have been in the very first version for Windows only. I've used WordPerfect 6.x, 7.x, 8.x, 9.x, 11.x, and 13.x, and it doesn't do what you describe. Never used 5.x for Windows, though.
WP 7 was available for Unix (including Linux) as a native X11 application. The interface was horrid, though.
WP 9 for Linux was just the Windows app bundled with WINE, and was too alien to use with any Linux desktop (all the dialogs were wrong, especially the print dialog).
If you don't have the reveal codes screen up, then the cursor moves normally.
It's *ONLY* while the reveal codes screen is up, that the cursor seems to move strange... but only in the top-half of the screen. If you want the bottom, in the reveal codes pane, the cursor is moving correctly. And why would you have the reveal codes pane up if you weren't looking at it?
WordPerfect 6.x for Windows 3.x was a pig. It worked, barely, but it wasn't pleasant to use in any way, shape, or form. You could tell this was their first GUI app.
WordPerfect 7.x for Windows 95 was better, but still didn't feel like a native Win95 app. There wasn't much integration between WP and QP, either. The database software was horrible to use unless you had a degree in DBs (Borland something or other). Their PIM app (Corel Central) was just horrible, horrible, horrible. There was also a bunch of extra, useless software on the CD, like the DAD. At least there were more fonts than anyone could ever think of using in several lifetimes.
WordPerfect 8.x was decent, and things started coming together. They replaced the PIM app, although it wasn't much better.
But it was WordPerfect 2000 (9.x) where things started to shine. Haven't had to use another wordprocessor, spreadsheet, or presentations app since. (At least on Windows.) All the functionality and power of WP 5.x is there, but in a nice GUI environment that doesn't get in your way. The apps are nicely integrated together and work like a real office suite. And there's no PIM app, although it does integrate nicely with various PIM/contact managers.
I've tried WordPerfect 10, 11, and X3, but keep coming back to 9.x. IMO, that's the pinnacle of office suite technology. Nothing MS, IBM, or Sun have comes anywhere near it. X3 is nice, though, and if you had to buy an office suite, I'd recommend it over anything else.
Uhm, you do realise that Windows NT was designed from the ground-up to be a multi-user desktop and server system? And that all current versions of Windows are based upon that codebase, and not on Windows 3.x/9x or DOS? Such that none of what you wrote has applied for over 8 years?
Slashdot Mirror
Doing any single RAID level over 12 disks is just foolhardy, including RAID6. Multiple smaller arrays striped together is the way to go (RAID10, RAID50, RAID60,. Or is that multiple stripesets RAID'd together? I always get the terms backwards.
You'll also get better throughput, at the cost of a bit of storage space.
You don't need smartmontools with 3Ware cards. The controllers monitor SMART health themselves, and can do auto-verify in the background, on a schedule that you choose. And the cards will send e-mail alerts whenever they repair a bad sector, lose cache coherency, lose a drive, etc.
The web-based 3dm2 management tool has to be one of the easiest RAID management tools to use (of the ones I've used, anyway, which include LSI, 3Ware, IBM, and Dell).
No OS support required, it's all handled at the hardware level.
3Ware cards are infinitely better than LSI MegaRAID crap. Especially when it comes to the management features. The web-based 3dm2 may have its shortcomings, but at least it's usable, understandable, and simple without skimping on features. The megacli/megamon stuff from LSI most definitely is not.
3Ware also had native SATA chipsets long before LSI. Most of the LSI SATA RAID controllers used PATA bridge chips, so you didn't actually get any of the good SATA features like over 100 MBps throughput, hot-plug/hot-swap, NCQ, etc.
3Ware also employs FreeBSD and Linux developers, so there's proper driver support, with techs/support that know how the hardware works and how the open-source drivers work.
We started with LSI MegaRAID controllers, and quickly returned them all for 3Ware. We were originally thinking Areca, but our local suppliers couldn't order them in quick enough, so we went with 3Ware. Haven't looked back.
The truly unfortunate (ironic) thing: LSI just bought AMCC, who currently owns 3Ware. :( Here's hoping that the 3Ware engineering team takes over from LSI ... otherwise ...
Don't bother using ZFS on Linux, as it goes through the FUSE framework, absolutely killing throughput. It's okay for prototyping and testing, but I wouldn't use it in production.
If you know Linux, then give FreeBSD a try. It'll be less of a culture shock than Solaris.
As one of the techs behind the solution linked to on FreeBSD forums, I just wanted to chime in with a "definitely give ZFS a try". Whether you run it on FreeBSD or Solaris (or even Linux via FUSE if you don't really care about the throughput) doesn't really matter.
You don't even need to use RAID controllers like we did (although the individual drives are configured as "Single Drive" "arrays" so non of the actual RAID hardware was used). Just throw in some good SATA controllers into PCIe or PCI-X slots and you're set. (We used RAID controllers for the management features, and extra level of cache.)
ZFS takes care of the RAID setup (RAID1, RAID5, RAID6, with built-in striping across arrays/vdevs), detects data corruption via end-to-end checksumming, can alert you to when a drive has issues (and tell you which one), gives you in-filesystems snapshots, filesystem compression, and a whole bunch more.
Add in rsync for network transfers (the built-in snapshot send/receive feature still needs a bit of work) and you have a very nice backup setup, even across redundant servers.
Add iSCSI and you have a very nice SAN setup.
Add Samba or NFS and you have a very nice NAS setup.
There's even support for thin-provisioning (create a volume that's 500 GB in size, but only give it 100 GB of actual disk space) making it ideal for virtualisation setups.
And you can "stack" storage boxes to create a virtually infinite storage setup (create a pair of storage servers using disks and ZFS, export a single iSCSI volume -- then use those iSCSI exports on a third server to create a storage pool -- when you need more storage, just add another pair of storage servers).
You can also replace the drives with larger drives and get (almost) instant access to the extra space.
Finally, since it's a copy-on-write, transactional filesystem, you don't lose any write speed, since you always write out new files; which also eliminates the "RAID5/6 write hole".
Once you start using ZFS and pooled storage, you'll find the whole Linux storage stack (disks -> md -> lvm pv -> lvm vg -> lvm lv -> filesystem) to be unbelievable unwieldy and wonder how you ever managed TBs of disk before.
They're not. Of course, it depends on the game--many games don't translate well to 3D, and the retro charm of 8bit is always nice--but let's not kid ourselves that "immersion" (yeah, yeah) is a part of game enjoyment. You can't make an 8bit WoW, now would you want to try, but a 3D pacman isn't going to translate well either.
Hey! 3D Pac-man was a hell of a lot of fun in the arcade. Especially being able to jump over ghosts and some walls! Really added a nice twist to an old favourite. Don't knock it until you've tried it. :)
Also if they want to ask "We Rent Movies, So Why Not Textbooks?" then why not also have public libraries of movies, as its worked for hundreds of years for books. The libraries buy the books and our taxes pay for the libraries so they can buy movies (and music) the same way. After all books don't earn their living from libraries as books are still also sold to fans of the books, so its not as if libraries are the only source of income for books.
This is already being done. Our local library has a large selection of VHS and DVD movies available for signing out, along with an even larger selection of CDs. And these aren't just ancient, crappy releases either, around half of the items are current releases.
It would be so much nicer if the consumer industry would abandon optical disks for something like a flash-based "cartridge". SD cards are a bit too small, physically, for this purpose, but the technology behind them is pretty much perfect.
Come out with a physical form-factor around the size of a floppy diskette or maybe a GameBoy game: big enough to not lose too easily, small enough to not hog a lot of shelf space, lots of room internally to store flash chips. Put a controller onto the diskette/cartridge, and use a standardised physical interface.
Voila! A future-proof storage medium. Need more space? Just put more (and/or higher-density) flash chips inside. The hardware interface doesn't have to change. The reader hardware doesn't have to change. And you don't have to worry about wear-levelling and what-not, as it's a read-only setup.
You could even reserve a little section of the storage space to include the media codec, so you could "upgrade" the player when you play the media. No more format wars!! Turns the player into a "dumb" device, with a general-purpose CPU/DSP/whatever, where all the smarts needed to play the media is included with the media. (Isn't that how Blu-Ray works?)
Learn from the harddrive market: a single IDE connector/controller can be used for drives as small as 10 MB and as large as 500 GB (I think that was the largest IDE drive). A single SATA connector/controller can be used for drives as small as 80 GB and as large as 2 TB.
Standardise the physical interface ... and you can change the innards as needed.
This could be used for pretty much any kind of media: audio/music, video, applications, you name it.
Of course, the RIAA/MPAA would have a coronary if this ever happened. No more forced re-purchasing of your entire library (Beta -> VHS -> DVD -> HD-DVD -> Blu-Ray -> whatever; vinyl -> 8-track -> cassette -> MD -> CD -> DVD -> whatever). No more forced upgrades of your home theatre equipment to read the latest optical format. Etc.
But wouldn't that be an end-user nirvana!?
Until someone posts on the Internet how to beat the "mileage indicator" built into the battery. :)
Same way you can get unlimited km on a rental vehicle by disconnecting the odometer.
Or, use a zvol instead of a full-fledged zfs filesystem. You get all the benefits of the storage pool and easy snapshotting, as well as all the benefits of a "bare blockdevice".
ZFS is layered. It's just layered differently. People need to stop thinking of ZFS as *just* a filesystem. It's a full-fledged storage management system. Personally, I think they chose the wrong name -- it should have been ZSMS (Zettabyte Storage Management System), where ZFS (the filesystem) is just the top layer.
In a nutshell:
At the bottom is the vdev. Those are joined together into the pool.
Then there's the volume manager.
Then there's the filesystem (ZFS) itself.
Instead of a massive stack with tonnes of layers that don't understand each other, there's a shorter stack where the layers know about each other, and work together to provide end-to-end reliability.
Think of the difference between the OSI network stack and the TCP/IP stack. One could also call the entire TCP/IP stack a "massive layering violation".
And when your house burns down, turning that nice little NAS box into charcoal, what then? ;)
Or, if someone breaks into the house and takes the NAS box along with the computer?
There's a reason for having off-site backups, whether it be a box discs in the safe deposit box, a removable drive at a friend's, stuck into your gmail account, or somewhere "in the cloud". Just so long as it's not right next to the device it's backing up. :)
$25,000 for 10 TB of disk storage? That's outrageously expensive. Considering it's "in the cloud", you'll be limited to whatever your Internet connection is for throughput. Unless you have a couple of massive fibre links, with very few hops between you and "the cloud", performance will suck.
For $10,000 CDN ($8,000 US) you can put together a 10 TB storage box using multiple 3Ware RAID controllers (or even just plain SATA controllers), 24x 500 GB SATA drives, a 5U rackmount case, redundant power supplies, and quad-port NIC, using FreeBSD or OpenSolaris with ZFS.
Export zvols using iSCSI to use it as a SAN, or export zfs filesystems using CIFS or NFS to use it as NAS.
Need more space, then just replace the 500 GB drives with larger ones as needed, and the zpool expands out to use the extra space.
Need off-site storage? Build two servers, and configure rsync to run everyday. If they ever fix the performance issues, you could even use the zfs snapshot send/recv to keep the servers in sync. It'd still be cheaper than their $25,000.
And the performance will be a hell of a lot better.
BTW, what OSes with BSD support also have a package manager? I take it from your comment that FreeBSD does, but do any of the others? I've used both NetBSD and OpenBSD, and neither had a package manager, just a "ports" automatic source-compiling system.
FreeBSD has pkg_add, including remote fetching, dependency tracking, and installation.
OpenBSD has a package manager, as well, and they prefer it if you install using packages. The ports tree is only for when you need something not offered by default in the packages. I think it's also called pkg_add, but don't know for sure.
I've never used NetBSD, so I don't know if there's a package installation tool or not.
It's Xen, and you can run FreeBSD in an HVM just fine, no hoop-jumping required. Works for 32-bit and 64-bit guests, all the way back to 6.2 (earliest version I used in my testing).
It's true that PV support for FreeBSD guests is lacking, but that's not to say you can't run FreeBSD on Xen.
It all depends on what you do with it, and how much time you put into it. If you are willing to put in the effort up front, to plan out how the pool is created, to tune the OS and ZFS, and to test things ... then you can get a very performant, very stable system.
We have a pair of storage boxes running FreeBSD 7.1-RELEASE with / and /usr on CompactFlash on one and USB sticks on the other (2x 2GB mirrored using geom_mirror), with /var, /tmp, /usr/local, /usr/src, /usr/obj, /usr/ports, /usr/ports/distfiles, and /storage/backup as ZFS filesystems. Some have compression enabled (gzip-9 or lzjb), some have atime enabled, some have snapshots created daily.
They're 5U rackmounts with 2x dual-core Opterons, 8 GB of ECC SDRAM, a 3Ware 9550SXU PCI-X RAID controller, a 3Ware 96650SE PCIe RAID controller, a 1350-watt 4-way redundant power supply, 4-port Intel gigabit NIC, as 24x 500 GB SATA harddrives.
The zpool is configured using 3x 8-drive raidz2 vdevs, giving a total usable space of just over 10 TB.
The primary server does an rsync-based backup of 90 FreeBSD and Linux servers every night. Takes just under 5 hours. According to MRTG, which polls the 32-bit storage counters every 60 seconds, it sustains 80 MBytes/sec for the bulk of the 5 hours. We can't get the 64-bit storage counters to work, so it's possible the counters are looping. We're also limited by the remote site bandwidth, as most sites are ADSL with ~768 Kbps uploads.
Other than some problems with the initial configuration of the server (don't use more than 9 drives in a raidz2 vdev), and with the initial "trial and error" for the kmem, ARC, and network tuning, it's been running very smoothly. We have daily backups going back four months, and only used 2 TB so far.
At this rate, we'll be able to keep a full year of daily backups without running out of space. And if we start to get low on space, we just swap out the drives in one of the raidz2 vdevs for larger ones, and continue one with the extra space.
We're really looking forward to the changes that FreeBSD 8.0 will bring, as it includes ZFS v13, removes the 2 GB kmem barrier, and a bunch of other unrelated performance enhancements.
Just because some people have run into some problems with ZFS, doesn't mean it's crap for everyone. You just need to run the 64-bit version, with lots of RAM, or be willing to test and tune to find the right settings for your 32-bit setup.
Once you've used a pooled storage system, or any filesystem with inline snapshots, you won't be able to use LVM anymore. LVM is okay, and has it's uses, but the snapshots system for LVM is crap, more crap, and worse crap.
Can't remember the exact name (Pac-3D or Pac-World or Pac-Maze, something like that), but there was a pseudo-3D version of Pac-Man where you could jump over the ghosts and some of the walls of the maze. Was quite fun, although slow to play in the arcades.
Don't think a first-person viewpoint would work, but an updated 3D engine and higher framerates could make it a lot more fun. :)
Don't know which "Canada" you live in, but you're full of shit.
Rogers doesn't charge for incoming text messages, never has (at least in the 8 years I've had a phone with them). Incoming text messages are included in all phone plans, even the $10/mth ones. You only pay for outgoing text messages, with prices ranging from $0.10/ea (no plan) to $7 for 2500 (and can be included in an Essentials pack dropping it even further).
Can't comment on their pay-as-you-go service, as I don't know anyone with a Rogers pay-as-you-go. Seems all the p-a-y-g users here in BC are Telus.
Don't know about Bell. Don't know anyone who has a Bell plan/phone.
Telus was the first to charge for incoming text messages, and that was only on the pay-as-you-go plans. Regular phone plans (contracts) include free incoming text messages.
Koodo (part of Telus) doesn't charge for incoming test messages either.
Don't know about Fido (part of Rogers).
Nor about Virgin (don't know which of the Big Three they piggy-back on).
Generally I don't get all the fuss about ZFS... Okay, it's cool that we'll need more energy to reach it's limits than needed to boil this oceans...
smb/nfs/iscsi support integrated, Volume AND partition manager.
Yes, that's cool... But why does it need to be *in* the filesystem???
Snapshots and pooled storage.
LVM snapshots are okay for doing online backups, as you only have to stop access for a second or two (freeze filesystem, create snapshot, unfreeze filesystem, do backup using snapshot, destroy snapshot). But there's a whole lot of administrative overhead involved (you have to leave unallocated space in the VG for the snapshots, you have to figure out how much data will change during the backup in order to size the snapshot).
And replacing drives in a VG is a headache, especially if you are near capacity in the VG (where do you move the data to, in order to replace the drive?).
Then there's all the extra administrative pain of trying to plan out any kind of software RAID setup, and layering LVM on top, especially when it comes time to replace a failed drive.
I used to think LVM was the neatest thing ever ... until I had to replace a drive. Suddenly, it wasn't all that great.
I'm not expert but AFAIK we've got LVM for volume management, and we can run any filesystem ontop of LVM... I think the idea of having volume management separate from filesystem sounds like a good idea, as it would enable you to used different filesystems for different purposes...
With ZFS, the volume management isn't "in" the filesystem. There's several layers to the ZFS system, just not the same layering that currently exists in the hardware/md/lvm/fs stack. The filesystem is just the top layer.
For example, you can create zvols and use UFS instead of ZFS. You can even create zvols to use for swap space. You can probably put other FS on top of zvols, although I've never tried. You still get all the volume management features and pooled storage capabilities, everything you can with the zpool(8) command is still available.
Urgh, I wish games would go back to being games.
All the sports games now are just league simulators. All the racing games are just driving sims (Gran Tourismo is the worst for this). All the flying games are just flight sims with machines guns.
Oh, for the days when you could race without ever taking your thumb off the accelerator, when you could strategically bounce around corners, and you could drive your opponents into walls.
Or for the days when you could play sports games where the players had super-powers, or could be knocked down from 10' away, or could steam roll through defenders like a hot knife through butter?
Or you could fly a fighter jet without worrying about which position the ailerons were in, or whether or not the fuselage could withstand the Gs while doing nose-dives and pull-ups to get away? Or a space game where you don't have to worry about weightless drift, and firing the retro-rockets to slow down, just whip the joystick around and fire?
Why all the demand for "realism"? Doesn't anyone have an imagination anymore? Can't people "suspend their disbelief" for a few hours?
Give me realistic graphics, sure. But give me a GAME, damnit!
Then it must have been in the very first version for Windows only. I've used WordPerfect 6.x, 7.x, 8.x, 9.x, 11.x, and 13.x, and it doesn't do what you describe. Never used 5.x for Windows, though.
WP 7 was available for Unix (including Linux) as a native X11 application. The interface was horrid, though.
WP 9 for Linux was just the Windows app bundled with WINE, and was too alien to use with any Linux desktop (all the dialogs were wrong, especially the print dialog).
If you don't have the reveal codes screen up, then the cursor moves normally.
... but only in the top-half of the screen. If you want the bottom, in the reveal codes pane, the cursor is moving correctly. And why would you have the reveal codes pane up if you weren't looking at it?
It's *ONLY* while the reveal codes screen is up, that the cursor seems to move strange
WordPerfect 6.x for Windows 3.x was a pig. It worked, barely, but it wasn't pleasant to use in any way, shape, or form. You could tell this was their first GUI app.
WordPerfect 7.x for Windows 95 was better, but still didn't feel like a native Win95 app. There wasn't much integration between WP and QP, either. The database software was horrible to use unless you had a degree in DBs (Borland something or other). Their PIM app (Corel Central) was just horrible, horrible, horrible. There was also a bunch of extra, useless software on the CD, like the DAD. At least there were more fonts than anyone could ever think of using in several lifetimes.
WordPerfect 8.x was decent, and things started coming together. They replaced the PIM app, although it wasn't much better.
But it was WordPerfect 2000 (9.x) where things started to shine. Haven't had to use another wordprocessor, spreadsheet, or presentations app since. (At least on Windows.) All the functionality and power of WP 5.x is there, but in a nice GUI environment that doesn't get in your way. The apps are nicely integrated together and work like a real office suite. And there's no PIM app, although it does integrate nicely with various PIM/contact managers.
I've tried WordPerfect 10, 11, and X3, but keep coming back to 9.x. IMO, that's the pinnacle of office suite technology. Nothing MS, IBM, or Sun have comes anywhere near it. X3 is nice, though, and if you had to buy an office suite, I'd recommend it over anything else.
Uhm, you do realise that Windows NT was designed from the ground-up to be a multi-user desktop and server system? And that all current versions of Windows are based upon that codebase, and not on Windows 3.x/9x or DOS? Such that none of what you wrote has applied for over 8 years?