I've been thinking of getting one of those hard disks with the network connector on the back. If you combine this with one of those "network across power lines" adapters you could put the hard disk anywhere in the house (attic, basement...) and still access it from your main PC. This does not protect you from disasters like fire - the data plus backups should not be in the same building. I've got a "garden shed" on my property. Chances are, that it would survive if my house burnt down. Network-over-powerline would be a nice way to get a network connection in there.
However, that scenario still does not protect against things like lightning strikes... Unless you use decent surge protectors etc.
Data protection is not for the faint of heart, and unfortunately not for the average user either.
I've seen good results with Acronis TrueImage, in automatic mode. For "home user" backups, not disaster recovery that is.
Actually, one of the electronics labs I worked for needed to stay with Win95/Win98 for an extended period of time, because their MPEG stream generators needed the "realtime" capabilities provided by the cooperative multitasking. In other words, correct output from the MPEG generator depended on being able to "hijack" the CPU.
The newer schedulers in 2k/XP prevented the card from functioning properly. As a workaround for the lack of security updates on these old OS's, the machines were put on a protected LAN (SSN)
All in all, playing with parallelized OS startup is very nice but the real problem lies elsewhere...
In my experience (data center, 350+ Intel-based servers, Linux + Windows plus a bunch of SPARC Solaris boxes), the OS boot time is negligible compared to the time needed for hardware initialization:
Therefore, if I loaded up all 170TB on a truck my effective bandwidth would be about 3.06e28 bps (or roughly 3e16 Tbps). Once again for huge data repositories there is no substitute for shipping physical media.
Bandwidth is not the issue here - latency is...
A 70-minute Ping can really ruin your killing spree in Quake;-)
In the past I've had misgivings about using Dell hardware, partly because of their habit of changing the chipset in a product line and keeping the model number exactly the same. We also used to (18 months or so back) have trouble with shipments getting delayed in order approval or at Fedex.
They've gotten out of this habit, and even moved to slightly more open-source-friendly chipsets for servers -- Intel instead of Broadcom, AMI instead of Adaptec, etc. And in my experience over the past six months or so, Dell has significantly cleaned up their order processing and shipping, no more mystery delays.
Seconded - I loathed Dell PE's for a while, because of their decision to use Broadcom chipsets and change PERCs every moon. They caused me headaches, trying to patch the KickStart installation server - keeping up with PERC changes, monstrous network configs. I resorted to ordering a large box of Intel PRO1000 dual gigabit cards, disabling the onboard NICs - it sure added to the cost, but made life a lot easier for us.
Recently (apart from the PE1850 snafu where you cannot physically insert proper network cabling) they've cleaned up their act considerably - no more hacking to get things working properly under RHEL or OpenBSD.
Sun. Not much experience there but good stories. They make absolutely the best hardware. The drive trays, the cables, the labels, the cases, the connectors, even the screws are well-placed and well-sized. Even better is theyre the biggest UNIX out there and dont support Windows, which means good hardware is always accompanied by good software.
Hah. Sorry to break this to you, but our new Sun Galaxy system is happily running Windows, Linux and Solaris/x86... And fully supported too;-)
Lately, I've been underwhelmed by Sun Support (no, not their Field Circus Engineer - he's a *great* help to us). We've had to explain to their phone staff (apparently using a call center) what "NIS" is... Really!
And the time we were told that disks were not meant to be running 24x7, THAT must have the reason for our disk failure. We're talking fscking SERVERS here! Truly priceless! (I'm not sh*tting you, this too really happened)
Nah, give me NetApp anytime - the replacement disk arrives before we notice they failed, so to speak. Yay NetApp;-)
After months of problems with DMA timeouts and lockups caused by using a Highpoint RAID controller and a Promise IDE controller I finally bit the bullet and bought a 3Ware Escalade controller. All the sudden, everything is completely stable.
The horror of PCI ATA-cards... I once had 2 Promise TX-100 cards in one machine (4 disks / 4 channels).
After some unnerving rebuilds (raidhotadd) due to IRQ problems I ended up connecting the 4 drives to 2 channels - blergh! Replacing the cards with TX-133 didn't help either. Right now, I'm running the master RAID off a HighPoint dual-channel card, for lack of better hardware...
Someone, please point me to some decent multi-port (non-RAID) ATA expansion cards. I can only find Promise and Highpoint here.
Or is a 3Ware also configurable as a "8-port ATA card"?
How the heck am I supposed to backup 500 GB of data? Set up another raid and copy it over?
Yup - that's exactly the setup I have at home:
- 4x 160GB SW-RAID5 on the master fileserver - 4x 120GB SW-RAID0 on the backup fileserver
The 120GB disks used to make up the master fileserver; they now act as a safety net against complete data loss on the master.
I run rsync to mirror all data from the master to the backup fileserver; in unattended mode, no deletes are performed.
About once a week I explicitly check the RAID statuses and issue the 'purge' command on the backup fileserver. This protects me against accidental file deletes on the master fileserver.
Let's say you have Disk1 and Disk2. The contents of the parity disk, Disk3, are calculated with the following "RAID formula":
"Disk3 = Disk1 - Disk2"
This formula is embedded in the RAID controller (or software driver).
Say, Disk1 = 12; Disk2 = 3 => Disk3 = 9.
Now, Disk1 fails. The RAID formula now misses one value:
"Disk3 = Disk1 - Disk2"
9 = Disk1 - 3
This yields: Disk1 = 12
Say, instead of Disk1, Disk2 fails:
"Disk3 = Disk1 - Disk2"
9 = 12 - Disk2
Same calculation: Disk2 = 3
Finally, assume Disk3 fails:
"Disk3 = Disk1 - Disk2"
Disk3 = 12 - 3
Therefore, Disk3 = 9
This shows that a 3-drive RAID can survive the loss of any one disk. Of course, the actual formula is more complicated but it shows the general principle.
Depends what kind of RAID you're doing. If it's just a mirror, writes are slowed slightly, and read performance is significantly improved over a single drive. Don't even bother trying to do RAID 5 in software. Buy a 3ware Escalade controller or a SCSI RAID controller if you need RAID 5. Keep in mind that many of the cheaper RAID IDE cards (Promise, for one) do much of their work in software too, and often perform about as well or even worse than straight software RAID.
I've run software RAID-5 on Linux for several years on two of my home fileservers.
The only problem I ever encountered were hardware failures (Promise *ack* *spit* PCI IDE cards) and one drive failure. Performance is not really an issue for home use; I can easily saturate my 100Mbps network card.
My Fileserver: AMD Duron 1300MHz, 768MB RAM
/dev/md0 441G 339G 93G 79%/home
This device was built from 4x 160GB 7200rpm SW RAID-5 for online storage (including all of my digital photos, and my collection of CD's ripped to MP3).
For backup I have an old Celeron 433, 512MB RAM box with 4x 120GB 5400rpm SW RAID-5
The main fileserver is rsynced to the backup server once a week. CPU on the backup server is a bottleneck; the Celeron is a bit underpowered for rsync, but it works;)
My $0.02: - Software RAID is perfectly usable, especially for typical home use. Performance is adequate. - With RAID-5 you "lose" only one disk to parity so it is quite cheap to build - Yes, I'd really like a 3Ware Escalade but if the card fails I need to get a new one pronto; software RAID sets can be migrated to most PCs.
If I delete something, I want it GONE. It's not as good as an idle-time thread that 11-pass nukes unallocated sectors at random, but it'll do for a start.
Totally agree - Windows has had a similar feature for weeks now. It's about time the OpenSource community finally catches up!
The benefits of this move are stunning. We have been able to hire 16 additional employees to handle our own fork of Portage, and 22 additional employees to provide support.
Errrmmm... So basically you're saying that abandoning RedHat made you hire an extra 38 people for IT support. If I do my math right, you have at least 38 people out of a total of 137 doing support.
That's more than a 1:3 ratio of indirect vs. direct personnel (unless your business is selling your Gentoo-fork).
I'm not sure if I want to use your example in convincing my company to go the Open Source way...
SCOldera is starting to look a lot like that other lawfirm passing for a tech company. They really should change their company motto to
"SCOldera - You Innovate, We Litigate"
With the current legal system, those with the money always win. Companies like SuSE cannot possibly sustain legal action. So in a way it's good they picked a fight with IBM - at least they've got lawyers to spare.
It'll be interesting to see how SCOldera either tries to backpedal or dies a horrible flaming death. I vote for option 2;)
Err... One appropriate quote springs to mind after reading your scenario;)
"There is a theory which states that if anyone discovers exactly what the Universe if for and why it is here, it will instantly disappear and be replaced by something even more bizarre and inexplicable. There is another which states that this has already happened." -- Douglas Adams
Slashdot Mirror
However, that scenario still does not protect against things like lightning strikes... Unless you use decent surge protectors etc.
Data protection is not for the faint of heart, and unfortunately not for the average user either.
I've seen good results with Acronis TrueImage, in automatic mode. For "home user" backups, not disaster recovery that is.
Actually, one of the electronics labs I worked for needed to stay with Win95/Win98 for an extended period of time, because their MPEG stream generators needed the "realtime" capabilities provided by the cooperative multitasking. In other words, correct output from the MPEG generator depended on being able to "hijack" the CPU.
The newer schedulers in 2k/XP prevented the card from functioning properly. As a workaround for the lack of security updates on these old OS's, the machines were put on a protected LAN (SSN)
All in all, playing with parallelized OS startup is very nice but the real problem lies elsewhere...
In my experience (data center, 350+ Intel-based servers, Linux + Windows plus a bunch of SPARC Solaris boxes), the OS boot time is negligible compared to the time needed for hardware initialization:
- BIOS startup
- Memory check
- Remote Console init (DRAC/XSCF etc.)
- RAID Controller(s) init, disk spinup
- RAID Consistency Check, volume initialization
- Start Boot Sequence
Especially the disk subsystems cause large delays - most time is spent waiting for the GRUB screen.
Parallelizing the hardware initialization is where we could make some significant progress.
Somebody set up us the bomb.
I agree completely - prepositions are not words to end sentences with.
Aaaargh! The memories! When does the hurting stop??
Bandwidth is not the issue here - latency is...
A 70-minute Ping can really ruin your killing spree in Quake
Well, it does matter for Kerberos / MS Active Directory authentication.
In any shared software development environment, time needs to be accurate or your builds will fail in strange ways.
And I'd like to be able to correllate our syslog output, too...
Seconded - I loathed Dell PE's for a while, because of their decision to use Broadcom chipsets and change PERCs every moon. They caused me headaches, trying to patch the KickStart installation server - keeping up with PERC changes, monstrous network configs. I resorted to ordering a large box of Intel PRO1000 dual gigabit cards, disabling the onboard NICs - it sure added to the cost, but made life a lot easier for us.
Recently (apart from the PE1850 snafu where you cannot physically insert proper network cabling) they've cleaned up their act considerably - no more hacking to get things working properly under RHEL or OpenBSD.
Hah. Sorry to break this to you, but our new Sun Galaxy system is happily running Windows, Linux and Solaris/x86... And fully supported too
Lately, I've been underwhelmed by Sun Support (no, not their Field Circus Engineer - he's a *great* help to us). We've had to explain to their phone staff (apparently using a call center) what "NIS" is... Really!
And the time we were told that disks were not meant to be running 24x7, THAT must have the reason for our disk failure. We're talking fscking SERVERS here! Truly priceless! (I'm not sh*tting you, this too really happened)
Nah, give me NetApp anytime - the replacement disk arrives before we notice they failed, so to speak. Yay NetApp
"Beware of bugs in the above code; I have only proved it correct, not tried it."
-- Donald Knuth
The horror of PCI ATA-cards... I once had 2 Promise TX-100 cards in one machine (4 disks / 4 channels).
After some unnerving rebuilds (raidhotadd) due to IRQ problems I ended up connecting the 4 drives to 2 channels - blergh! Replacing the cards with TX-133 didn't help either. Right now, I'm running the master RAID off a HighPoint dual-channel card, for lack of better hardware...
Someone, please point me to some decent multi-port (non-RAID) ATA expansion cards. I can only find Promise and Highpoint here.
Or is a 3Ware also configurable as a "8-port ATA card"?
Yup - that's exactly the setup I have at home:
- 4x 160GB SW-RAID5 on the master fileserver
- 4x 120GB SW-RAID0 on the backup fileserver
The 120GB disks used to make up the master fileserver; they now act as a safety net against complete data loss on the master.
I run rsync to mirror all data from the master to the backup fileserver; in unattended mode, no deletes are performed.
About once a week I explicitly check the RAID statuses and issue the 'purge' command on the backup fileserver. This protects me against accidental file deletes on the master fileserver.
Err - let me oversimplify a bit...
Let's say you have Disk1 and Disk2. The contents of the parity disk, Disk3, are calculated with the following "RAID formula":
"Disk3 = Disk1 - Disk2"
This formula is embedded in the RAID controller (or software driver).
Say, Disk1 = 12; Disk2 = 3 => Disk3 = 9.
Now, Disk1 fails. The RAID formula now misses one value:
"Disk3 = Disk1 - Disk2"
9 = Disk1 - 3
This yields: Disk1 = 12
Say, instead of Disk1, Disk2 fails:
"Disk3 = Disk1 - Disk2"
9 = 12 - Disk2
Same calculation: Disk2 = 3
Finally, assume Disk3 fails:
"Disk3 = Disk1 - Disk2"
Disk3 = 12 - 3
Therefore, Disk3 = 9
This shows that a 3-drive RAID can survive the loss of any one disk. Of course, the actual formula is more complicated but it shows the general principle.
HTH!
I've run software RAID-5 on Linux for several years on two of my home fileservers.
The only problem I ever encountered were hardware failures (Promise *ack* *spit* PCI IDE cards) and one drive failure. Performance is not really an issue for home use; I can easily saturate my 100Mbps network card.
My Fileserver: AMD Duron 1300MHz, 768MB RAM
This device was built from 4x 160GB 7200rpm SW RAID-5 for online storage (including all of my digital photos, and my collection of CD's ripped to MP3).
For backup I have an old Celeron 433, 512MB RAM box with 4x 120GB 5400rpm SW RAID-5
The main fileserver is rsynced to the backup server once a week. CPU on the backup server is a bottleneck; the Celeron is a bit underpowered for rsync, but it works ;)
My $0.02:
- Software RAID is perfectly usable, especially for typical home use. Performance is adequate.
- With RAID-5 you "lose" only one disk to parity so it is quite cheap to build
- Yes, I'd really like a 3Ware Escalade but if the card fails I need to get a new one pronto; software RAID sets can be migrated to most PCs.
Totally agree - Windows has had a similar feature for weeks now. It's about time the OpenSource community finally catches up!
MS "Witty" Disk Security
;) (for the humor-impaired)
Errrmmm... So basically you're saying that abandoning RedHat made you hire an extra 38 people for IT support. If I do my math right, you have at least 38 people out of a total of 137 doing support.
That's more than a 1:3 ratio of indirect vs. direct personnel (unless your business is selling your Gentoo-fork).
I'm not sure if I want to use your example in convincing my company to go the Open Source way...
Speech != beer
Lemme see... 007 Agent under Fire is used to circumvent the XBOX copy protection mechanism.
;-)
;)
Tools that circumvent copy protection are of course terrorist^h^h^h^h^h^h^h^h a violation of the DMCA.
Ergo: 007 Agent under Fire is illegal, since it is a tool to circumvent copy protection. Boy, I can't wait until EA is sued into oblivion
Great. I love US law
"Firings will continue until morale improves"
;)
- The Management
Sorry, couldn't resist
Name one product that lost money for 3+ years and then came to dominence?
;-)
MS Internet Explorer?
SCOldera is starting to look a lot like that other lawfirm passing for a tech company. They really should change their company motto to
;)
"SCOldera - You Innovate, We Litigate"
With the current legal system, those with the money always win. Companies like SuSE cannot possibly sustain legal action. So in a way it's good they picked a fight with IBM - at least they've got lawyers to spare.
It'll be interesting to see how SCOldera either tries to backpedal or dies a horrible flaming death. I vote for option 2
Err... One appropriate quote springs to mind after reading your scenario ;)
"There is a theory which states that if anyone discovers exactly what the Universe if for and why it is here, it will instantly disappear and be replaced by something even more bizarre and inexplicable. There is another which states that this has already happened." -- Douglas Adams
Hrm... "Slashdot Community" -> "AC dimly mouths snot"... Coincidence?