You've never been sick or had a child, have you? Current individual health care premiums approach nearly 50% of the average family income, and one kid with a broken arm can drive you into bankrupcy.
Actually, you can do pretty will with a high deductible plan+HSA. Also, check for group rates from any professional organization you belong to, and/or your local chamber of commerce.
A 9-person company should be able to get a decent group rate. Especially if most of your employees are young and healthy. And they probably would be, because it's not like a new, small business can afford someone with 20 years' experience.:)
I don't know if this is a bad thing or not, but the availability test does seem to measure what is considered intrusive by society.
I'm not sure I buy it. Police LIDAR (LASER vehicle speed measurement device) units cost about $4,000, or about the same cost as FLIR (thermal imaging) equipment. Given their similar sticker prices, would you say that LIDAR and FLIR are roughly equivalently invasive?
Personally, I do not. I think that FLIR, which is used to enable law enforcement to detect what is going on inside my home, is much more invasive than LIDAR, which is used to measure the velocity of my vehicle. This is mostly because I feel that police have the right to measure the velocity of my automobile, whereas I don't feel they have the right to try to detect what I'm doing in the privacy of my own home, absent probable cause (i.e. evidence of illegal activity.)
In other words, get a warrant if you want to use FLIR. I don't care how much or little it costs.
As a developer, I'd say you've been dealing with some very bad developers and some incredibly enlightened administrators.
I dunno. After all these years, I still remember my algorithms from college, and I can still specify their growth rates in Big O notation, but when Windows gives me some stack dump in some random DLL, I don't have clue #1 what it's trying to tell me.
But maybe that's because I'm too disgusted to want to know.
Any developer who can't competently administer his own machine is incompetent.
I used to do a lot of dev work, and I'll have to be honest with you: I don't know a damn thing about Windows administration. Linux, yes. Windows, no.
If you were to give me a messed up Windows machine and asked me to try to fix it, I could google for answers, but that's about it. I simply have no idea what Windows is trying to tell me when it gives me some memory dump of some DLL or somesuch.
Not that Linux error messages are any more descriptive ("Printer on fire?", anyone?), but at least I know what they mean.
That's the way it always is. The admins want to limit control to make their jobs easier, and the developers want full control to make their jobs easier, and never the twain shall meet.
About the best compromise I've ever seen is where admins say to the developers, "You can have local admin rights. However, don't keep anything important on your local disk (use network shares and source control), because we're not going to even attempt to support your unsupported software. If you bring your machine in with a problem, it's getting imaged, and that's that."
That usually makes the admins happy, because they don't have to increase their workload, and makes developers less likely to bork their machines, because no developer wants to lose a day reinstalling IDEs, etc.
If I were the IT manager, I wouldn't want extra visibility for the IT guys, because it seems to me it would encourage users to circumvent the proper helpdesk channels. Whenever a uniformed IT guy walks by, I can just hear the one-off requests/questions, that would be completely untrackable.
Besides, I always thought in the best-run shops, you never knew that IT existed.:)
Put it on two machines, duh! It should be in two separate locations anyway - power supply failures can take out multiple hard drives and your raid is then a total waste.
Ok, so help me picture how this might work. Let's say I have a database. How do I put the database on two machines (presumably at two separate physical locations, as a lightning strike could take out both servers, no?) in such a way that the data is in sync on both machines? Now, how do I do it in a way with acceptable performance?
The original article was someone asking about problems with their dedicated server. What I guess I'm getting at is that EC2 isn't a dedicated server / VPS provider, and therefore not suitable for their use case. Which you agree with:-)
Well, now I'm beginning to wonder if EC2 might fit their use case. The OP was asking for a provider that wouldn't root his instances, and as Amazon points out in their forums all the time, they won't access customer instances for any reason short of a search warrant. So if you want them to root your box. If you give them a key. They still won't touch your instance.
As far as pricing goes, if you know that you are basically running a server 24/7, you'd use their newish "reserved instances". Now I don't know his server specs, but let's just assume an EC2 "large" instance, which is probably pretty good for his workload of 300 websites, some code hosting, and a few other things.
If he were to purchase a large reserved instance and run it 24/7, it would cost him $1400 + $0.12/hr or $1400+3153.6=4553.6 for 3 years, or $126.48/mo+bandwidth.
Now what would a similarly-speced dedicated server run him from a reputable provider? Basically, a Quad Xeon with 7.5GB of RAM and 850GB local storage. I looked at some of the name-brand folks, and was coming up with some really, really high quotes.
Yes, I know servers will fail. But as long as the disks still spin and haven't had crap written over them, then random VPS/dedicated server provider will be able to resurrect your box without you having to lift a finger. Which is a lot less work for you to do. I still keep backups myself:-)
Well, look at the time The Planet caught fire vs. the time that EC2 got struck by lightning. I forget how long The Planet customers were down for, but if you were an EC2 customer, you could have just relaunched and you'd have been fine. People who really engineered their apps correctly wouldn't have even noticed the lightning strike.
Provided you're using EBS correctly, as you pointed out before.:)
raid was not supposed to be about fault tolerance. Look at the name - redundant array of inexpensive disks. The idea was to make a bunch of cheap disks look like one big expensive disk - the "redundant" just happened to be a misnomer, because it really doesn't have a heck of a lot of redundancy with only one parity drive (or a mirror of one drive, or, worst-case - no parity in a jbod).
Actually, if you read the original Case for Redundant Arrays of Inexpensive Disks, you'll see that in the strictest sense, you assertion is completely wrong. Look at this quote from the authors' conclusions: "With advantages cost-performance, reliability, power consumption, and modular growth, we expect RAIDS to replace SLEDS in future I/O systems."
Even a few years ago, almost 20% of all raid failures resulted in total loss of the raid because a second drive would fail during rebuilding. "Silent failures" are only detected during the rebuild, so it doesn't matter how much you mix drives by manufacturer and batch and age - your risks of a total loss increase with individual disk size.
This is also false for any modern RAID implementation. Any decent RAID controller (including mdadm for Linux software RAID) will run periodic parity checks. For instance, mdadm on Debian runs a full parity check monthly on the first Sunday of the month. So if you have a bad block somewhere that doesn't ordinarily get accessed, it will be detected during the monthly check.
raid is not a backup, but people take the "redundant" part and think - "I can recover from a failure" when in many cases that's simply not true, especially if it's a controller failure and the company is no longer in business
Oh, I agree that many people get bitten by the "RAID is not a backup" issue. But an errant "rm" is orders of magnitude more likely to bite a "RAID=backup" adherent than multiple simultaneous disk failure.
Regarding proprietary hardware raid controllers, I've only ever used software RAID, but people who I know who use hardware controllers buy spares in case the company discontinues the product or goes out of business. This is good practice anyway, as even if the controller is still in production, who wants to wait for fedex delivery when a raid controller flakes out?
Large disks are incredibly cheap compared to historic prices. Keep multiple backups on a couple of machines instead of a raid.
I'll remind you, RAID is not backup. RAID is about high-availability, not about backup. It's great that you do multiple backups, but you'd still have to do that even with RAID.
If you have a disk failure with RAID, the situation is, "oh crap, I need to replace a failed disk sometime soon." If you have a disk failure with no RAID, the situation is, "oh crap, I have to restore from backup."
I realize you've had some nightmares with RAID. But your personal unluckiness does not make RAID a bad idea. Personally, I've had several instances of single disk failure, and I've been very happy to have had RAID. It meant I had an urgent situation instead of an emergency situation.
If either you or I have 4 disks fail simultaneously, we're pretty much in the same boat: restoring from backup. But if we both have 1 disk fail, you are restoring from backup. I am not.
I knew about EBS, didn't realise they did EBS backed images. It's good to know they warn you that your VM is going away (I didn't know that; is it advertised at all?).
It isn't advertised because they can't guarantee you'll get a warning. Basically, if they can run a host machine in a degraded state and send out an email warning, they will do that. However, if a machine simply fails, they can't exactly send out a warning.:)
If you run an application on EC2, you still need to architect your application so that an individual instance can terminate with no notice.
But isn't that always the case? I mean, my physical machines don't usually give me any warning before a crash. What do you do when one of your physical servers fails?
But still, if I want a stable box I'd rather get one that stayed up without me having to check my email to make sure it wasn't about to get shut down.
I'm not sure I know what these "stable boxes" are. Any server can fail. And what's your plan when (not if) one of those "stable boxes" fails?
And there are lots of VPS providers who'll sell you one of those for less than what EC2 charge.
Comparing EC2 prices to another VPS provider doesn't make sense, because EC2 is not a VPS provider. That's like saying zipcar is ripping you off when they charge $7/hr for a car because it'd be cheaper to lease the car for 3 years from the dealership.
The more exacting and alike two or more items are made, the more likely that they will experience near-simultaneous failure when doing the exact same task - which is the behaviour you'll see in a raid setup.
For many years, the general advice in setting up a RAID has been never to use disks from the same batch in an array. Indeed my home disk array has drives of different ages and different manufacturers.
That being said, I would still view 2/3 of an array failing simultaneously as a freak occurrence that was probably caused by improper use, as I pointed out before.
Anyone using a raid a decade from now will be looked upon as a blithering idiot.
To me, anyone who makes confident predictions about what the computing landscape will look like in 10 years looks like a blathering idiot.
RAID has uses beyond fault tolerance, and I hope that in 10 years our storage media won't involve moving around metal filings on a spinning platter. Would RAID still have any use when storage looks more like Solid State Drives?
Possibly, depending on the read/write throughput of these SSDs. Currently, SSD random access kicks the behind of HDD spindles, but for sequential reads, HDDs will give more throughput. Striping data across multiple SSDs could increase throughput, and that might be a use of RAID even 10 years into the future.
Will that be common? I have no idea. But depending on the performance characteristics of whatever storage is cost effective in 2020, I can't see any reason to rule RAID out summarily.
Relaunch... with a new disk loaded from S3. So relaunch from a backup that you had to make yourself.
It was definitely like that a few years ago, but now things are totally different.
Read up on EBS. It's persistent storage that you can attach to an EC2 instance. As of a few months ago, you can even have EBS-backed images. Worth a read, even if you don't plan to ever use it.
My reading of the EC2 terms suggested that they could make your VM go away for maintenance reasons, whereas "normal" VPS hosting will try to keep your data around etc.
In practice, you'll usually get an email a few days to a week before your instance gets terminated. And with EBS, you should be able to relaunch from where you left off. Not from the point of your last backup.
This is a highly unusual event, but I could certainly understand your reasoning, given your experience.
Without knowing anything about your setup, I would guess that you suffered from insufficient cooling, a physical shock to the system, and/or a power surge. The usual culprit in premature drive failure would be prolonged exposure to out-of-spec temperatures, and with 6 drives in one system, it's very easy to run a little hot.
Good luck!
P.S. Yes, I could tolerate 4 of 6 drives failing about as well as you can. My NAS would go down, for sure. But my backup strategy is sufficient for my requirements.
When one of my virtual instances goes down, a replacement instance is launched automatically. If I didn't check my email, I'd never even know it happened.
But don't let the cloud beckon you away. Not when there's LA traffic to drive through...;)
I'm pretty sure that an EC2 instance can just go away at any time... it's not supposed to be a permanent VM that lasts for ever.
Any single machine can "just go away at any time". How does that make EC2 any different?
What EC2 does offer you is the ability to relaunch your instance (in another data center, if desired, in case yours went offline due to natural disaster, fire, or something). If you use EC2 correctly, you should have some pretty robust uptime, for the price.
The solution for many comp sci students is to double major comp sci with one of the above "demand" areas, pass the professional engineering exam, and then the money issue is a non issue. [....] My son's solution to the ratio issue is to attend a large university where there are more female students overall.
Why bother going somewhere with female students? With a ChemE/CS double major, he's never going to see the light of day, anyhow.;)
Banks charge businesses for processing cheques, too. As a business you cannot escape these charges except by taking cash and stuffing it under a matress (business accounts charge for cash deposits too).
You need to change banks. My bank does not charge for business deposits, either cash or check.
Slashdot Mirror
You've never been sick or had a child, have you? Current individual health care premiums approach nearly 50% of the average family income, and one kid with a broken arm can drive you into bankrupcy.
Actually, you can do pretty will with a high deductible plan+HSA. Also, check for group rates from any professional organization you belong to, and/or your local chamber of commerce.
If you don't have your "public option" then you simply don't have health care.
Are you trying to tell me I don't currently have health care?
If I understand your constitution correctly, assassination is acceptable for those who would threaten that option. It is, technically, self-defence.
No, you do not understand our constitution correctly.
A 9-person company should be able to get a decent group rate. Especially if most of your employees are young and healthy. And they probably would be, because it's not like a new, small business can afford someone with 20 years' experience. :)
I don't know if this is a bad thing or not, but the availability test does seem to measure what is considered intrusive by society.
I'm not sure I buy it. Police LIDAR (LASER vehicle speed measurement device) units cost about $4,000, or about the same cost as FLIR (thermal imaging) equipment. Given their similar sticker prices, would you say that LIDAR and FLIR are roughly equivalently invasive?
Personally, I do not. I think that FLIR, which is used to enable law enforcement to detect what is going on inside my home, is much more invasive than LIDAR, which is used to measure the velocity of my vehicle. This is mostly because I feel that police have the right to measure the velocity of my automobile, whereas I don't feel they have the right to try to detect what I'm doing in the privacy of my own home, absent probable cause (i.e. evidence of illegal activity.)
In other words, get a warrant if you want to use FLIR. I don't care how much or little it costs.
Why do you need 3 IDEs, an SQL authoring tool, and a database explorer open at the same time? Ever hear of IDE plugins?
As a developer, I'd say you've been dealing with some very bad developers and some incredibly enlightened administrators.
I dunno. After all these years, I still remember my algorithms from college, and I can still specify their growth rates in Big O notation, but when Windows gives me some stack dump in some random DLL, I don't have clue #1 what it's trying to tell me.
But maybe that's because I'm too disgusted to want to know.
Any developer who can't competently administer his own machine is incompetent.
I used to do a lot of dev work, and I'll have to be honest with you: I don't know a damn thing about Windows administration. Linux, yes. Windows, no.
If you were to give me a messed up Windows machine and asked me to try to fix it, I could google for answers, but that's about it. I simply have no idea what Windows is trying to tell me when it gives me some memory dump of some DLL or somesuch.
Not that Linux error messages are any more descriptive ("Printer on fire?", anyone?), but at least I know what they mean.
That's the way it always is. The admins want to limit control to make their jobs easier, and the developers want full control to make their jobs easier, and never the twain shall meet.
About the best compromise I've ever seen is where admins say to the developers, "You can have local admin rights. However, don't keep anything important on your local disk (use network shares and source control), because we're not going to even attempt to support your unsupported software. If you bring your machine in with a problem, it's getting imaged, and that's that."
That usually makes the admins happy, because they don't have to increase their workload, and makes developers less likely to bork their machines, because no developer wants to lose a day reinstalling IDEs, etc.
If I were the IT manager, I wouldn't want extra visibility for the IT guys, because it seems to me it would encourage users to circumvent the proper helpdesk channels. Whenever a uniformed IT guy walks by, I can just hear the one-off requests/questions, that would be completely untrackable.
Besides, I always thought in the best-run shops, you never knew that IT existed. :)
Put it on two machines, duh! It should be in two separate locations anyway - power supply failures can take out multiple hard drives and your raid is then a total waste.
Ok, so help me picture how this might work. Let's say I have a database. How do I put the database on two machines (presumably at two separate physical locations, as a lightning strike could take out both servers, no?) in such a way that the data is in sync on both machines? Now, how do I do it in a way with acceptable performance?
No - redundant boxes are the way to go. We're at the point where we should be thinking "redundant array of inexpensive computers".
But what about the data?
The original article was someone asking about problems with their dedicated server. What I guess I'm getting at is that EC2 isn't a dedicated server / VPS provider, and therefore not suitable for their use case. Which you agree with :-)
Well, now I'm beginning to wonder if EC2 might fit their use case. The OP was asking for a provider that wouldn't root his instances, and as Amazon points out in their forums all the time, they won't access customer instances for any reason short of a search warrant. So if you want them to root your box. If you give them a key. They still won't touch your instance.
As far as pricing goes, if you know that you are basically running a server 24/7, you'd use their newish "reserved instances". Now I don't know his server specs, but let's just assume an EC2 "large" instance, which is probably pretty good for his workload of 300 websites, some code hosting, and a few other things.
If he were to purchase a large reserved instance and run it 24/7, it would cost him $1400 + $0.12/hr or $1400+3153.6=4553.6 for 3 years, or $126.48/mo+bandwidth.
Now what would a similarly-speced dedicated server run him from a reputable provider? Basically, a Quad Xeon with 7.5GB of RAM and 850GB local storage. I looked at some of the name-brand folks, and was coming up with some really, really high quotes.
Yes, I know servers will fail. But as long as the disks still spin and haven't had crap written over them, then random VPS/dedicated server provider will be able to resurrect your box without you having to lift a finger. Which is a lot less work for you to do. I still keep backups myself :-)
Well, look at the time The Planet caught fire vs. the time that EC2 got struck by lightning. I forget how long The Planet customers were down for, but if you were an EC2 customer, you could have just relaunched and you'd have been fine. People who really engineered their apps correctly wouldn't have even noticed the lightning strike.
Provided you're using EBS correctly, as you pointed out before. :)
raid was not supposed to be about fault tolerance. Look at the name - redundant array of inexpensive disks. The idea was to make a bunch of cheap disks look like one big expensive disk - the "redundant" just happened to be a misnomer, because it really doesn't have a heck of a lot of redundancy with only one parity drive (or a mirror of one drive, or, worst-case - no parity in a jbod).
Actually, if you read the original Case for Redundant Arrays of Inexpensive Disks, you'll see that in the strictest sense, you assertion is completely wrong. Look at this quote from the authors' conclusions: "With advantages cost-performance, reliability, power consumption, and modular growth, we expect RAIDS to replace SLEDS in future I/O systems."
Even a few years ago, almost 20% of all raid failures resulted in total loss of the raid because a second drive would fail during rebuilding. "Silent failures" are only detected during the rebuild, so it doesn't matter how much you mix drives by manufacturer and batch and age - your risks of a total loss increase with individual disk size.
This is also false for any modern RAID implementation. Any decent RAID controller (including mdadm for Linux software RAID) will run periodic parity checks. For instance, mdadm on Debian runs a full parity check monthly on the first Sunday of the month. So if you have a bad block somewhere that doesn't ordinarily get accessed, it will be detected during the monthly check.
raid is not a backup, but people take the "redundant" part and think - "I can recover from a failure" when in many cases that's simply not true, especially if it's a controller failure and the company is no longer in business
Oh, I agree that many people get bitten by the "RAID is not a backup" issue. But an errant "rm" is orders of magnitude more likely to bite a "RAID=backup" adherent than multiple simultaneous disk failure.
Regarding proprietary hardware raid controllers, I've only ever used software RAID, but people who I know who use hardware controllers buy spares in case the company discontinues the product or goes out of business. This is good practice anyway, as even if the controller is still in production, who wants to wait for fedex delivery when a raid controller flakes out?
Large disks are incredibly cheap compared to historic prices. Keep multiple backups on a couple of machines instead of a raid.
I'll remind you, RAID is not backup. RAID is about high-availability, not about backup. It's great that you do multiple backups, but you'd still have to do that even with RAID.
If you have a disk failure with RAID, the situation is, "oh crap, I need to replace a failed disk sometime soon." If you have a disk failure with no RAID, the situation is, "oh crap, I have to restore from backup."
I realize you've had some nightmares with RAID. But your personal unluckiness does not make RAID a bad idea. Personally, I've had several instances of single disk failure, and I've been very happy to have had RAID. It meant I had an urgent situation instead of an emergency situation.
If either you or I have 4 disks fail simultaneously, we're pretty much in the same boat: restoring from backup. But if we both have 1 disk fail, you are restoring from backup. I am not.
I knew about EBS, didn't realise they did EBS backed images. It's good to know they warn you that your VM is going away (I didn't know that; is it advertised at all?).
It isn't advertised because they can't guarantee you'll get a warning. Basically, if they can run a host machine in a degraded state and send out an email warning, they will do that. However, if a machine simply fails, they can't exactly send out a warning. :)
If you run an application on EC2, you still need to architect your application so that an individual instance can terminate with no notice.
But isn't that always the case? I mean, my physical machines don't usually give me any warning before a crash. What do you do when one of your physical servers fails?
But still, if I want a stable box I'd rather get one that stayed up without me having to check my email to make sure it wasn't about to get shut down.
I'm not sure I know what these "stable boxes" are. Any server can fail. And what's your plan when (not if) one of those "stable boxes" fails?
And there are lots of VPS providers who'll sell you one of those for less than what EC2 charge.
Comparing EC2 prices to another VPS provider doesn't make sense, because EC2 is not a VPS provider. That's like saying zipcar is ripping you off when they charge $7/hr for a car because it'd be cheaper to lease the car for 3 years from the dealership.
The more exacting and alike two or more items are made, the more likely that they will experience near-simultaneous failure when doing the exact same task - which is the behaviour you'll see in a raid setup.
For many years, the general advice in setting up a RAID has been never to use disks from the same batch in an array. Indeed my home disk array has drives of different ages and different manufacturers.
That being said, I would still view 2/3 of an array failing simultaneously as a freak occurrence that was probably caused by improper use, as I pointed out before.
Anyone using a raid a decade from now will be looked upon as a blithering idiot.
To me, anyone who makes confident predictions about what the computing landscape will look like in 10 years looks like a blathering idiot.
RAID has uses beyond fault tolerance, and I hope that in 10 years our storage media won't involve moving around metal filings on a spinning platter. Would RAID still have any use when storage looks more like Solid State Drives?
Possibly, depending on the read/write throughput of these SSDs. Currently, SSD random access kicks the behind of HDD spindles, but for sequential reads, HDDs will give more throughput. Striping data across multiple SSDs could increase throughput, and that might be a use of RAID even 10 years into the future.
Will that be common? I have no idea. But depending on the performance characteristics of whatever storage is cost effective in 2020, I can't see any reason to rule RAID out summarily.
Relaunch ... with a new disk loaded from S3. So relaunch from a backup that you had to make yourself.
It was definitely like that a few years ago, but now things are totally different.
Read up on EBS. It's persistent storage that you can attach to an EC2 instance. As of a few months ago, you can even have EBS-backed images. Worth a read, even if you don't plan to ever use it.
My reading of the EC2 terms suggested that they could make your VM go away for maintenance reasons, whereas "normal" VPS hosting will try to keep your data around etc.
In practice, you'll usually get an email a few days to a week before your instance gets terminated. And with EBS, you should be able to relaunch from where you left off. Not from the point of your last backup.
Hard drives fail. Same as anything else mechanical.
Of course they do. However, they don't typically fail simultaneously without some sort of outside influence.
I've seen plenty of hard drives fail, but I have never had 2/3 of them fail all at once. That feels like a fairly rare event to me.
This is a highly unusual event, but I could certainly understand your reasoning, given your experience.
Without knowing anything about your setup, I would guess that you suffered from insufficient cooling, a physical shock to the system, and/or a power surge. The usual culprit in premature drive failure would be prolonged exposure to out-of-spec temperatures, and with 6 drives in one system, it's very easy to run a little hot.
Good luck!
P.S. Yes, I could tolerate 4 of 6 drives failing about as well as you can. My NAS would go down, for sure. But my backup strategy is sufficient for my requirements.
If you used RAID with a hot spare, you'd be a lot happier/safer. What happens if /home fails next time? :P
Not even the cloud could beckon me away.
When one of my virtual instances goes down, a replacement instance is launched automatically. If I didn't check my email, I'd never even know it happened.
But don't let the cloud beckon you away. Not when there's LA traffic to drive through... ;)
HA HA HA EC2 uptime SLA HA HA HA
I'm pretty sure that an EC2 instance can just go away at any time... it's not supposed to be a permanent VM that lasts for ever.
Any single machine can "just go away at any time". How does that make EC2 any different?
What EC2 does offer you is the ability to relaunch your instance (in another data center, if desired, in case yours went offline due to natural disaster, fire, or something). If you use EC2 correctly, you should have some pretty robust uptime, for the price.
The solution for many comp sci students is to double major comp sci with one of the above "demand" areas, pass the professional engineering exam, and then the money issue is a non issue.
[....]
My son's solution to the ratio issue is to attend a large university where there are more female students overall.
Why bother going somewhere with female students? With a ChemE/CS double major, he's never going to see the light of day, anyhow. ;)
I think you'd be surprised at the amount of tail male nurses pull.
I've seen what nurses do, and I'm sorry, but it is so not worth it.
Most US credit cards carry no annual fee, and all debit cards are free.
Banks charge businesses for processing cheques, too. As a business you cannot escape these charges except by taking cash and stuffing it under a matress (business accounts charge for cash deposits too).
You need to change banks. My bank does not charge for business deposits, either cash or check.