"To be honest I can't imagine you've ever had to admin too large a setup."
That's true. The environment I manage is only about 100 servers, 25 of which are more or less the same (web servers JBoss-based while still each one with its own peculiarities). Hardly a "large" setup while still not a "short" one.
"If you have say 100 machines that a lot of which perform unique functions you'd not want to roll out a new OS release annually, just too much stuff changes/breaks."
Truly not. But I feel quite on point upgrading (not rolling up anew but just upgrading) each two years and a half if the upgrade goes quite comfortably. And you know what? That's the case since I use Debian.
"Or else consider you are an enterprise application developer you have to port/repair your app as libs change with a whole new OS release."
I -well, not "I" but the team at my company, does indeed have to port/repair our apps with each new OS release on Debian. But do you know what? They still have to do it for Red Hat (and SUSE) too. And being forced to work against Debian, SUSE and Red Hat, they feel Debian to be the easy part -or at least so they say.
"And you end up supporting all the old OS versions out there as your customers don't or refuse to switch their OS annually."
Debian doesn't move anually but more somewhere between each two/three years. And you know what? Most of our Debian users tend to be on "Debian Stable" whatever it is at the moment, while we still have to support Red Hat 3, 4 and 5 (at some dot releases at each of them -we are so unlucky that Red Hat tends to break our products even at dot releases) and the same goes for SUSE, mainly because our Red Hat (and SUSE)-base users "on't or refuse to switch their OS annually" (which is nothing but due to the "bad mentality" I already stated by the very beginning).
"I have Fedora on my desktop"
End of the argument. Whatever you say based on such assertion is completly out of scope (you even agreed to this later on your post).
"We go through the same pain with a new RH release"
You lost my point about Debian's "quite easy upgrade path", didn't you?
Of course Fedora breaks things at each upgrade: it's meant to be that way since is the bleeding edge experiment from Red Hat.
Of course you want as long as possible release support from Red Hat -ideally the whole life of your computer/service, since experience dictates that you are far better reinstalling and/or redeploying than trying to cleanly upgrade a system from RH 3 to 4 to 5. But that's not the only world you can get: I manage boxes, by the tens no less, out of the about one hundred I manage grand-total, that happily upgraded -even remotely from a different continent, from Woody to Sarge to Etch to Lenny (and some of them probably will be upgraded to Squeeze too, if not already decomissioned, a 2-to-3x factor older than others would say it's sane) with no or little pains at most. That means upgrading from 2001 to date so there you have your seven years period and then a bit more.
"Nothing against Debian, it's great, but in enterprise space the predictability of a 7 year OS release..."... is more than surpassed by the stability and open development style (which allows for almost three years release windows for those that care) from Debian.
That's my fifteen year-span experience. I don't mean to own the ultimate truth but please, don't be too unpleased if I prefer my own *experience* backed opinion to your *wishful* thinking one.
"A DNS server running on say RHEL 4 which is up to date is just as good as the day it was installed."
That's true, but the last version is even better: software is still evolutioning so fast that it's hard not to see advantages aplicable to your environment at each major release (hey! now I can bind registries with LDAP, or hey! the new version supports views the way I need, etc).
"For their needs, it's fantastic, brilliant and as a techie I absolutely love it. But the argument they're making is a red herring."
Well, you have a point. But see, there is no single messege in all this saying something on the lines of "you had such missplaced price tags because you were not looking at the right places: [disk cabin vendor X] provides 1PB of data per [price tag] which it's on your own league". I thought some would mention i.e. Coraid. This probably mean that they went to the most obvious providers so telling they are 10x more expensive than the solution they finally came with (disregarding all the cool features they didn't need) becomes a fair comparation.
"Lack of a in advanced long term planned update support is one drawback of Debian."
Only if you approach with the wrong mentality. Why would you want really long term support when you have no less than a year window for a free and quite easy upgrade?
"I sure wouldn't trust these guys to back up so much as one bit of my data after seeing what they're backing it up on. I don't care how good they think their software is, this hardware is unacceptable."
Good to know you would never use Amazon or Google services, since they both use the same kind of hardware.
"then there is the support of having someone onsite in less than 4 hours repairing the problems."
They are clever enough not to need such kind of support.
"These are just a couple of advantages these EMC / NetApp / HDS / IBM provide"
They might be advantages *if* you need them, which this company doesn't. If you don't need all those "advantages", then they are only money paid for nothing, which usually is not a sensible bussiness choice.
" I wouldn't allow a support (sub) contractor unrestricted access to the floor of our datacenter"
That's quite true.
But then you need to have somebody round there in case a (sub) contractor needs to go in there. Given overall operation costs, the difference from the guy on almost minimal wages (or maybe not, if he needs to be trained and carrying weapons) to the technician that could do it himself X the risk of lowering the avaliability of your solution because the four hours reaction time (if they are indeed four hours instead of three fifty on the phone, then three fifty for the technicial appearing, then three fifte again to go for the proper spare part) versus the guy being already there maybe makes inhousing your support quite a sensible choice. These guys obviously do think so, and you?
"If you post some expensive system everyone here will immediately tell you how they could build the same thing in their garage over the weekend for pennies."
Maybe. But in this case they are telling how they have in fact *done* it, comparatively for pennies and fully covering their bussiness needs. Being that it works as per the needed specs, are you really trying to tell us that "the clever thing" is paying 10x in order to get the same bussiness value?
"Comparing this thing to enterprise class storage is like some sixteen year old adding a cold air intake and a coat of red paint to his Honda civic"
Maybe, but they are the ones successfully driving their bussiness on operation costs an order of magnitude cheaper than your "enterprise class" solution. Pufff! How unprofessional from them!
"Every time I see something like this the only thing I learn is that yet another person doesn't actually "Get It" when it comes to storage."
Yes, that's why I think too... about you, of course. You obviously don't "Get" that there's more than a class of enterprise needs -even regarding storage, and that choosing the one that best fits the needs is indeed quite "enterprisey".
"you slide it out, pop off the top cover (I've seen both 2/3 length and full-length covers that come off), and work on the guts of the server. Pop the lid back on, and slide it back into the rack."
Good look gaining access to a disk on the middle raw or, god forbids, to the backplane under them.
No; my bet is that as soon as they detect some problem they turn down the whole box and take it directly to the lab without trying to reservice it on field. After all it is not as if they'd had one expensive chunk of hardware they need five nines on, but just a "brick" among dozens of twins with a software layer on top ready to manage a whole node down here and there. Why work within a fridge about 100dB loud when you just can take it on a cart to your lab and redeploy it as a new in a few days?
Some others have told they should have used hot-swappable drives; what they fail to understand is that here the "hot-swappability" is not at the disk level but at the whole computer level.
"Redundancy: You mean as in plain redundancy? These are RAID arrays are they not?"
They seem not even looking for redundancy at "the brick level". It's obvious from its design those boxes either run or doesn't run; the only redundancy is software RAID6, neither EEC memory nor redundant power suplies... even if it's only a malfunctioning disk it's obvious they wouldn't try to correct it and get the machine recovered (did you see how the disks are packaged?): they'll decomission the node, take it to the lab, refurbish, reflash and redeploy it as a new some few days later. They even say that data integrity is above and beyond the "brick" and that it is not a solution 'per se' but a part of one.
"Cooling: I could probably get the whole project chilled for less than 6% of the total cost, depending on how cool you want the rig to run"
From the see of the design with those disks so densely packaged, they'll need almost chilling temperature off the rack to refresh them, specially the disk rank in the middle. The only rack-level photo shows them deployed on opened racks they'd probably would save quite a lot of money by forcing air directly onto the boxes.
"Can you give me a operative definition of free will"
Not, or at least not now. I never said that providing an operative definition for free will was easy; all that I say is that testing for free will is easy. Is it easy to see a big schumultz the size of a truck? Well, is it really that big? yes; is it colorful and opaque? yes. Then it's easy to see a big schumultz; it's only a matter that somebody puts it in front of you. That you don't know what in hell an schumultz is and that you know nobody able to hunt one down to put it in front of me so I in fact see it it's a whole different story. You'll probably think I'm nitpicking but logics is full of such nitpicks and that's what make it work.
"but seem to have trouble with reading comprehension"
Well, I had problem with reading comprehension, but I don't think my reading comprehension was where you think it was.
English is not my mother language and I had to look twice (and even thrice) to ya'll to understand you meant "you all", but it seems it's you the one that didn't understand what I said: "other than here in the South this *word* sounds" heck, the problem is not how it sounds, the problem is that it's not even a word!
"For the rest of the world, who don't quite grok our quaint pronunciation here in the good ole South, the word ya'll is pronounced like [yawl] similar to [yawn]."
For the rest of the world, even the English speaking world, "ya'll" isn't even a word.
"There are a variety of techniques to hide backdoors even from the majority other security experts, such as portknocking."
And surely those trojans would sustain reimagin the devices with clean vendor media and then upload new clean configurations, wouldn't they? It has been 14 months, for godsshake!
"If my ski boots can be custom fitted for a couple hundred dollars then why can't we fit soldiers with a decent helmet?"
Time.
There was an humorist in my country, Gila by name, that was famous for his "on the telephone" numbers. One of them started (while at a mock-up telephone and all war-dressed): -Is it the enemy? -Err... Could you attack this morning a bit latter than usually? -... -No... It's my wife: she insists on me going with her to the dentistry... She's so scared about that, you know...
It seems that real war is not exactly that way with regards to disposables.
"Current seatbelt designs have been shown to contribute to shoulder and abdominal injuries."
Yes, you got more or less the point. Now for a nearer for the bulleye, what if you found "current seatbelt designs have been shown to contribute to *avoidable* shoulder and abdominal injuries"? Well, that's more or less the case here.
"Now, all of apps will have to be web-based SAAS for anyone to get rewarded for their effort implementing their ideas."
If there's really no other way, so what? Now, all street water dealers are forced to get jobs elsewhere because street water selling doesn't make money -damn water pipes! But, wait, you can see people selling bottled water in the street in Sevilla in mid august! so if you can make money, you go for it; if it doesn't, look elsewhere but don't ask for a state-granted monopoly. Isn't it USA the land of the free... market?
"On the other hand it is also scientific statement because it can be falsified (and has been by Relativity)."
That's false. "Force" has not been falsified -it can't, Newtonian dynamics has... which was my point. You can declare a cosmogony that includes both a definition and a test for "free will" and you either observe such "free will" or not, and that's all about free will. But if within your cosmogony includes some predictions (such as "free will will appear on such and such occasions and will measure this or that") that result to be false, it will be the theory as a whole the one that will be falsified not any of their defined observables. Again, you don't falsify F=m*a, you falsify the theory that predicts that says F=m*a (by finding, for instance that F!=m*a).
"I do not know of any such "operative definition and test for it"."
"Free will" is how I call when the Sun first appears over the horizon in the morning. I have a video recording of such fact. See? Demonstrated.
"I don't believe it exists and I'm not convinced that there ever will be one."
One? Which "one"? I told two things: 1) Operative definition of "free will" 2) A test to demonstrate the object defined on point one does indeed exists.
See? Easy.
What you can't probe is that a previously unseen object does *not* exist (for all that we know, it can be on the far side of the Moon where, unkown to us, the Universe is managed by totally different physics laws; not a specially interesting idea, as Ockham would tell us, but still possible and, from time to time, even tried -i.e. non-local non-entanglement theories for quantum mechanics). And you can't falsify observables, just theories. Since "free will" is such an observable (a kind of measurable "object") you can't falsify it in principle. You either observe it or not. But the path to be able to observe an object is an easy one: you first clearly define it (in order to avoid the "true scotchman" fallacy) and then you plan for a test that will detect it (the fact that you don't have the technology to make the measure is not a problem on the theory -take gravity lenses when first proposed, for instance; the fact that your test does not detect it is not a problem with the process either, take gravitons, for instance).
Slashdot Mirror
"To be honest I can't imagine you've ever had to admin too large a setup."
That's true. The environment I manage is only about 100 servers, 25 of which are more or less the same (web servers JBoss-based while still each one with its own peculiarities). Hardly a "large" setup while still not a "short" one.
"If you have say 100 machines that a lot of which perform unique functions you'd not want to roll out a new OS release annually, just too much stuff changes/breaks."
Truly not. But I feel quite on point upgrading (not rolling up anew but just upgrading) each two years and a half if the upgrade goes quite comfortably. And you know what? That's the case since I use Debian.
"Or else consider you are an enterprise application developer you have to port/repair your app as libs change with a whole new OS release."
I -well, not "I" but the team at my company, does indeed have to port/repair our apps with each new OS release on Debian. But do you know what? They still have to do it for Red Hat (and SUSE) too. And being forced to work against Debian, SUSE and Red Hat, they feel Debian to be the easy part -or at least so they say.
"And you end up supporting all the old OS versions out there as your customers don't or refuse to switch their OS annually."
Debian doesn't move anually but more somewhere between each two/three years. And you know what? Most of our Debian users tend to be on "Debian Stable" whatever it is at the moment, while we still have to support Red Hat 3, 4 and 5 (at some dot releases at each of them -we are so unlucky that Red Hat tends to break our products even at dot releases) and the same goes for SUSE, mainly because our Red Hat (and SUSE)-base users "on't or refuse to switch their OS annually" (which is nothing but due to the "bad mentality" I already stated by the very beginning).
"I have Fedora on my desktop"
End of the argument. Whatever you say based on such assertion is completly out of scope (you even agreed to this later on your post).
"We go through the same pain with a new RH release"
You lost my point about Debian's "quite easy upgrade path", didn't you?
Of course Fedora breaks things at each upgrade: it's meant to be that way since is the bleeding edge experiment from Red Hat.
Of course you want as long as possible release support from Red Hat -ideally the whole life of your computer/service, since experience dictates that you are far better reinstalling and/or redeploying than trying to cleanly upgrade a system from RH 3 to 4 to 5. But that's not the only world you can get: I manage boxes, by the tens no less, out of the about one hundred I manage grand-total, that happily upgraded -even remotely from a different continent, from Woody to Sarge to Etch to Lenny (and some of them probably will be upgraded to Squeeze too, if not already decomissioned, a 2-to-3x factor older than others would say it's sane) with no or little pains at most. That means upgrading from 2001 to date so there you have your seven years period and then a bit more.
"Nothing against Debian, it's great, but in enterprise space the predictability of a 7 year OS release..." ... is more than surpassed by the stability and open development style (which allows for almost three years release windows for those that care) from Debian.
That's my fifteen year-span experience. I don't mean to own the ultimate truth but please, don't be too unpleased if I prefer my own *experience* backed opinion to your *wishful* thinking one.
"A DNS server running on say RHEL 4 which is up to date is just as good as the day it was installed."
That's true, but the last version is even better: software is still evolutioning so fast that it's hard not to see advantages aplicable to your environment at each major release (hey! now I can bind registries with LDAP, or hey! the new version supports views the way I need, etc).
"For their needs, it's fantastic, brilliant and as a techie I absolutely love it. But the argument they're making is a red herring."
Well, you have a point. But see, there is no single messege in all this saying something on the lines of "you had such missplaced price tags because you were not looking at the right places: [disk cabin vendor X] provides 1PB of data per [price tag] which it's on your own league". I thought some would mention i.e. Coraid. This probably mean that they went to the most obvious providers so telling they are 10x more expensive than the solution they finally came with (disregarding all the cool features they didn't need) becomes a fair comparation.
"I reported to X, but was typically working for A, B, C... Not a good scene"
Because?
""Except" and "accept" are both homophonic (same sound)"
Except that it doesn't sound like accept, of course.
except: ik-sept
accept: ak-sept
"Lack of a in advanced long term planned update support is one drawback of Debian."
Only if you approach with the wrong mentality. Why would you want really long term support when you have no less than a year window for a free and quite easy upgrade?
"I sure wouldn't trust these guys to back up so much as one bit of my data after seeing what they're backing it up on. I don't care how good they think their software is, this hardware is unacceptable."
Good to know you would never use Amazon or Google services, since they both use the same kind of hardware.
"I wonder if this $117k solution will do (...)"
No, it doesn't.
"then there is the support of having someone onsite in less than 4 hours repairing the problems."
They are clever enough not to need such kind of support.
"These are just a couple of advantages these EMC / NetApp / HDS / IBM provide"
They might be advantages *if* you need them, which this company doesn't. If you don't need all those "advantages", then they are only money paid for nothing, which usually is not a sensible bussiness choice.
" I wouldn't allow a support (sub) contractor unrestricted access to the floor of our datacenter"
That's quite true.
But then you need to have somebody round there in case a (sub) contractor needs to go in there. Given overall operation costs, the difference from the guy on almost minimal wages (or maybe not, if he needs to be trained and carrying weapons) to the technician that could do it himself X the risk of lowering the avaliability of your solution because the four hours reaction time (if they are indeed four hours instead of three fifty on the phone, then three fifty for the technicial appearing, then three fifte again to go for the proper spare part) versus the guy being already there maybe makes inhousing your support quite a sensible choice. These guys obviously do think so, and you?
"If you post some expensive system everyone here will immediately tell you how they could build the same thing in their garage over the weekend for pennies."
Maybe. But in this case they are telling how they have in fact *done* it, comparatively for pennies and fully covering their bussiness needs. Being that it works as per the needed specs, are you really trying to tell us that "the clever thing" is paying 10x in order to get the same bussiness value?
"Comparing this thing to enterprise class storage is like some sixteen year old adding a cold air intake and a coat of red paint to his Honda civic"
Maybe, but they are the ones successfully driving their bussiness on operation costs an order of magnitude cheaper than your "enterprise class" solution. Pufff! How unprofessional from them!
"Every time I see something like this the only thing I learn is that yet another person doesn't actually "Get It" when it comes to storage."
Yes, that's why I think too... about you, of course. You obviously don't "Get" that there's more than a class of enterprise needs -even regarding storage, and that choosing the one that best fits the needs is indeed quite "enterprisey".
"you slide it out, pop off the top cover (I've seen both 2/3 length and full-length covers that come off), and work on the guts of the server. Pop the lid back on, and slide it back into the rack."
Good look gaining access to a disk on the middle raw or, god forbids, to the backplane under them.
No; my bet is that as soon as they detect some problem they turn down the whole box and take it directly to the lab without trying to reservice it on field. After all it is not as if they'd had one expensive chunk of hardware they need five nines on, but just a "brick" among dozens of twins with a software layer on top ready to manage a whole node down here and there. Why work within a fridge about 100dB loud when you just can take it on a cart to your lab and redeploy it as a new in a few days?
Some others have told they should have used hot-swappable drives; what they fail to understand is that here the "hot-swappability" is not at the disk level but at the whole computer level.
"Redundancy: You mean as in plain redundancy? These are RAID arrays are they not?"
They seem not even looking for redundancy at "the brick level". It's obvious from its design those boxes either run or doesn't run; the only redundancy is software RAID6, neither EEC memory nor redundant power suplies... even if it's only a malfunctioning disk it's obvious they wouldn't try to correct it and get the machine recovered (did you see how the disks are packaged?): they'll decomission the node, take it to the lab, refurbish, reflash and redeploy it as a new some few days later. They even say that data integrity is above and beyond the "brick" and that it is not a solution 'per se' but a part of one.
"Cooling: I could probably get the whole project chilled for less than 6% of the total cost, depending on how cool you want the rig to run"
From the see of the design with those disks so densely packaged, they'll need almost chilling temperature off the rack to refresh them, specially the disk rank in the middle. The only rack-level photo shows them deployed on opened racks they'd probably would save quite a lot of money by forcing air directly onto the boxes.
"Can you give me a operative definition of free will"
Not, or at least not now. I never said that providing an operative definition for free will was easy; all that I say is that testing for free will is easy. Is it easy to see a big schumultz the size of a truck? Well, is it really that big? yes; is it colorful and opaque? yes. Then it's easy to see a big schumultz; it's only a matter that somebody puts it in front of you. That you don't know what in hell an schumultz is and that you know nobody able to hunt one down to put it in front of me so I in fact see it it's a whole different story. You'll probably think I'm nitpicking but logics is full of such nitpicks and that's what make it work.
"but seem to have trouble with reading comprehension"
Well, I had problem with reading comprehension, but I don't think my reading comprehension was where you think it was.
English is not my mother language and I had to look twice (and even thrice) to ya'll to understand you meant "you all", but it seems it's you the one that didn't understand what I said: "other than here in the South this *word* sounds" heck, the problem is not how it sounds, the problem is that it's not even a word!
"For the rest of the world, who don't quite grok our quaint pronunciation here in the good ole South, the word ya'll is pronounced like [yawl] similar to [yawn]."
For the rest of the world, even the English speaking world, "ya'll" isn't even a word.
"There are a variety of techniques to hide backdoors even from the majority other security experts, such as portknocking."
And surely those trojans would sustain reimagin the devices with clean vendor media and then upload new clean configurations, wouldn't they? It has been 14 months, for godsshake!
"Or if he had previously planted a trojan he could remotely trigger..."
Well, or he installed a dead-man trigger and it is him retained in prision the thing that will shoot armaggedon.
"If my ski boots can be custom fitted for a couple hundred dollars then why can't we fit soldiers with a decent helmet?"
Time.
There was an humorist in my country, Gila by name, that was famous for his "on the telephone" numbers. One of them started (while at a mock-up telephone and all war-dressed):
-Is it the enemy?
-Err... Could you attack this morning a bit latter than usually?
-...
-No... It's my wife: she insists on me going with her to the dentistry... She's so scared about that, you know...
It seems that real war is not exactly that way with regards to disposables.
"Current seatbelt designs have been shown to contribute to shoulder and abdominal injuries."
Yes, you got more or less the point. Now for a nearer for the bulleye, what if you found "current seatbelt designs have been shown to contribute to *avoidable* shoulder and abdominal injuries"? Well, that's more or less the case here.
"Can someone give me the management summary in laymen terms ?"
Do-Your-Maths (full stop).
"Now, all of apps will have to be web-based SAAS for anyone to get rewarded for their effort implementing their ideas."
If there's really no other way, so what? Now, all street water dealers are forced to get jobs elsewhere because street water selling doesn't make money -damn water pipes! But, wait, you can see people selling bottled water in the street in Sevilla in mid august! so if you can make money, you go for it; if it doesn't, look elsewhere but don't ask for a state-granted monopoly. Isn't it USA the land of the free... market?
"On the other hand it is also scientific statement because it can be falsified (and has been by Relativity)."
That's false. "Force" has not been falsified -it can't, Newtonian dynamics has... which was my point. You can declare a cosmogony that includes both a definition and a test for "free will" and you either observe such "free will" or not, and that's all about free will. But if within your cosmogony includes some predictions (such as "free will will appear on such and such occasions and will measure this or that") that result to be false, it will be the theory as a whole the one that will be falsified not any of their defined observables. Again, you don't falsify F=m*a, you falsify the theory that predicts that says F=m*a (by finding, for instance that F!=m*a).
"I do not know of any such "operative definition and test for it"."
"Free will" is how I call when the Sun first appears over the horizon in the morning. I have a video recording of such fact. See? Demonstrated.
"I don't believe it exists and I'm not convinced that there ever will be one."
One? Which "one"? I told two things:
1) Operative definition of "free will"
2) A test to demonstrate the object defined on point one does indeed exists.
See? Easy.
What you can't probe is that a previously unseen object does *not* exist (for all that we know, it can be on the far side of the Moon where, unkown to us, the Universe is managed by totally different physics laws; not a specially interesting idea, as Ockham would tell us, but still possible and, from time to time, even tried -i.e. non-local non-entanglement theories for quantum mechanics). And you can't falsify observables, just theories. Since "free will" is such an observable (a kind of measurable "object") you can't falsify it in principle. You either observe it or not. But the path to be able to observe an object is an easy one: you first clearly define it (in order to avoid the "true scotchman" fallacy) and then you plan for a test that will detect it (the fact that you don't have the technology to make the measure is not a problem on the theory -take gravity lenses when first proposed, for instance; the fact that your test does not detect it is not a problem with the process either, take gravitons, for instance).
"Your (lacky) reading comprehension misleads you.
wat"
QED.