I know one of them went to get his PhD...No idea about the other guy.
New company has been a bunch of idiots, as far as tech goes. They made a bunch of stupid decisions in the first year or so, and they're slowly coming to the realization that, actually, the original people knew what the hell they were doing. Not that everything we did was great, but it had come about through common sense responses to an extremely knotty problem.
I used to work for a company that had a huge managed information infrastructure built up of a number of XML feeds from various business units that went through a custom, in-house, processing system, were categorized, databased, and aggregated out to various parties. The in house system was huge and idiosyncratic, but it worked. There were a number of people (2) who maintained it, and were well paid.
So the old company gets bought by the new company, and the new company derides the old system as worthless, fires all the developers, and discontinues the use of the code. The developers ask for, and are granted, the right to open source the code (who's going to want it, right?)
So the new company shops around to a bunch of third party people, and finds someone who is willing to take on the whole infrastructure for a nice low price. Managers are patting themselves on the back so hard they're getting shoulder problems, "This is so much better than that old crap system HA HA HA!"
Well, as I "migrate" all my information stuff it quickly becomes clear that no one at the new 3rd party company understands their processing software, but that all our old codes, all our weird categorizations...All that stuff still works. Well, that's damn peculiar.
The old processing system used to send back an acknowledgement if you sent it a certain series of codes, telling you receipt time, process time, etc, etc. So I sent up the codes, and got back a response, complete with software version information. Fuckers had taken our OWN CODE and SOLD IT BACK TO US, and like a bunch of morons, the goddamn PHBs had PAID for it!
There is a tendency to trust a 3rd party just because you don't know the problems they're having. Be wary, however, that they don't just turn around and make you pay for what you already had for free.
Different people care about different things. There are people who LOVE to know about house fires. There are people who buy the newspaper only for the obituaries.
You'll never hear me argue that everything in a newspaper is worthwhile; I think a lot of the stuff is fricking pointless, at least from my point of view. I never see any relevant national news, and I seldom see anything in the features section (or the sports section, frankly) that I find to be worth a damn.
But all those things are worthwhile to some people, and those people raise hell if their favorite bit changes. Now, if the number of people who raise hell is small enough, they make the change anyway.
But there are parts of the paper that the companies would LOVE to get rid of entirely (like the goddamn TV section) but where, every time the company tries to cut it, the outcry from the bluehairs is such that they have to back down. It's incredible.
I'm sure they're the exception to the rule, and that they only print hard line journalism. Lets compare the top stories on the web sites:
11th hour: Free Concert in Front of the Terminal Station (wtf is a terminal station?), an article I notice to be written by the editor in chief. Nice.
Macon Telegraph (nice, up-to-date name there, jesus you need to move to a new town): Story about a running gunfight, story about a corrupt judge, story about declining state revenue.
Yea, I see what you mean. I know where I'd go if I actually wanted news.
Wow, an indie weekly that is ad supported? How imaginative! It's certainly different from every other ad supported indie weekly!
The problem is, indie weeklies have crap news. If you want to know what band is playing at what club, you can check the weekly. If you want free personal ads, you can check the weekly. If you want well researched news articles about the place where you live, you're outta luck. They may have a couple of op-ed pieces, with-- maybe--one source, and, if you're lucky, the source will actually be a reliable source.
I actually used to run an indie weekly, so I know that of which I speak. Tiny staff, constant pressure to get ads, no ability to tell off an advertiser...I mean, if you were getting ads from the Religious Right, you couldn't write op eds about them, because the money was more important than your integrity. Having to do your own collections; getting paid in fricking barter from small advertisers. It's not a great business.
Your argument is like something I'd imagine hearing when cable companies were starting up. "Who's going to pay for TV?" Answer: people who want more than what you can get in a model that is completely reliant on ad revenue. If your customer is the advertiser, then you are beholden to the advertiser. If your customer is an individual who pays then you have some independence.
Well, I'll qualify that by saying I've never built my own hotswap system, but my experience says that drives usually cost about half of what a stupid removable drive TRAY costs, and the servers with externally accessible drive rails tend to cost more than their counterparts as well.
There was some discussion below that this design actually includes hotswappable drives, so I'll just back off on that assertion. I could very well be wrong, and it could be that the vendors I'm using are raping me on the costs.
People made the same argument toward Google, when they were using off-the-shelf commodity hardware to run their search operation. Do you think they use that sort of hardware anymore? But deploying it quick got them in the game.
Adding hotswappable drive trays to a server triples the cost. If you can just triple the number of servers instead, you can come out ahead, at least in the short term.
Maintenance would be an issue though...I'd hate to be the poor bastard tasked with pulling the node out of the rack and taking it apart to find the bad drive.
Why would you bother? Just start off by writing the data to three nodes, and then you can swap new ones in and out silently. If your space really is cheap, then that's not a problem.
This sort of attitude is how Sun got it's lunch eaten in the market in the first place.
Yes, your hardware rocks. It's so fucking sexy I need new pants when I come into contact with it.
It also costs more than a fucking italian sports car.
Turns out that if your awesome hardware is 10 times better than commodity hardware, but also 25 times as expensive, people are just going to buy more commodity hardware.
I've got some Sun data appliances and I've got some Dell data appliances, and the only difference I've seen between them is purely one of cost. The only thing that ever breaks is drives.
It's a little odd they didn't just choose to netboot, or boot off a cd or something. Having a boot drive at all seems like an unnecessary point of failure.
67 terabytes for under 8000 dollars isn't interesting? Ooookay...
I don't give a damn about iSCSI; this isn't a database server, it's just a flat data file server...Most datacenters are limited by their network bandwidth anyway, not their internal bandwidth, and https isn't any worse than sftp. Paying Amazon a thousand times more, and I'd still be limited by MY bandwidth, not their internal bandwidth.
If they can deliver more storage for less price, then more power to 'em.
The focus of the article was only on the hardware, which was extremely low cost to the point of allowing massive redundancy...This is not an inherently flawed methodology.
If you can deploy cheap 67 terabyte nodes, then you can treat each node like an individual drive, and swap them out accordingly.
I'd need some actual uptime data to make a real judgment on their service vs their competitors, but I don't see any inherent flaws in building their own servers.
Nominally a Slashvertisement, but the detailed specs for their "pods" (watch out guys, Apples gonna SUE YOU) are pretty damn cool. 45 drives on two consumer grade power supplies gives me the heebie jeebies though (powering up in stages sounds like it would take a lot of manual cycling, if you were rebooting a whole rack, for instance), and I'd be interested to know why they chose JFS (perfectly valid choice) over some other alternative...There are plenty of petabyte capable filesystems out there.
Very interesting though. I tried to push a much less ambitious version of this for work, and got slapped down because it wasn't made by (insert proprietary vendor here). Of course, we're still having storage issues because we can't afford the proprietary solution, but at least there is no non-branded hardware in our server room.
And both of you are ignoring any other potential impact from applying nanotech to simple things like harvesting crops!
Take a best case: special ants that are bred to plant/harvest fields for us. What's the cost? Unknown. What's the major benefit? Is there going to even be one? What's the effect on the rest of the ecosystem? Impossible to predict.
Frankly, I think it's much more likely that we'll move to centralized hydroponics than that we'll try anything like a micro-solution to this very macro problem. Central hydro solves the whole problem (no more need for field crops at all), and reduces transport costs as well.
I think monocultures are a big problem, I think till farming is a bad idea, excessive fertilization/pesticide use, inefficient water use, etc, etc. I don't, however, think there is anything significantly wrong with mechanized harvesting; given all the other crap farming practices we condone, that is one of the last things I'm going to worry about.
Disagree. Sure, a combine will leave a pair of wheel ruts that are pretty deep, but that's not remotely enough to call the whole field hardpan, or kill off all the happy bacteria, soil-loosening worms, and biomass that makes a decent growth medium for crops.
There is already a move to re-adopt no-till agriculture. Using macro machinery doesn't prohibit that, and using micro-machinery doesn't mean that it's more likely.
Contrast those big ruts with the sort of scorched earth devastation left behind by the kinds of swarm ants that could take down a corn field...That's not a healthy environment either.
I agree vis a vis monocultures, etc, but I think that is a separate issue.
I'm trying to imagine a horde of tiny robots lugging a single ear of corn a few miles to a drop off point, and then I'm picturing a combine harvester harvesting a whole acre every few minutes, while also doing processing!
Likewise blight and disease. It's usually pretty obvious. You could make little machines to eat pests, but nature has been doing it for a lot longer, and the bitch is pretty good at it.
One thing I could really do without in the Linux community is the attitude that, if you're not willing to compile your own binaries, or write your own upgrade scripts, that is your failing, and not the fact that the software is byzantine or difficult to use.
You can strip down any common linux distro to only the bare bones. Many of them have that as a preconfigured install option, and nearly all of them will let you choose only those services you want/need.
But if you use a common distro, that will completely negate your ability to walk around with a supercilious sneer for all those lesser people who care about putting together a system that a normal person will be able to use or support! And by "normal", in this case, I'm talking about the majority of the minority who can use *nix in the first place!
The reason people use package managers and such is because not everyone has time to resolve every dependency problem by hand.
I've used Slackware, and yes, it's stable, and yes it's reasonably bug-free, and it gets these things by pruning down the default install. If you want to run a bare-bones install of Fedora, it's also extremely stable. It's also of limited utility.
I'm not fond of Slackware in a production environment because upgrading and package maintenance is a pain in the ass. Instead of typing (for example) rpm -q *program*, you have to teach people how to determine which binary version is present, where it is, and coach them in installing new ones, and making sure the dependencies are okay, and if all your machines aren't doing the same thing, this can be a huge time sink.
I'm not terribly fond of "off the shelf" rpms, but it's easy to make my own, and then put them in my own repository, and push them out to every machine that needs one. It's a simple and effective infrastructure, and one that can be grasped by minions who are not capable of scratch building binaries with weird dependencies.
Slashdot Mirror
I know one of them went to get his PhD...No idea about the other guy.
New company has been a bunch of idiots, as far as tech goes. They made a bunch of stupid decisions in the first year or so, and they're slowly coming to the realization that, actually, the original people knew what the hell they were doing. Not that everything we did was great, but it had come about through common sense responses to an extremely knotty problem.
Here's the best/worst example I've ever seen.
I used to work for a company that had a huge managed information infrastructure built up of a number of XML feeds from various business units that went through a custom, in-house, processing system, were categorized, databased, and aggregated out to various parties. The in house system was huge and idiosyncratic, but it worked. There were a number of people (2) who maintained it, and were well paid.
So the old company gets bought by the new company, and the new company derides the old system as worthless, fires all the developers, and discontinues the use of the code. The developers ask for, and are granted, the right to open source the code (who's going to want it, right?)
So the new company shops around to a bunch of third party people, and finds someone who is willing to take on the whole infrastructure for a nice low price. Managers are patting themselves on the back so hard they're getting shoulder problems, "This is so much better than that old crap system HA HA HA!"
Well, as I "migrate" all my information stuff it quickly becomes clear that no one at the new 3rd party company understands their processing software, but that all our old codes, all our weird categorizations...All that stuff still works. Well, that's damn peculiar.
The old processing system used to send back an acknowledgement if you sent it a certain series of codes, telling you receipt time, process time, etc, etc. So I sent up the codes, and got back a response, complete with software version information. Fuckers had taken our OWN CODE and SOLD IT BACK TO US, and like a bunch of morons, the goddamn PHBs had PAID for it!
There is a tendency to trust a 3rd party just because you don't know the problems they're having. Be wary, however, that they don't just turn around and make you pay for what you already had for free.
In WoW that's not so much the case; gear is vitally important. It's the difference between 1000 dps/15,000 hps and 4000 dps/30,000 hps.
That's actually one of the things I like least about WoW: weenies like the OP who have zero life can become little demi-deities.
Different people care about different things. There are people who LOVE to know about house fires. There are people who buy the newspaper only for the obituaries.
You'll never hear me argue that everything in a newspaper is worthwhile; I think a lot of the stuff is fricking pointless, at least from my point of view. I never see any relevant national news, and I seldom see anything in the features section (or the sports section, frankly) that I find to be worth a damn.
But all those things are worthwhile to some people, and those people raise hell if their favorite bit changes. Now, if the number of people who raise hell is small enough, they make the change anyway.
But there are parts of the paper that the companies would LOVE to get rid of entirely (like the goddamn TV section) but where, every time the company tries to cut it, the outcry from the bluehairs is such that they have to back down. It's incredible.
I'm sure they're the exception to the rule, and that they only print hard line journalism. Lets compare the top stories on the web sites:
11th hour: Free Concert in Front of the Terminal Station (wtf is a terminal station?), an article I notice to be written by the editor in chief. Nice.
Macon Telegraph (nice, up-to-date name there, jesus you need to move to a new town): Story about a running gunfight, story about a corrupt judge, story about declining state revenue.
Yea, I see what you mean. I know where I'd go if I actually wanted news.
Wow, an indie weekly that is ad supported? How imaginative! It's certainly different from every other ad supported indie weekly!
The problem is, indie weeklies have crap news. If you want to know what band is playing at what club, you can check the weekly. If you want free personal ads, you can check the weekly. If you want well researched news articles about the place where you live, you're outta luck. They may have a couple of op-ed pieces, with-- maybe--one source, and, if you're lucky, the source will actually be a reliable source.
I actually used to run an indie weekly, so I know that of which I speak. Tiny staff, constant pressure to get ads, no ability to tell off an advertiser...I mean, if you were getting ads from the Religious Right, you couldn't write op eds about them, because the money was more important than your integrity. Having to do your own collections; getting paid in fricking barter from small advertisers. It's not a great business.
Your argument is like something I'd imagine hearing when cable companies were starting up. "Who's going to pay for TV?" Answer: people who want more than what you can get in a model that is completely reliant on ad revenue. If your customer is the advertiser, then you are beholden to the advertiser. If your customer is an individual who pays then you have some independence.
The White House ;)
1600 Pennsylvania Avenue NW
Washington, DC 20500
Am I the only one who always gives their birthday as 01/01/1970 and their zip code as 20500?
I mean, seriously. They don't need to know. Why would I give 'em the right numbers? They're lucky I even allow them to have rough demographic data.
The punishment was so insane, it's no wonder they were afraid...The brits especially would track their mutineers to the ends of the earth.
Pretty much a requirement when most of the crew was press-ganged. It's amazing that system worked as well as it did.
You know what would be even better? Magic.
Ships were often at sea for a period of years, and the crew was often whoever was abducted on the waterfront the day before it sailed.
Mutinies of professional crew on ships that actually saw land sometimes, rare...Rare-er, since mutinies in general were pretty rare.
Well, I'll qualify that by saying I've never built my own hotswap system, but my experience says that drives usually cost about half of what a stupid removable drive TRAY costs, and the servers with externally accessible drive rails tend to cost more than their counterparts as well.
There was some discussion below that this design actually includes hotswappable drives, so I'll just back off on that assertion. I could very well be wrong, and it could be that the vendors I'm using are raping me on the costs.
People made the same argument toward Google, when they were using off-the-shelf commodity hardware to run their search operation. Do you think they use that sort of hardware anymore? But deploying it quick got them in the game.
Adding hotswappable drive trays to a server triples the cost. If you can just triple the number of servers instead, you can come out ahead, at least in the short term.
Maintenance would be an issue though...I'd hate to be the poor bastard tasked with pulling the node out of the rack and taking it apart to find the bad drive.
Why would you bother? Just start off by writing the data to three nodes, and then you can swap new ones in and out silently. If your space really is cheap, then that's not a problem.
This sort of attitude is how Sun got it's lunch eaten in the market in the first place.
Yes, your hardware rocks. It's so fucking sexy I need new pants when I come into contact with it.
It also costs more than a fucking italian sports car.
Turns out that if your awesome hardware is 10 times better than commodity hardware, but also 25 times as expensive, people are just going to buy more commodity hardware.
I've got some Sun data appliances and I've got some Dell data appliances, and the only difference I've seen between them is purely one of cost. The only thing that ever breaks is drives.
It's a little odd they didn't just choose to netboot, or boot off a cd or something. Having a boot drive at all seems like an unnecessary point of failure.
67 terabytes for under 8000 dollars isn't interesting? Ooookay...
I don't give a damn about iSCSI; this isn't a database server, it's just a flat data file server...Most datacenters are limited by their network bandwidth anyway, not their internal bandwidth, and https isn't any worse than sftp. Paying Amazon a thousand times more, and I'd still be limited by MY bandwidth, not their internal bandwidth.
If they can deliver more storage for less price, then more power to 'em.
The focus of the article was only on the hardware, which was extremely low cost to the point of allowing massive redundancy...This is not an inherently flawed methodology.
If you can deploy cheap 67 terabyte nodes, then you can treat each node like an individual drive, and swap them out accordingly.
I'd need some actual uptime data to make a real judgment on their service vs their competitors, but I don't see any inherent flaws in building their own servers.
Nominally a Slashvertisement, but the detailed specs for their "pods" (watch out guys, Apples gonna SUE YOU) are pretty damn cool. 45 drives on two consumer grade power supplies gives me the heebie jeebies though (powering up in stages sounds like it would take a lot of manual cycling, if you were rebooting a whole rack, for instance), and I'd be interested to know why they chose JFS (perfectly valid choice) over some other alternative...There are plenty of petabyte capable filesystems out there.
Very interesting though. I tried to push a much less ambitious version of this for work, and got slapped down because it wasn't made by (insert proprietary vendor here). Of course, we're still having storage issues because we can't afford the proprietary solution, but at least there is no non-branded hardware in our server room.
And both of you are ignoring any other potential impact from applying nanotech to simple things like harvesting crops!
Take a best case: special ants that are bred to plant/harvest fields for us. What's the cost? Unknown. What's the major benefit? Is there going to even be one? What's the effect on the rest of the ecosystem? Impossible to predict.
Frankly, I think it's much more likely that we'll move to centralized hydroponics than that we'll try anything like a micro-solution to this very macro problem. Central hydro solves the whole problem (no more need for field crops at all), and reduces transport costs as well.
I think monocultures are a big problem, I think till farming is a bad idea, excessive fertilization/pesticide use, inefficient water use, etc, etc. I don't, however, think there is anything significantly wrong with mechanized harvesting; given all the other crap farming practices we condone, that is one of the last things I'm going to worry about.
What the hell are you talking about? The issue here is how to harvest a field...The fact that I even brought up no-till at all is a side issue.
Move your soapbox somewhere else.
Disagree. Sure, a combine will leave a pair of wheel ruts that are pretty deep, but that's not remotely enough to call the whole field hardpan, or kill off all the happy bacteria, soil-loosening worms, and biomass that makes a decent growth medium for crops.
There is already a move to re-adopt no-till agriculture. Using macro machinery doesn't prohibit that, and using micro-machinery doesn't mean that it's more likely.
Contrast those big ruts with the sort of scorched earth devastation left behind by the kinds of swarm ants that could take down a corn field...That's not a healthy environment either.
I agree vis a vis monocultures, etc, but I think that is a separate issue.
Now there is a solution in search of a problem.
I'm trying to imagine a horde of tiny robots lugging a single ear of corn a few miles to a drop off point, and then I'm picturing a combine harvester harvesting a whole acre every few minutes, while also doing processing!
Likewise blight and disease. It's usually pretty obvious. You could make little machines to eat pests, but nature has been doing it for a lot longer, and the bitch is pretty good at it.
One thing I could really do without in the Linux community is the attitude that, if you're not willing to compile your own binaries, or write your own upgrade scripts, that is your failing, and not the fact that the software is byzantine or difficult to use.
You can strip down any common linux distro to only the bare bones. Many of them have that as a preconfigured install option, and nearly all of them will let you choose only those services you want/need.
But if you use a common distro, that will completely negate your ability to walk around with a supercilious sneer for all those lesser people who care about putting together a system that a normal person will be able to use or support! And by "normal", in this case, I'm talking about the majority of the minority who can use *nix in the first place!
The reason people use package managers and such is because not everyone has time to resolve every dependency problem by hand.
I've used Slackware, and yes, it's stable, and yes it's reasonably bug-free, and it gets these things by pruning down the default install. If you want to run a bare-bones install of Fedora, it's also extremely stable. It's also of limited utility.
I'm not fond of Slackware in a production environment because upgrading and package maintenance is a pain in the ass. Instead of typing (for example) rpm -q *program*, you have to teach people how to determine which binary version is present, where it is, and coach them in installing new ones, and making sure the dependencies are okay, and if all your machines aren't doing the same thing, this can be a huge time sink.
I'm not terribly fond of "off the shelf" rpms, but it's easy to make my own, and then put them in my own repository, and push them out to every machine that needs one. It's a simple and effective infrastructure, and one that can be grasped by minions who are not capable of scratch building binaries with weird dependencies.