Slashdot Mirror
Never Underestimate the Bandwidth of a Suburban Filled With MicroSD Cards
toygeek writes "If you've been in IT long enough, you're bound to have heard the phrase 'Never underestimate the bandwidth of a station wagon filled with backup tapes.' These days moving data has become so much easier; We've surpassed baud rates and are into Gbps fiber on the backbones, and even in some homes. So, what's the modern equivalent to this, and what does it take to make the OC fiber connections cringe? Follow along as we theoretically stuff MicroSD cards into a Chevy Suburban and see what happens, and take sneakernet to a whole new level."
In my high-school days, we talked about a 747 full of CDs...
I think it may have something to do with growing up on an isolated island nation... there's not many useful places a station-wagon will go.
My book about LSD and Self-Discovery
Also on facebook as: DroppingAcidDaleBewan
The original saying was coined in a time where reading from a tape *was considerably faster* than reading over a network. Hence, transferring data via sneakernet was quicker, inclusive of the read-write times.
Now, with multi-gigabit pipes making up the networks, data can be written, pushed, and read again, all at much higher bitrates than reading any storage medium. It's the read-write to physical medium that are the bottleneck with the sneakernet now.
From the article: "A MicroSD card is only .1 cubic inches, so if all things were equal you could stuff 100 64gb cards into a cubic inch of space! But, that does not seem realistic. In fact it doesn't even seem remotely possible."
Perhaps that's because 1 cubic inch = 10 * 0.1 cubic inches and not 100 * 0.1 cubic inches.
http://what-if.xkcd.com/31/
That is all
Swallowing capsules containing a terabyte (about 12mm in diameter, and 15mm long of microSDs) is quite plausible.
You can easily swallow a hundred of these, and it'll come out over the next 2 days.
100TB/2 days = 600 megabytes a second.
As they point out in the article, the tremendous bandwidth achieved does not include the logistics or time required to initially copy the data onto SD cards, and then back off of the cards upon reaching the destination. Still, beats a flock of parrots trained in Morse code.
I did something similar after a conversation at work using a 53' Semi Trailer and 4TB HDDs.
Semi = 630" x 94" x 102"
HDD = 4" x 1" x 5.75"
Total HDDs = 262628
Total Storage = 1050512 TB
Bandwidth from NYC to SF = 55.58 Tbps (42 hours according to google maps)
-SaNo
Great, you've managed to transfer X terrabytes of data across state lines. I salute you.
Now... go find this list of files that I need. Also that collection of data that our servers need to process. What, that's going to take you HOW long?
Yes might be quicker to send the data from point A to point B by just shipping disks... but only for archiving purposes. If you actually need to access the data, then you still have that last mile (or 10') of having to load that data into a system or network.
Now, if you were shipping a truck load of servers, or maybe a car full of NAS devices that you could just plug into a network and be done with it... then it's not too bad. Then you can start electronically searching the data within minutes, index it, use it for your data-store, whatever.
But just a car load of disks for non-archival purposes... you're asking for a headache.
This is neither news nor does it matter. You should consider changing the site's tagline to "We'll post anything!".
they forgot the time it takes to transfer data onto the SD cards, remove them, and fill the truck. add another 24 hours
For real-time gaming, this would be awful. Well, except compared to the average American ISP.
I have left slashdot and am now on Soylent News. FUCK YOU DICE.
Station wagon? Is that like the paddy wagon?
I'm fairly certain that we'd have to dedicate a second lane of traffic to pornstars in order to keep up that rate of porn downloads for very long.
"That's more porn than the whole Slashdot can consume."
Challenge: ACCEPTED!
FTA:
These days you can't really get a good sturdy station wagon, but the modern equivalent seems like it would be the SUV. Since Chevrolet Suburbans have been around for so long, I'm going to pick that.
Subaru still makes the Legacy station wagon which is sold in the US as the gussied up Outback. That would be a fairer vehicle for comparison. It has 71.3 cu. ft. of space with the seats down vs. the stated 137 cu. ft. of a Suburban so the end result needs to be scaled by 0.52. The stated 68057Gbps bit rate should really be 35390Gbps in a fair evaluation.
I am becoming gerund, destroyer of verbs.
In 3001: The Final Odyssey, Arthur C. Clarke wrote about a petabyte tablet as though it were the ultimate storage medium, something humanity finally arrived at after a millennium. In the book it was enough to store the contents of a human brain!
16,000 or so microSD cards could store a petabyte in roughly a 1-foot by 1-foot by 2-inch space, probably leaving enough room to wire them up as well. Of course, it would cost nearly a million bucks, not counting the hardware necessary to wire it up to be accessed. But, still, I find that very impressive.
(T>t && O(n)--) == sqrt(666)
Assuming 2500kg for the car, 0.0005kg per cards, $50000 for the car and $50 per card:
Not only a bandwidth of 68Tbps, but a mass of 12 metric tons (4.8 empty cars).
Also a market value of $0.96B, or the equivalent of 19,141 new cars, plus one car with a broken suspension.
Probably from the quote referred to in the summary about tapes and station wagons... which predates xkcd by a few decades...
What about round trip latency?? How long does it take me to get my 10GB Full HD video after I've clicked??
John_Chalisque
... but if they actually DID IT that would be way cool... and a waste of money.
But it WOULD BE way cool to get a 1970s station wagon and fill it with 1970s-technology backup tapes and drive it across the country.
Bonus points if the "trek" was widely publicized and made part of a "follow that station wagon online" educational event for kids.
Don't have enough 1970s-era backup tapes? Simulate it with something of the same dimensions with a micro-SD card taped to it, holding one tape-full of data.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
I first heard it as "truckload of CD-ROMs".
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Wouldn't that be 10? You know, with the decimal and all that.
Seriously, off by a factor of 10 is still a lot, no matter what scale you're talking about. Shouldn't someone check this stuff?
Beyond that, the time to read those things has got to be enormous, MicroSD can only be read at 104 MB/s, how many cards do you need to read in parallel to match a decent backbone link? These are not the days where you can have scores of malnourished workers clambering to sort and connect things for you, like the manual switchboards of old, you cannot assume these 19,141,092 MicroSDs can possibly loaded and read apart through the efforts of hundreds of workers, each taking union mandated work breaks and working no more than 8 hours a day without overtime pay.
Take 1.9 million SD cards, after reaching your destination, being loaded in parallel into thousands of slots by an army of whatever minimum wage yokels you can find, each card transferring at 104 MB/s and waiting for checksums to determine which cards must be reshipped in the same Chevy Suburban. Compare this to a single DWDM fibre optic link transferring 100 gigabits per second, day and night, unceasingly, unerringly, unquestioningly until every last bit is transmitted, in order and has been acknowledged.
Sneakernet has gone the way of mercury delay line memory, thermionic valves and punch cards. Storage has not gotten smaller and cars have not gotten slower, so this is the time to celebrate the leap that computer networks have taken.
When Argumentum ad Hominem falls short, try Argumentum ad Matrem
A station wagon is what moms wanted to drive. Whenever somebody says "Americans want SUVs". They're wrong. The SUV is a legal hack to get around fuel economy standards. See? If we build a station wagon with a center of gravity so high that it tips over in the parking lot, it's legally defined as a truck and we don't have to meet the same standards.
Sorry to get into this, but it's one of my pet peeves. Whenever I hear, "Americans want SUVs" it just grates on my nerves. No we didn't. We wanted station wagons. Mom didn't want to tip the kids over and throw them, her, and the groceries into a ditch. Shortsighted regulators left a loophole in CAFE, and they literally drove a truck through it.
Now all these kids don't even know what a station wagon is. Sounds about right. It's the vehicle that the mom down the block had. I distinctly remember us piling in there with the neighbor kids on more than one occasion, and she smoked like a chimney. Shotgun! I get to ride up front with Mrs. Potter and yeah, it smells up here but we didn't know nothin'. We didn't wear seatbelts and... well... I know this is survivorship bias talking but... we survived!
In other words, get your damned SUV off my lawn.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
However many cubic inches it is, it's the fundamental problem with this method. The one with the really crucial data on it is bound to find its way under a seat, into an ashtray etc.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
We can overnight a padded envelope full of 32GB drives anywhere in the country. That's hard to beat when you need to send a few dozen gigabytes in 12 hours.
As I read this, I'm on a trip to Houston to retrieve several terabytes of data for Clonebox. At 90 minutes each way, if I bring back 8TB, that's what, 60 Gbps. There's no way I could transfer that data over my cable modem.
Try doing a DC Promo of a very, very large Active Directory (think Air Force large) to an island with a satellite link of 512K then tell me the SneakerNet is dead.
Yeah we did the DC Promo on a machine here then sent the disks via the next plane. I think that qualifies as a SneakerNet for today's kids.
Short of an actual man in the middle attack, with rifles and warrants, your information doesn't get mirrored to the NSA's data center. Being able to see your packet in transit means security.
A few years ago, I was involved in the conversion of the Stanford AI lab tape archive to modern media. This involved reading thousands of reels of 1/2" magnetic tape. It was a slow process. Volunteers were loading a tape onto a tape drive every 15 minutes for weeks. After each tape was loaded, its contents were sent over an Internet connection in under a minute. It took much longer to wind through the tapes than to transmit the data.
The data went to a server farm at IBM Almaden Research, where the file systems were reassembled (these were incremental dump tapes) and text files were converted from the Stanford AI lab's unique character set to Unicode.
The result was the SAIL DART archive. See the source code for EMACS, the early years.
Yeah the probability of carbon based life is 100%, just look into a mirror.
I think sneaker-net is funny phrase from the days of 10baseT networking being slower than just copying to another media and walking it over. Also used when the stupid networking just wouldn't work...
This article clearly shows why you should go metric.
Tape drives like T10kD and LTO6 have sustained transfer rates that are individually larger than single (non SSD) hard drives. Combined into tape libraries, they are capable of outrunning the fastest RAID arrays on the market.
That of course assumes you are using them as a backup/archive medium and streaming data from a RAID to the tape drive rather than trying to use them in random access mode.
Otherwise you don't have a chance reading them in the right order when restoring the backup.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
http://www.dansdata.com/gz105.htm
The capacity of MicroSD cards has improved a bit since then, resulting in a moderate increase in achievable bandwidth; but other than that the analysis is still essentially the same.
Whenever somebody says "Americans want SUVs". They're wrong.
Men don't like to drive station wagons and women like to feel invulnerable. Tipping over is a lot less likely than hitting or getting hit and women as a group are paranoid that if they don't have the biggest monster on the road, they and their children will be killed by someone else driving the biggest monster on the road who hits them.
You might argue that if nobody had SUVs then nobody would want them. But that ain't the world we live in. So it is entirely true to say that "American want SUVs" no matter how we arrived at the current state.
When information is power, privacy is freedom.
Did no one else immediately think of the weight as soon as the author started talking about filling an SUV with microSD cards? I'm reminded of the saying '100lbs of pillows/feathers is still 100lbs', in reference to how people seem to overlook that very light objects are still heavy if you carry enough of them.
While the exact weight of each of the 19 million microSD card would vary a nice starting point is about 0.4 grams plus or minus 0.1 based on general specs. That's well over 16,000 lbs or 8 tons of microSD cards in the back of that SUV, which according to the page linked in the article is rated for a payload of only 1580 lbs. To get an idea of how much 8 tons is, that's the weight of a medium sized Caterpillar backhoe.
The American's that want SUV's want them for themselves, but not for other people. By far the worst drivers on the road are SUV drivers.
Even worse then that Pravo Jazdy guy in Ireland.
I'm not for banning SUV's though, just requiring that drivers get a special license. Something stricter then a regular license, but less strict then a CDL.
You cannot eavesdrop on a station wagon full of SD or even micro-SD cards or sneaker net in general.
Undetectable Steganography? Yep, there's an app fo
You'd get a very large amount of out-of-sequence deliveries and packet loss. Better get a Toyota Hi-Lux or something like that if you want your data to actually arrive in a predictable amount of time.
I was promised a flying car. Where is my flying car?
"A station wagon is what moms wanted to drive. Whenever somebody says "Americans want SUVs". They're wrong. The SUV is a legal hack to get around fuel economy standards. See? If we build a station wagon with a center of gravity so high that it tips over in the parking lot, it's legally defined as a truck and we don't have to meet the same standards.
Sorry to get into this, but it's one of my pet peeves. Whenever I hear, "Americans want SUVs" it just grates on my nerves. No we didn't. We wanted station wagons. Mom didn't want to tip the kids over and throw them, her, and the groceries into a ditch. Shortsighted regulators left a loophole in CAFE, and they literally drove a truck through it.
Now all these kids don't even know what a station wagon is. Sounds about right. It's the vehicle that the mom down the block had. I distinctly remember us piling in there with the neighbor kids on more than one occasion, and she smoked like a chimney. Shotgun! I get to ride up front with Mrs. Potter and yeah, it smells up here but we didn't know nothin'. We didn't wear seatbelts and... well... I know this is survivorship bias talking but... we survived!
In other words, get your damned SUV off my lawn."
This American wants his SUV, not a station wagon.
I have an SUV instead of a station wagon. Not because I'm a mom (I'm not), and not for grocery getting, but because I live in a place with 4 real seasons and big 4x4 SUVs are handy for getting around for the 2 days it takes the DOT to clear roads after a winter storm. Besides, a car won't pull my snowmobile trailer or the boat.
That said, I don't use the SUV for daily driving. It's 17 years old and unlike the vast majority of SUVs, is muddy as often as not since it also does go off road. As a 2nd vehicle, it is great. Mine is nowhere near as big as a Suburban (2 door full size Yukon GT), and I sure as hell wouldn't want to load it up with micro SD cards; each one may be tiny, but enough to fill the thing up would have it so grossly overloaded it wouldn't be funny.
I'll try to keep my SUV off your lawn.
OSX pwns.
A micro SD card weighs about 0.25 grams, so their calculated 19,141,092 micro-SDs weigh in at 4785 kg. The maximum load of a Chevy Suburban is 2561 pounds (1161kg). Assuming you have a 75kg driver, that lonly leaves 1086 kg for micro-SD cards. So you can only carry about 4,346,596 micro SD cards, less than a quarter of what the authors estimated. The bit capacity is 278 petabytes.
http://what-if.xkcd.com/31/
...and a play ticket to Hong Kong (with transfer to Moscow).
"Love is a familiar; Love is a devil: there is no evil angel but Love." --William Shakespeare ('Love's Labors Lost')
Which is still true between many locations on the planet. The big change is that instead of tapes being used for large volumes you see hard drives being posted around instead. Uploading 2TB via ADSL2 with an upchannel of 256kbps or so would suck immensely, and 10Mbps is still making sending the thing by courier a much more attractive idea.
Comparing LTO5 at 140MB/s to ADSL upload speeds shows a considerable difference (close to four orders of magnitude so *considerably faster*). Beyond a certain amount of data you can get a tape delivered and read in long before someone on a suburban network connection could have uploaded it. Increase the volume and eventually everything apart from really good fibre all the way loses.
Even 10Mbps of HSDSL is roughly two orders of magnitude slower than that 140MB/s tape.
Yes I had a 4WD when I didn't just drive around in a city too, but I think the above poster is writing about the huge number of trucks that never get mud on them of which many are such crap designs that you'd never want to go anywhere near mud with them in the first place. Monster shopping carts built to low safety standards. A van would be cheaper, safer and have more space but the "glamour" of an SUV has hooked them.
Well, I thought f it as well. ctrl-f "weight" brought me to your comment:)
The other factor is cost. Backup tapes aren't used just because they're small, but because they're cheap. Amazon charges $50 for that SD card (For comparison, you can buy a tape that stores 40 times the amount for $78). That's retail, so bulk buy might be cheaper, but even at half that price, you're pushing a billion US dollars.
Laying fibre optic cable costs about $200,000 per mile according to the highest estimate. So that's about $550 million for the same journey. Although you'll need a lot of parallel cables, and I have no idea how this breaks down between the cost of cable and the cost of digging trenches. I do think the high bandwidth cable would be a much better investment if you did need to shift all that data.
When you transfer data from A to B, the needed data is directly accessible in B after the transfer.
In case of the SD cards, you have to count in also the time you need to plug them in and probably copy them to another media in B.
Anybody added this to the bandwith calculation?
At $45/per 64GB card, that's almost a billion dollars!
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
I have worked in IT long enough and I have never once heard anyone say that...
but because I live in a place with 4 real seasons and big 4x4 SUVs are handy for getting around for the 2 days it takes the DOT to clear roads after a winter storm. Besides, a car won't pull my snowmobile trailer or the boat.
I live in Chicago. SUVs are not required. A coupe with snow tires can do everything you listed except pull a large boat (yes they make trailer hitches for small cars).
19,141,092 MicroSD cards will fit in a 2014 Chevy Suburban. [...] 1.14 EiB
What about seek time? If I need file #4455256, How do I go through all those 19M cards? You need a lot more infrastructure and SPACE to find something in that huge stack of cards.