Slashdot Mirror
Why the Coming Data Flood Won't Drown the Internet
High Waters writes "Ars Technica examines predictions of an 'exaflood' of data that some alarmists believe will overwhelm the Internet. A closer look reveals that many of those raising the alarm about an exaflood are generally doing so to make the case against internet neutrality regulation. 'There's a reason that "exaflood" sounds scary. It's supposed to. Though Brett Swanson's Wall Street Journal piece tried to avoid alarmism, it did have an explicitly political point in mind: net neutrality is bad, and it could turn the coming exaflood into a real disaster'."
Grab two of every packet and burn them to a HDVD!
5 And the ISPs saw that the wickedness of man was great in the earth, and that every imagination of the thoughts of his heart was only evil continually, swapping copyrighted music, filching pre-release movies, placing phone calls all about the earth as if information were a mere fluid, like the sea.
6 And it repented the ISP that Oscar winner, Nobel laureate, and all around handsome fellow Al Gore, Junior, had made man to surf on the Internet, and it grieved them at their heart.
7 And the ISPs said, we will destroy the neutral face of the Internet, (which we have implemented from the primordial swellness of Gore) from the face of the earth; both man, and beast, and the creeping thing, and the fowls of the air; for it repenteth us that we have made them to access information in an inexpensive and convenient way.
8 But NOAA found grace in the eyes of the ISPs.
9 These are the generations of NOAA: NOAA was a tidy little bureaucracy, and perfect in its generations, and NOAA walked with the ISPs.
10 And NOAA begat three acronyms: SHEM, HAM, and JAPHETH, which are not relevant to this jape at the moment, but will be cleverly decoded later for humorous effect if need be.
11 The Internet also was corrupt before God, and the earth was filled with sex and violence, because it was just another show, like the news.
12 And the ISPs looked upon the earth, and, behold, it was corrupt; for all flesh had corrupted his way upon the earth.
13 And God said unto NOAA, The end of all flesh is come before me; for the earth is filled with violence through them; and, behold, I will destroy them with the earth with a bolt from my wand of bogus legislation. 14 Make thee an ark of gopher wood; rooms shalt thou make in the ark, and shalt pitch it within and without with pitch.
15 And this is the fashion which thou shalt make it of: The length of the ark shall be three hundred cubits, the breadth of it fifty cubits, and the height of it thirty cubits. If ye know not the length of the cubit, check http://www.wikipedia.org/ but make haste, because Moby Dick shall be sent to devour Jimmy Wales shortly after this post self-destructs.
16 A window shalt thou make to the ark, and in a cubit shalt thou finish it above; and the door of the ark shalt thou set in the side thereof; with lower, second, and third stories shalt thou make it. And though shalt part one mother of a datacenter therein; such that yea, even Marc Andreesen shall be made to blush at the smoking bandwidth thereof.
17 And, behold, I, even I, do bring an exaflood of data upon the earth, to destroy all data, wherein is the breath of binary life, from under heaven; and every thing that is in the earth shall crash like Internet Explorer.
18 But with thee will I establish my covenant; and thou shalt come into the ark, thou, and thy acronyms, and thy support contractor, and thy acronyms' support contractors with thee.
19 And of every living thing of all data, two of every sort shalt thou bring into the ark, to keep them alive with thee; they shall be stored at RAID99.
20 Of fowls after their kind, and of cattle after their kind, of every creeping thing of the earth after his kind, two of every sort shall come unto thee, to keep them alive, but they only need, say, RAID5.
21 And take thou unto thee of all food that is eaten, and thou shalt gather it to thee; and it shall be for food for thee, and for them: plenty of frozen pizza and jolt.
22 Thus did NOAA; according to all that God commanded him, so did they, once they got the budget plus-up.
Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
Simple. It didn't happen before. The Internet has experienced 'exafloods' before. The amount of data and traffic have skyrocketted exponentially every year since this big major growth spurts started in the 1980s and 1990s. How can the Internet do that?
Because it was designed that way, that's why. The Internet is the largest distributed network in the world. TCP/IP was purposefully designed to be scalable on a massively large scale. Sure, we've improved the technology along the way, but the bottom line is that the routers directing all those tubes aren't going to buckle under the pressure anytime soon, and routing technology is just getting better all the time.
My blog
All i'm reading here is: We need to update to a properly working fiber optic system but instead we're gonna use it to stop net neutrality. Or am I mistaken?
There is more info at Ars, and they also mention Brett Swanson's name - he's from the 'discovery' institute.
"We are all geniuses when we dream"
- E.M. Cioran
Predict bad things are going to happen unless people do what you say/buy what you sell/give what you want/etc.
Nothing new here.
I've built an ark out of Ethernet cables and welcome all of slashdot onboard!
Does this mean that I should run to the store and buy up everything on the shelves?
...are the people who want to control the internet.
Media companies wanting to shut down distribution of content not authorized by them (not just illegally copied content but content created and shared under licenses that specifically ALLOW sharing)
News organizations and governments wanting to continue to maintain control over what news we read, view and listen to so they can make sure that the "sheeple" stay "sheeple" and dont actually try to CHANGE their lot in life
Telecommunications providers (including providers of cellular telecommunications) who want to maintain profits for services THEY control and not allow the growth of alternatives to the telco-provided services
Churches and other groups opposed to pornography, gambling and other "vices" who want to be able to ban such content (or if thats not possible, at least control it to the point where its effectively banned)
Manufacturers, distributors and retailers who want to control your abillity to buy stuff to keep bricks & mortar stores alive or to keep people from buying stuff from a country where its cheaper than their own (for example, here in australia, a number of online stores were selling Panasonic DVD recorders really cheap due to the low overheads of those stores. Bricks & Mortar electrical stores complained since they couldn't sell at the price the online guys were selling at and actually make any money. So Panasonic stopped selling the DVD recorders to the online stores)
Governments and spy agencies who want to control the internet so that its easier to spy on the people and look for people who might "rock the boat" or that want to use internet control as a way to hang on to power (look at what happened recently in Burma for example where the government restricted internet access to try to stop the world from finding out how many innocent civilians were being hurt and killed in the name of keeping the dictatorship in power)
When traffic increases (overall, or peaky) to handle more video (for example), capacity has to be added or it quite simply will not get moved. Squeezing out/delaying other traffic will not go very far. Dark fiber has to be lit. When capacity is added because there is more traffic, there is also more "gaps" to fit in "low priority" traffic.
The fundamental problem is people think of the internet as a water pipe, with very simple capacity constraints. It is not. You don't care about water latency while data packet latency or jitter are extremely important.
It is beyond annoying that certain commercial entities are exploiting this misunderstanding to further their own interests at the expense of their customers. One cannot help but see them as grasping and acting out of malice.
to allow the Internet to bog down in data. Those serving this massive amount of data (video, music, etc.) will ensure the infrastructure exists so that their profits are not threatened. This is very basic business administration, if you run out of bandwidth, it's the same as running out of product, and you are turning away willing customers. Losing Money. Don't underestimate the market forces driving this exaflood.
Dominant Meme
Hurry! Send the US$2,000 immediately or be cursed forever and maybe even suffer a fate worse than death!!!
My blog
I wonder how long it took the spin masters to work up that word; It has to be something that people remember that has a hint of disaster in the sound of it, but does not hurt their cause.
"Enjoy what you're doing! If it becomes drudgery, you're doing it wrong!" - Jim Butterfield
I don't like the way it seems like a link is being provided for Brett Swanson's Wall Street Journal piece, but it is actually a link to the Washington Post, and isn't an article by Brett Swanson at all, but Bruce Mehlman and Larry Irving. Maybe the add'l link could be posted as well. Although grammatically ok, the use of a link in this manner is weird and confusing.
Please don't use "umm" or "err" or "erm".
Where are you?!?
They say the first thing to go is your penis. Well, it's either that or your brain. I forget which...
Inviting slashdot onto an ark? Aren't you kind of missing the whole repopulate the earth thing? Or were you hoping they'd get laid in close quarters, where the opposite sex has no where to hide?
Excellent logic!
In Soviet Washington the swamp drains you.
if it can handle that... then need we say more!
Don't believe it'll for a second.. at least not for a few more decades. When we start hitting a technology boundary, then we'll have problems. We haven't hit one yet and people are still inventing better and faster ways to use the exact same fibres without having to re-lay anything. Until that stops, you ain't gonna see much panicking unless it's by scaremongerers.
And if it did, Internet2, with all it's research, technology and connectivity is just over there -> somewhere.
I have a hard time giving any credence to someone that believes that birds always had feathers, fish always had scales, the earth is 5k years old and Adam & Eve jammed in the Garden of Eden with dinosaurs.
There is no need for scare.
* If ISP never really reaches the bandwidth they somehow promised or - god forbid advertised - he'll be sued, anyway.
* For video? Have buffer time. DVB-T already lags analog cable two seconds on live events just for recoding and buffering.
* Mesh radio concepts became technically viable before broadband became really cheap.
* And those people in rural areas won't see a difference anyway
In December of 1995, he wrote: "I predict the Internet...will soon go spectacularly supernova and in 1996 catastrophically collapse."
The only news here is the invention of a new scare word, "exaflood."
The only thing that could really make the Internet collapse would be to abandon the principles of neutrality and end-to-end connectivity, and I'm sure the dire alarmist predictions are intended to soften us up for some proposal... like one to hand over control of the Internet to the telcos so they can allocate bandwidth and prevent "exafloods."
By the way, what happened to all the "dark fiber" that was so spectacularly overbuilt during the dot-bomb era? Is all of it lit up now?
"How to Do Nothing," kids activities, back in print!
checkout the massive growth for last year at the worlds biggest Internet exchange
These are the guys that thought that ATM would rule the world-- a very deterministic bunch at best. Not being able to understand Internet infrastructure- even though they 'run' big portions of it- is normal.
Let's say you needed your own acquired infrastructure to run your own cable system or your own cell/mobiles system. Let's say you didn't want your competitors services and content to be clogging your wires at your 'expense'. Let's say that it galls the living hell out of you that you can't control or throttle the full breadth of packets going over your own network!
And worse, some damn US Senator from Conn. decided to derail your immunity from prosecution over handing over data to the Bush Administration. Can't win that one? Then inject the fear of an 'exa' or peta or oogle event to scare the living shit out of people.
Propaganda. Every last fear-mongering fib.
---- Teach Peace. It's Cheaper Than War.
Dire predictions are usually followed by a project/business plan on how to fix things. In other words someone wants money to fix something that won't need fixing.
How many times did people predict that Usenet would collapse due to the massive amount of data being passed around on the old modem network? It never did happen.
I have to say, an exaflood really *does* sound about a thousand times worse than a petaflood.
I1: I have no problem with two (or more) Internets.
.. and I have NO problems with my ISP filtering all the crap from I2 that tries to cross over to my I1 link. Or with my ISP providing me with "white list" or "black list" filter facilities (which would take care of the spam, thank you verra much).
I2: One for the original intention (legitimate email, web browsing, perhaps online gaming, minor file transfers).
One for the massive data transfers (to include streaming): video, file sharing, online or internet backups, etc.
Take your steenking music and video downloads to the overloaded one, and leave the _real_ internet clear for my WoW, if you please.
Oh
I'd pay for that. Yes, I would.
I'm glad that the exaflood is coming, it gives more warning that goatse is about to appear on my screen when I dont read links carefully enough (about 5% of the picture loaded before I was like wtfx0rz I'm at work and my boss is sitting behind me talking to someone and goatse is appearing on my screen!)
which is totally what she said
Download as many cute kitten and Family Guy video snippets as you can! The continuity of western civilization depends on it! RUN! No, wait, don't run - SIT DOWN AND LOG IN!!!
Prisencolinensinainciusol. Ol Rait!
No good tech story is complete without a comparison of a tangible tech object to *bytes. FTA:
Cisco notes that three exabytes is equivalent to 750 million DVDs.I'm having a little trouble wrapping my mind around that number. Tell me, how many songs is that? How many 40GB iPods Beowulf clustered together make three exabytes? Consider this, you could pave a 4 lane highway from New York to LA with 1GB flash drives and that road still wouldn't have enough space to hold three exabytes.
Except maybe for the shitty ISP who over-sold their bandwith (100k customers, "25Mpbs" sold to each customer, 1Gbps connection to back-bone). As internet consumption habits go up, their customer will start to realise that they don't get the bandwidth they got promised.
That's exactly the kind of enterprise that are going to spread big scares about "exaflood" and try to justify why "net neutrality is bad, throttling bit-torrent is necessary". Whereas the actual problem isn't the growth of internet, but the wrong advertising of dishonest corporations.
Big flood aren't very likely to happen in the near future simply because the speed of progress in the field of communication speed is much higher than the progress of storage space or processor speed, in addition of all you mentioned too.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
The exaspam!
I thought this paragraph from TFA was especially interesting:
But the growth in file sizes is made worse by a concurrent increase in the use of P2P as a delivery mechanism. Distribution gets pushed from the center of the network to the edges as users increasingly become both the consumers and providers of content, so the tubes could be clogged in both directions.... The [US Internet Industry Association] describes this transition as a traffic shift "from the Internet backbone to a peered system in which content is streamed directly to consumers," and the group notes that it will require ISPs to upgrade the most expensive part of their networks to keep pace: the last mile.
Wasn't the Internet designed from the ground up to be "peer-to-peer?" Yes, I know we started with client/server technologies and "the Internet backbone," but fundamentally every machine with a public IP address is, and has always been, the peer of all the other millions of machines with public addresses. That's what makes the Internet so profoundly democratic and so profoundly threatening to established interests.
I suppose cable operators weren't used to seeing the world in those terms, but telcos certainly were. Voice/data services were always interactive, not unidirectional broadcasting. Why should anyone be surprised that the Internet is being used for the purposes its designers envisioned?
Oh, and why is a system where "content is streamed directly to consumers" described as "peered?"
The Washington post article also mentions nothing about network neutrality. IMHO, if it is a disguised case against it, it is very very well disguised. The only thing even possibly relevant is this line: The formula for encouraging such extraordinary investments is clear: minimize tax and regulatory constraints and maximize competition. This line is followed by a list of things that should be passed, and NN is not one of them. Perhaps it is intentionally absent, perhaps it is not. Either way, it really isn't worth using the term "network neutrality" to stir up interest in the article.
is all the advertising. It slows everything down, including Slashdot. Well, it still doesn't slow down the sites that don't carry it. Sure slowed the article some. The graphics are just a bit too heavy also. Blocking doubleclick goes a long way to alleviate the problem. Hint, Hint! But all this switching around must be hell on the DNS servers. Latency is a big problem due to it. If and when we ever break free of the corporate wire with the a wireless cloud or whatever, net neutrality will be a non issue, and the net can be the anarchists' paradise in an authoritarian world. Those who wish to communicate must be protected in those endeavors. IP spoofing and similar "disguises" are such things that are needed to get around government firewalls.
What?
That article is specifically mentioned in TFA. Right at the top of TFA, in fact.
For that matter, TFA is strongly agreeing with you that it's not a problem. It's more of an analysis of different ways of solving the problem -- for instance, do we get to keep net neutrality?
Don't thank God, thank a doctor!
An Exaflood would be 1024 times a petaflood.
The "Wall Street Journal" article actually links to a Washington Post article written by Bruce Mehlman and Larry Irving.
With regards to the dark fiber: Oops.
It's certainly not all lit, but much of it has been leased to third parties. Most of these can't or don't use it to capacity, effectively wasting bandwidth. In many regions, there isn't much dark fiber available, as carriers have stopped offering it and indeed are trying to buy back previously leased dark fiber. The carrier can then use the fiber much more efficiently (by employing WDM technologies, for example) and sell far more services over the same fiber.
The internet is in no way going to collapse, but it does require maintenance and planning to keep things running smoothly.
First time I've ever almost done a spit-take while reading /. Awesome.
I hate to be so lame, but in addition to the editor making his usual mistakes, I can't believe that so many posted without even noticing this. Is there even any way to get the WSJ piece for free? Obviously no one RTFA's anymore.
In before 'you must be new here'
Actually, there are things to worry about.
Too many new applications have hard real time constraints. Copying movie-sized files around, no problem - TCP will throttle. Streaming HDTV without stuttering is much tougher. We're entering an era where the highest-traffic application needs a high quality of service. If resources are tight, there's good no place to throttle. VoIP works because it's a small fraction of traffic. Streaming HDTV looks to be a much larger fraction of traffic.
We still don't have a good answer to managing backbone congestion in pure datagram networks. The Internet today works because the congestion is out near the edges. If we get enough "last mile" bandwidth deployed that the backbone congests before the edges, packet loss rates will go way up. If we have about 2x excess capacity in the backbone, no problem. That's the solution we know.
Microsoft has proposed systems where "broadcast" video is multicast in real time with a high quality of service, while "video on demand" is heavily buffered and sent with a lower quality of service. That's an obvious solution; it's what multicast is for.
(Amusing thought: one solution to video buffering problems is commercials. When transport can't keep up and the player is getting close to running out of buffered content, play an extra locally-stored commercial or two. This lets the buffering refill. Download commercials in advance based on personalization info, then insert them as needed during playback. Don't put them in the main video streams at all.)
Yes, demand for bytes is increasing exponentially while investment in network infrastructure is increasing linearly. This would be alarming, if one was born yesterday, or if one was a knuckle-dragging moron. Those who aren't either note that the amount you get for each dollar spent on technology increases exponentially over time. Thus, not only is a linear spending "curve" capable of keeping up with an exponential required capabilities curve, often it can be done even if the spending line is going down. My most recent gigabit ethernet switch cost a fraction of what my first 10mbps hub. My conclusion from this simple observation is that there's an exponential curve in the increase of stupidity at Nemertes Research.
"Convictions are more dangerous enemies of truth than lies."
Because the internet is a dump truck, not a series of tubes!
ask a guy who might know. Vint Cerf wrote an article in IEEE Computer back on January that put forth his worries that our data comms bandwidth and our content packaging automation will outstrip human capacities to absorb and understand [I think it already has...most people don't even know how much of the flood of data has passed them by]. You may have a problem with the link [to the abstract] because Computer is a subscription journal for IEEE members. Cerf's short piece is mostly concerned with the way the explosion of ways and speeds for copy, transmit and search have and will continue to put all DRM attempts in a losing position. He also thinks the 75 years-after-death copy right is overkill.
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
this one?
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
Ah yes, the Versatile Disc..
Sometimes it fucks you...
And sometimes you fuck it...
The distance from New York to Los Angeles is 3930km (great circle route, longer by a real road, of course). A four lane highway is about 15m wide (two lanes each way, wider if there's a hard shoulder or median). Total area of this road is 609150000m^2. A compact flash card is 43mm by 36mm giving an area of 0.001548m^2, so paving the road would require 393507751938 of them. For 1GB CF cards, that would be 393½ exabytes, so you were only out by a factor of 132.
Of course, if 32GB microSDHC (11mm by 15mm) cards were used instead of 1GB CF cards, then paving the road would need 300.2 times as much storage, or a total of 118 zettabytes.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
RR
My blog with its 200 hits might be contributing to the flood, TED was right.
Dont worry chesus will come to the rescue with a D-Link router and a hub, creating a reduction on the amount of bandwith and freeing some in to what I called possible bandwith that will never be used.