Slashdot Mirror
Another Internet2 Speed Record Broken
rdwald writes "An international team of scientists led by Caltech have set a new Internet2 speed record of 101 gigabits per second. They even helpfully converted this into one LoC/15 minutes. Lots of technical details in this press release; in addition to the obviously better network infrastructure, new TCP protocols were used."
One Line of Code every 15 minutes? Seems slow to me.
You are being MICROattacked, from various angles, in a SOFT manner.
The speed is 101 Gigabits per second (Gbps), not Gigabytes.
One line of code per 15 minutes? I'd sure like to know what language that'd be...
quidquid latine dictum sit altum videtur.
zoom zoom
Bring on the Porn comments.
But remember, never underestimate the bandwidth of a 747 full of Blueray disks.
--sig fault--
>. if only my HDD would write that fast!
Queue 101 "How fast does it download porn?" jokes...
Cool stuff!
3rd line in the article.
Gbps is GigaBITS not GigaBytes... Big difference between the two.
TFA says 101 gigabits per second, not gigabytes.
Has anyone every stopped to think this might be too fast for its own good?
:P
Isn't there a point when we've reached a speed where rather than deciding what to send from one place to another, we become lazy and start sending everything?
And won't that just lead to massive researcher mp3 swaps?
Cue the gags about "Finally, I shall be able to download my pr0n collection".
/. posts including all the possible combinations of arguments started by SCO stories, how politics is treated here and a whole chapter on non-funny memes.
Cue questions about whether is gigabytes or gigabits.
Cue questions about "How can I get a such gaping-a$$ bandwidth.
One of these days I will write the ultimate FAQ to
Go! Pedal faster.
Natsu gusa-ya, Tsuwamono domo-ga, Yume no ato
or the cost ;)
Internet Speed Quadrupled by International Team During 2004 Bandwidth Challenge
PITTSBURGH, Pa.--For the second consecutive year, the "High Energy Physics" team of physicists, computer scientists, and network engineers have won the Supercomputing Bandwidth Challenge with a sustained data transfer of 101 gigabits per second (Gbps) between Pittsburgh and Los Angeles. This is more than four times faster than last year's record of 23.2 gigabits per second, which was set by the same team.
The team hopes this new demonstration will encourage scientists and engineers in many sectors of society to develop and deploy a new generation of revolutionary Internet applications.
The international team is led by the California Institute of Technology and includes as partners the Stanford Linear Accelerator Center (SLAC), Fermilab, CERN, the University of Florida, the University of Manchester, University College London (UCL) and the organization UKLight, Rio de Janeiro State University (UERJ), the state universities of São Paulo (USP and UNESP), the Kyungpook National University, and the Korea Institute of Science and Technology Information (KISTI). The group's "High-Speed TeraByte Transfers for Physics" record data transfer speed is equivalent to downloading three full DVD movies per second, or transmitting all of the content of the Library of Congress in 15 minutes, and it corresponds to approximately 5% of the rate that all forms of digital content were produced on Earth during the test.
The new mark, according to Bandwidth Challenge (BWC) sponsor Wesley Kaplow, vice president of engineering and operations for Qwest Government Services exceeded the sum of all the throughput marks submitted in the present and previous years by other BWC entrants. The extraordinary achieved bandwidth was made possible in part through the use of the FAST TCP protocol developed by Professor Steven Low and his Caltech Netlab team. It was achieved through the use of seven 10 Gbps links to Cisco 7600 and 6500 series switch-routers provided by Cisco Systems at the Caltech Center for Advanced Computing (CACR) booth, and three 10 Gbps links to the SLAC/Fermilab booth. The external network connections included four dedicated wavelengths of National LambdaRail, between the SC2004 show floor in Pittsburgh and Los Angeles (two waves), Chicago, and Jacksonville, as well as three 10 Gbps connections across the Scinet network infrastructure at SC2004 with Qwest-provided wavelengths to the Internet2 Abilene Network (two 10 Gbps links), the TeraGrid (three 10 Gbps links) and ESnet. 10 gigabit ethernet (10 GbE) interfaces provided by S2io were used on servers running FAST at the Caltech/CACR booth, and interfaces from Chelsio equipped with transport offload engines (TOE) running standard TCP were used at the SLAC/FNAL booth. During the test, the network links over both the Abilene and National Lambda Rail networks were shown to operate successfully at up to 99 percent of full capacity.
The Bandwidth Challenge allowed the scientists and engineers involved to preview the globally distributed grid system that is now being developed in the US and Europe in preparation for the next generation of high-energy physics experiments at CERN's Large Hadron Collider (LHC), scheduled to begin operation in 2007. Physicists at the LHC will search for the Higgs particles thought to be responsible for mass in the universe and for supersymmetry and other fundamentally new phenomena bearing on the nature of matter and spacetime, in an energy range made accessible by the LHC for the first time.
The largest physics collaborations at the LHC, the Compact Muon Solenoid (CMS), and the Toroidal Large Hadron Collider Apparatus (ATLAS), each encompass more than 2000 physicists and engineers from 160 universities and laboratories spread around the globe. In order to fully exploit the potential for scientific discoveries, many petabytes of data will have to be processed, distributed, and analyzed. The key to discovery is the analysis phase, where i
Only morons moderate based on a sig.
TCP is a specific protocol, a "new" TCP protocol would suggest a different protocol, unless it means a revision of the current protocol.
Its GIGABITS not GIGABYTES
Is that with or without the pictures?
You have two hands and one brain, so always code twice as much as you think!
Best read using Christopher Lloyd's voice from Back to The Future, e.g.:
"101 jigowatts per second!!!" --Professor Emmett Brown
"All great things are simple & expressed in a single word: freedom, justice, honor, duty, mercy, hope." --Churchill
Jeez... When you're talking about new world records you think you'd stop to double check those IMPORTANT facts which appear in the first paragraph. :)
But then again this is slashdot.
Area51 - We are watching...
speed record of 101 gigabytes per second.
Wait, isn't this supposed to be a nerdy tech magazine?
I mean, I except this kind of Bit/Byte confusion on CNN, but on slashdot...
How did they sustain a transfer like that? Unless my math is wrong, that's 11GBps ... what has that kind of read/write speed?
I swear I see this sort of thing twice a month on slashdot when does a record being broken stop being news and start becoming expected?
I think the invisible hand of the market has its middle finger extended
--A wise old fart named SC0RN
You could send so much porn at 101 Gbps...*drool*
Yeah, I'm not really sure what the Library of Congress unit does for me. I'm more used to the European metric measurement of Geburninged Volkswagen.
Nowhere in the article does it say how long they ran the test for. A second? A minute? An hour?
I mean that's a full terabyte almost every minute and a half. What has so much data?
Small potatoes make the steak look bigger.
Here comes new super high quality porn on the internet!
A station wagon hauling backup tapes. Too bad the latency is so high!
15 Libraries of Congress in 15 seconds? Great! Anybody got a copy?
I can transfer one and a half terabits from one end of the room to the other in less than a second in two easy steps.
Step 1. Fill 200MB hard drive with data
Step 2. Fling aforementioned hard drive in a frizbee'esque motion across the room.
Expect some data loss however.
Take that Caltech!
They could probably get much better speeds if they compressed it first. The Library of Congress is quite compressible, as there is a lot of redundant data. Text in general is known to be quite compressible.
Here's a question. Sure, you can send 101 Gigabits per second. But what kind of power do you need on either end to send or interpret that much data? I know my hard drive doesn't go that fast. I don't even think my RAM is that fast.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
SCTP was specifically devised as a replacement for TCP as it can emulate the 1 -> 1 connection of TCP but can do connection based 1 -> N too. I thought it has been designed with high speed in mind too. Does anyone know whether this protocol is being used more and more or has it just become another good-idea-at-the-time that got run over by the backwards compatability steamroller?
This is great and all, but has anyone stopped to ask why we need such fast networks? The stock-frenzy driven surplus of unneeded bandwidth was a major contributing factor to the dot-com bust. I remember when I was working on a multi-gigabit, next-generation optical switch, and the project manager was assuring us that in just a few years, people would be downloading their movies from Blockbuster instead of actually traveling there to pick up a DVD. We were all supposed to be videoconferencing left and right by now, with holographic communications just around the corner. A massive growth in online gaming was supposed to cripple the existing legacy networks, forcing providers to upgrade or perish. All of this was supposed to generate a huge demand for bandwidth, which were were poised to deliver.
Well, as we all know, that demand never materialized. We had way more bandwidth than the market needed, and when the bandwidth finally became stressed, providers opted to cap bandwidth and push less-intensive services rather than pay for expensive upgrades to their infrastructures.
I think we should instead be focusing on technologies that can a) generate real new revenue to the providers that we're trying to sell these ultra-fast networks to, b) have obvious and legitimate research or quality of life improvements, and c) are sure-fire hits to attract consumer attention (and $$$).
Don't get me wrong, this is very cool and all, but until Netflix actually lives up to its moniker and sends me my rented movies through my phone/cable line rather than UPS, then it doesn't really matter to me if the network is capable of 5 Gbps or 500 Gbps. Slashdot will still load in a second or 2 either way. We need real products to take advantage of this massive bandwidth, and that revenue will drive research even further, faster. I fear we're going to stall out unless we find a way to embrace these faster networks and make money off of them.
Like woodworking? Build your own picture frames.
acrynums = acronyms
abbriviated = abbreviated
I dunno, my internet seems still pretty fast.
I guess I skip this internet2 thing and just wait for internet3.
I don't need a signature.
where is Al Gore, and when is he going to claim responsibility for it?
---
she won't let you fly, but she might let you sing
Medical imaging produces very large files, and the need to transfer them over distances quickly to save lives is real.
The possibility for video is great as well. Imagine getting multiple feeds of the next WTO event from different sources on the ground. Or quality alternative broadcasting that isn't just some postage-stamp-sized, pixelated blobs. Torrents are nice, but there is something to be said for being jacked in live.
And for those who didn't RTFA, it's 3 DVDs a second.
[insert sig file here]
Thanks for playing the home game. Unfortunately, due to a math error, we miscalculated the entry fee and have deducted $18000 from your bank account. Please come again.
500GB of disk, 5TB of transfer, $5.95/mo
Yep, your entire hard drive moved in 10 seconds but the question is: How do they got those read/write speeds?
Your HD would never reach that... hdparm gives me 40mb/s if I am lucky.
Maybe they have a *LOT* of RAM
Just get that backup tape from place A to place B, :)
so that you write to it at place A, then it scrolls to place B, then read it at place B, then gets written to, then scrolls to place A. Of course, that would take some 10-1000 km of tape with some exotic routing, but that would be cool
--Coder
Why is the Internet2 still developing protocols based on TCP. I thought that they were proven to be somewhat inefficient with others proving better. I could have also sworn I read I2 was developing a new protocol that was almost a blend on TCP and UDP. Maybe that is what this is? At least I am happy with the new speed records... Shows development!
_
Free 27" Sony WEGA TV
...how fast this could transfer the sum of all data (DNA, memory, etc.) contained in a human.
Yes, I'm kidding. But only half kidding. In some crazy future where we can reconstitute energy into matter, how much bandwidth would be needed to do this practically? Do we even have any ideas or estimates on how much storage would be needed to accurately represent the nature of the human body in terms of data? And no, I'm not talking about the "memory" of the brain - I'm talking about the physical manifestation of the body itself, of which the memory of the brain is a part.
Such a blazingly fast connection is amazing, but how the hell do they get the data onto and off the pipe? Are the disk read and write speeds up to that? What about the ram? how the hell do they do that!!!
And the people shall be oppressed, every one by another, and every one by his neighbour Isaiah 3:5
Alright, so Caltech made a special pipe transfer a hojillion bits in a few seconds, shattering all previous records. This sort of thing will become exciting for me when Comcast pulls up in front of my house with a spool of fiber...
But 101 Gib Sent AND recieved
Avg IN - 18.79 Gbps PEAK of 45.59 Gbps
Avg OUT - 26.81 Gbps PEAK of 53.82 Gbps
Avg In&Out - 45.59 Gbps PEAK of 101.6 Gbps
Still Pretty Impressive.
Canadian researchers at CaNet3 did an interesting experiment around this very question.
What do you do when your network is faster than your drives?
You turn the network itself into a drive - a giant drive made of light and 1,000 miles in diameter.
Basically, the idea is that instead of accessing data relatively slowly from a server's drive, you instead keep the data spinning around the fibre network at the speed of light. If anyone wants something - a DVD quality movie for example - they peel it off as it comes whipping by. I'm not sure what speeds they were working with, but I do recall that a DVD would take less than 1/4 of a second to download. Once you hit these kinds of speeds, everything is always everywhere.
FAST TCP = Better window handling, active congestion monitoring and optimization of link use.
It's better to be the foot on the boot than the face on the pavement. ~~ tkx Kadin2048
That was obviously some new movie that came out being transfered. We better get the MPAA access to Internet2 even faster now, otherwise box offices across the country will shut down.
Some Perceived Problems with the Introduction of Terabit Network Technologies.
This short paper attempts to highlight some potential problems associated with the introduction of high speed networking - specifically at the Terabit per second level. These problems are still in the theoretical arena as practical Terabit networks are probably still several weeks away from fabrication.
Introduction.
The primary problem when considering Terabit networks must be the enormous speed that the packets on such networks will be traveling. Naturally there are problems at the protocol level with very large window sizes necessary for useful throughput, and enormous quantities of data "in flight" at any one point. However, these problems are encountered at the Gigabit level and are solvable in principle (by appropriate window and packet size negotiation for instance).
The major problem that is perceived at such high speeds is that data is now flowing at a significant fraction of the speed of light. This brings into play a number of relativistic effects that must be taken into account when designing such high speed networks.
Physical Considerations.
There are firstly a number of physical considerations that must be taken into account. These are problems associated with any body traveling close to the speed of light (c).
A perhaps more serious problem is the case of collisions on a network technology such as ethernet. The collision of two very high speed packets could give rise to many spectacular effects, equivalent to those seen in current particle accelerators. In par
I think the subject says it all.
I don't care how long they ran that test for! where can I get one of those? I guess I'll have to wait 10 years...
I read a lot of : is this needed?, let's be clever and ask oneself what we are doing...
Frankly, it is hilarious from folks who probably jumped on GMail, IPods, stupid phone which does all but work when needed, and other devices which are arguably the most un-needed space on the planet. (No you won't get me to believe your 200MB emails are worth keeping...)
Ciao
As a reminder, the ALICE experiment at CERN will produce per year 1 PB ( Peta Byte ) of _raw_ data. This is only _one_ experiment out of _four_. Add DB overhead and you start getting the picture. And no: there won't be backups: too big. The nature of particle physics is to be statistics. The search is for slight deviations from what is predicted. So the amount of raw data is huge. It is also that the amount of (raw) data per second produced will be in some case magnitude of order bigger.
It is thought that some data will not be stored at all at CERN, but sent straight to remote storage farm. Too much data to be stored localy.
The people analysing those data will be scattered over the planet, involving indeed the need of big transfers.
Ha ha ha: is this needed ? Hi hi let's think about it... Please dump all the crap data you pretend to need and ask again the question.
http://boson.cacr.caltech.edu:8888/
A Jini-based, self-discovering network monitoring tool. That's pretty damn cool too.
And I thought that Jini was totally ignored after I bought "Jini in a Nutshell" for $0.50 at a church book sale!
Conformity is the jailer of freedom and enemy of growth. -JFK
I've wondered for some time when speeds get fast enough, what's to prevent movie delivery to the theaters from being digital?
How about this...a movie theater that can pay per-viewing to the studio?
OF course, most people will just stream movies across anyway...
Oh, well.
QuipWire - Bad Press Run Amok
What's that in Terabytes/Fortnight? Not as cool as the speed Furlongs/Fortnight, but. . .
You are not the customer.
This is not an Internet2 speed record (lsr.internet2.edu), which is measured between single host pair. This demo was done for the SC2004 bandwidth challenge using a large number of hosts.
It's about the size of 60x15x3 DVD's
I'm a professor. I need to run a calculation against a terabyte of satellite imagery data that's stored at sites scattered across the world.
The calculations aren't that complicated, the big headache is reading the data - the faster I can read it, the faster I get my answer.
If I can suck that data to my local mainframe at 1 TB/sec, I can get the answer in about 2 seconds. If it takes an hour to suck the data down, I'll get the data in about 1 hour and 1 second. Even at 100Gbit/sec, that's still only a few minutes. Compare that to the 100Mbit/sec. or so I probably had 10 years ago.
This is a one-off project and it's not worth trying to figure out how to move the calculations to the data, not if I have very fast data pipe.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
I think a suitably-fast mainframe can do the job:
Connect a suitably huge bank of hard disks to the mainframe either as suitably-fast raid-0 or on separate suitably-fast channels, and send the data to the disks as fast as it comes in.
For a sufficiently large value of "suitably" this is guarenteed to work.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Do you mean that we have no reason to be impressed or that the readers can't read.
Or do I miss something?
It's a good markup but not really worth (Score:5, Informative), right? There must be something I missed in your post.
If you mod this up, your slashdot background will turn into a beautiful sunset!
In particular, RFC-1149-compliant networks suffer from this problem, as do 747-based networks.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Meant as a joke, this is a very good remark. I have never understood hopw you can compare transfer speeds when no distance measure is involved...
LoC/minutes is a horrible measurement. With all the trashy romance novels, self-help, and diet books being published every day, we'll still be at a LoC/15 minutes in a millennia from now.
Don't let the MPAA hear, they'll try to use it as evidence of piracy.
Tierce
Who sponsors your feelings?
I, for one, welcome our massive bandwidth pr0n distributors overlords.
At last, we can get info faster than we can save it!
640K ought to be enough for anyone!
:)
I seem to have 380MB in use with just XP and this Mozilla Window... I wonder if Bill Gates' computer just has really good compression
-M
when you see the word 'Linux', drink!
Fermilab has already turned out a small number of petabytes (2 and a bit). CERN is expected to produce on the order of 10 pB/y.
"But all your emitter and collector are belong to me!"
From RFC3257 - Stream Control Transmission Protocol Applicability Statement"
SCTP is also connection-oriented and provides all the transport services that TCP provides. Many Internet applications therefore should find that either TCP or SCTP will meet their transport requirements. Note, for applications conscious about processing cost, there might be a difference in processing cost associated with running SCTP with only a single ordered stream and one address pair in comparison to running TCP.
So you're right, SCTP can perform an equivalent of TCP's functionality, although with additional features, and therefore additional complexity. Of course, increased complexity usually decreases performance. As performance was one of the major goals of this speed test, SCTP would probably have not have been appropriate.
The Internet's nature is peer to peer - 20050301_cs_profs.pdf
they should just use most dsl providers fav metric... how many times faster than 28.8 dialup???
Get your torrents...
Subject line gives at least one source of "lots" of data. Now, whether they're willing to share, that's another story. Perhaps if they were offered some quick form of backup.
When transfering files locally on my suposed 100Mbs ethernet, I can only get about 8Mbs max, this is with two linux boxes and ethernet cards that bus master. So what gives?
I asked some friends and they have the same problem, so it isn't just me. I also tried it on several machines here, all over 700mhz processors. It shouldn't take that much CPU to transfer data when bus mastering does most of the work.
So what is the 100Mbs spec for anyway? Raw bits on the wire? And its reduced that much by time tcp is applied or something?
In answer to the question: "Nowhere in the article does it say how long they ran the test for. A second? A minute? An hour?"
/ sc2004/hiperf.html
On many of the 10Gbps paths we were able to sustain sending over 99% of the available bandwidth for hours at a time. We sustained an aggregate of over 100Gbits/s for about 2 minutes. The median aggregate bandwidth over 48 minutes was about 66Gbits/s. The HEP bandwidth challenge test ran for about 48 minutes.
Much more information is available at: http://www-iepm.slac.stanford.edu/monitoring/bulk