Slashdot Mirror
Purdue Streams a Movie At 7.5Gb/sec
the_psilo writes, "My friend just got back from the Supercomputing conference in Tampa, FL where she and the rest of the Purdue Envision Center rocked the High Performance Computing Bandwidth Challenge by streaming a 2-minute-long, 125-GB movie over a 10-Gb link at 7.5 Gb/sec. They used 6 Apple Xserve RAIDs connected to 12 clients projecting onto their tiled wall (that's 12 streams in all). Lots of accolades from the people who set up the challenge. More links to articles and reviews can be found at the Envision Center Bandwidth Challenge FAQ page."
The two-minute video is a scientific visualization of a cell structure from a bacterium.
The Envision Center site hosts a reduced version of the video.
This just proves that porn really is the driving factor in Internet growth. Cells are the basic building blocks of people and people are what make (most) porn, porn.
---- aut viam inveniam aut faciam
Cue porn jokes.
Hmmm, looks a bit blocky to me. I think they need more key-frames, and less compression.
Get your own free personal location tracker
We all hope that this marvel of IT world will be used for things better than video streaming!
What is more important than video streaming? Oh wait, gaming.
Sorry.
$30 Off All Plans: Use code TRIPLESAWBUCK
We don't need no stinkin reduced version!
I'm sure our employers wouldn't mind if we took a look at the full version.
*psst* *psst* *psst* *mumble* *mumble* *mumble*
The whole thing? Really?
My boss has told me to take the full version of my personal desk stuff home now.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
There is significant overhead associated with the use of TCP/IP. A typical 6.0 Mb/s connection will deliver appx. 4.2 Mb/s this is only about 70% of the connections actual bandwidth. So, 75% is looking pretty good.
What rocks is the ability to reliably deliver 7.5 Gb/s AND do something useful with it.
2 minutes long & 125-GB
"A resolution of 4096x3072, with 24-bit color, running at 30 frames per second,"
What codec did they use?
[Fuck Beta]
o0t!
A 2-minute-long, 125-GB movie... that must have been one super-high resolution chicken.
Developers: We can use your help.
That's not that impressive in the scheme of things (ie Supercomputing 2006). We (Vanderbilt) were there, and we were streaming 35 Gb/sec for hours on end over a parallel filesystem we're developing. And even we weren't the fastest there. Going to Supercomputing is like stepping 3-4 years into the future.
Two minutes??? But I want it noooooow!
I have a friend who just got back from the same conference. He works at Ohio State University in the electron optics facility (read: likes to play with awesome equipment). Here's what he sent along to me: "This week I am in Tampa, Fl at the Supercomputing Conference "SC 06" with the guys from OSC (Ohio SuperComputer group). We have connected the Global Link boxes up and I am attempting to run the Quanta from Florida. Traffic is being routed from the conference to Atlanta, the Internet2, then though to the Third Frontier Network and back to OSU." With the bandwidth he had, he was able to observe, interact, and analyze a sample in a scanning electron microscope (the Quanta) in Ohio, in real time. Now that's a use for bandwidth; it's a lot cheaper to have a great internet connection than to buy the latest electron microscopes.
Not to mention the hardware itself (NICs, routers, bridges, etc.) adds some not-insignificant overhead as well.
Right. While any idiot can get 10 Gb/s link and get 7.5 Gb/s or more out of it, the real feat here was in doing something useful with it at the same time. Remember that in streaming video applications, there's typically a lot of dropped packets because the client has to actually do something with the video immediately. It may just buffer it, but the application that's doing the buffering is often busy doing other stuff as well, like, say decoding video and audio streams and actually piping all that data through to the I/O bus -- oh, and the NIC is usually sitting on that same I/O bus, so that makes things even worse.
My blog
When you try and fill a 10g pipe with a single tcp session, the congestion avoidance mechanisms of tcp will prevent you from filling the pipe. Essentially the sender will ramp up the rate of packets very quickly initially until the receiver sends back a congestion notification. The sender will then cut the send rate *in half*, and climb it back up very slowly - 1 extra byte per round-trip if memory serves (don't quote me on that). This works great for 100m, but to climb from 5g to 10g takes about 30 minutes if you have a cross-US round-trip-time (RTT).
e d_issues/ipj_9-2/gigabit_tcp.html
To get around this you can:
1. Patch your TCP stacks with a few high-performance modifications
2. Figure out - using the RTT, interface buffer sizes, and bandwidth - what the number of outstanding packets can be before the receiver sends back a "slow down" message. Then configure the sender to have a smaller packet queue.
Great article on this here:
http://www.cisco.com/web/about/ac123/ac147/archiv
It's tough to say if that was the problem here (I'm actually assuming it was not) since after a little digging I didn't see any details on their implementation. And no, I'm not interested in truly digging (I have a pesky job thingy to get back to).
Am I missing something?
Yes.
125 / 7.5 = 16.67 seconds.
You're not converting your units properly. The 125 is measured in GB, while the 7.5 is measured in Gb.
http://publicvoidlife.blogspot.com
There is significant overhead associated with the use of TCP/IP. A typical 6.0 Mb/s connection will deliver appx. 4.2 Mb/s this is only about 70% of the connections actual bandwidth.
While there is overhead associated with TCP/IP, it's nowhere near 30%. On a 100 Mbit link in a LAN, you routinely get 11 MB/s (just verified by transferring an Ubuntu image via FTP over the local ethernet with noname switching hardware). With a theoretical maximum of 12.5 MB/s, that's an efficacy of 90%.
6 Mbit sounds like a DSL connection to me. Quite possibly your provider or the servers you download from are responsible for your effective 4.2 Mbit, because TCP/IP isn't.
--- The light at the end of the tunnel is probably a burning truck.
what that much bandwith I hope there isn't a queue
'...if only "Jumping to a Conclusion" was an event in the Olympics.'
Maybe I just don't understand something stupid.
You see, the internet is a series of tubes...
If you mod me down the terrorists will have won
Actually it is that high. The faster you get, the higher the overhead percentage.
If the size of the packets scaled with the bandwidth then the overhead percentage would remain constant.
10gbps is around 150,000 packets (9k jumbo packets) per second. 100mbps is 1,500 packets per second (also using jumbo packets).
As you can see the overhead increases with the bandwidth.
No way. Everyone knows Porn >> Games in the scheme of cosmic importance!
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
I've seen this quite a lot recently here, linking videos from the front page and they are in
Please, this is
605413? Yes, it's a prime.
"The Envision Center site hosts a reduced version of the video."
Come on, this is slashdot, at least link to the full thing...
Task Mangler
How could they possibly reach 7.5 gbit with 6 Xserves, if each Xserve only has a 1 gbit ethernet connection?
:-)
This looks to me like some desperate attempt to justify the money they wasted on bad (Apple!) hardware, drugs and hookers