Slashdot Mirror
Purdue Streams a Movie At 7.5Gb/sec
the_psilo writes, "My friend just got back from the Supercomputing conference in Tampa, FL where she and the rest of the Purdue Envision Center rocked the High Performance Computing Bandwidth Challenge by streaming a 2-minute-long, 125-GB movie over a 10-Gb link at 7.5 Gb/sec. They used 6 Apple Xserve RAIDs connected to 12 clients projecting onto their tiled wall (that's 12 streams in all). Lots of accolades from the people who set up the challenge. More links to articles and reviews can be found at the Envision Center Bandwidth Challenge FAQ page."
The two-minute video is a scientific visualization of a cell structure from a bacterium.
The Envision Center site hosts a reduced version of the video.
We all hope that this marvel of IT world will be used for things better than video streaming!
Maybe Computers will never be as intelligent as Humans.
For sure they won't ever become so stupid. [VR-1988]
large telcoms will stop raping the little guy for bandwith?
Help me out: using 75% of a 10Gb/s link "rocks"? If I read that right, usage was 7.5Gb of 10Gb total. What exactly was the Feat? (No, it's not in the Player's Manual.)
This just proves that porn really is the driving factor in Internet growth. Cells are the basic building blocks of people and people are what make (most) porn, porn.
---- aut viam inveniam aut faciam
Daring to brave the power of this fully operational Slashdotting station.
(Then again, I got a 14-megabyte .MPG of the movie in less time than it took to post this. We'll see if they're still alive by the time this hits the board, though.)
Hmmm, looks a bit blocky to me. I think they need more key-frames, and less compression.
Get your own free personal location tracker
...brute force. What an impressive innovation!
We don't need no stinkin reduced version!
I'm sure our employers wouldn't mind if we took a look at the full version.
*psst* *psst* *psst* *mumble* *mumble* *mumble*
The whole thing? Really?
My boss has told me to take the full version of my personal desk stuff home now.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
There is significant overhead associated with the use of TCP/IP. A typical 6.0 Mb/s connection will deliver appx. 4.2 Mb/s this is only about 70% of the connections actual bandwidth. So, 75% is looking pretty good.
What rocks is the ability to reliably deliver 7.5 Gb/s AND do something useful with it.
That would have been worth watching if they were streaming an HD version of Snakes on a Plane.
- Just my $0.02, take with a grain of salt, your mileage may vary.
2 minutes long & 125-GB
"A resolution of 4096x3072, with 24-bit color, running at 30 frames per second,"
What codec did they use?
[Fuck Beta]
o0t!
A 2-minute-long, 125-GB movie... that must have been one super-high resolution chicken.
Developers: We can use your help.
That's not that impressive in the scheme of things (ie Supercomputing 2006). We (Vanderbilt) were there, and we were streaming 35 Gb/sec for hours on end over a parallel filesystem we're developing. And even we weren't the fastest there. Going to Supercomputing is like stepping 3-4 years into the future.
Two minutes??? But I want it noooooow!
Why in the world do they need 7.5Gb/sec video streaming? Are they ramping up for a hi-def knockoff of Burger King's silly idea?
Now if only my hard drive can cope up fast enough. Oh yeah who cares about hard drive if you can just get it somewhere and have it played 7.5 gb!
The only thing I see every day is my laptop dying on me. http://www.op3r.com
I have a friend who just got back from the same conference. He works at Ohio State University in the electron optics facility (read: likes to play with awesome equipment). Here's what he sent along to me: "This week I am in Tampa, Fl at the Supercomputing Conference "SC 06" with the guys from OSC (Ohio SuperComputer group). We have connected the Global Link boxes up and I am attempting to run the Quanta from Florida. Traffic is being routed from the conference to Atlanta, the Internet2, then though to the Third Frontier Network and back to OSU." With the bandwidth he had, he was able to observe, interact, and analyze a sample in a scanning electron microscope (the Quanta) in Ohio, in real time. Now that's a use for bandwidth; it's a lot cheaper to have a great internet connection than to buy the latest electron microscopes.
Am I missing something? A 2 minute, 125GB movie streamed at 7.5GB/second.
125 / 7.5 = 16.67 seconds.
Was this a 16 - 17 second video? Was it truly "streamed". Maybe I just don't understand something stupid.
Not to mention the hardware itself (NICs, routers, bridges, etc.) adds some not-insignificant overhead as well.
Right. While any idiot can get 10 Gb/s link and get 7.5 Gb/s or more out of it, the real feat here was in doing something useful with it at the same time. Remember that in streaming video applications, there's typically a lot of dropped packets because the client has to actually do something with the video immediately. It may just buffer it, but the application that's doing the buffering is often busy doing other stuff as well, like, say decoding video and audio streams and actually piping all that data through to the I/O bus -- oh, and the NIC is usually sitting on that same I/O bus, so that makes things even worse.
My blog
When you try and fill a 10g pipe with a single tcp session, the congestion avoidance mechanisms of tcp will prevent you from filling the pipe. Essentially the sender will ramp up the rate of packets very quickly initially until the receiver sends back a congestion notification. The sender will then cut the send rate *in half*, and climb it back up very slowly - 1 extra byte per round-trip if memory serves (don't quote me on that). This works great for 100m, but to climb from 5g to 10g takes about 30 minutes if you have a cross-US round-trip-time (RTT).
e d_issues/ipj_9-2/gigabit_tcp.html
To get around this you can:
1. Patch your TCP stacks with a few high-performance modifications
2. Figure out - using the RTT, interface buffer sizes, and bandwidth - what the number of outstanding packets can be before the receiver sends back a "slow down" message. Then configure the sender to have a smaller packet queue.
Great article on this here:
http://www.cisco.com/web/about/ac123/ac147/archiv
It's tough to say if that was the problem here (I'm actually assuming it was not) since after a little digging I didn't see any details on their implementation. And no, I'm not interested in truly digging (I have a pesky job thingy to get back to).
Oh believe me, the Envision Center doesn't really do all that much that's truly "useful". Cool as hell, and even some stuff my work is tangentially related to, but not "useful".
(Ex: immersive, interactive 3-d environment to teach math skills in American Sign Language. Really neat stuff. But until deaf schools get $10 million projection studios, um. Not "useful".)
There is significant overhead associated with the use of TCP/IP. A typical 6.0 Mb/s connection will deliver appx. 4.2 Mb/s this is only about 70% of the connections actual bandwidth.
While there is overhead associated with TCP/IP, it's nowhere near 30%. On a 100 Mbit link in a LAN, you routinely get 11 MB/s (just verified by transferring an Ubuntu image via FTP over the local ethernet with noname switching hardware). With a theoretical maximum of 12.5 MB/s, that's an efficacy of 90%.
6 Mbit sounds like a DSL connection to me. Quite possibly your provider or the servers you download from are responsible for your effective 4.2 Mbit, because TCP/IP isn't.
--- The light at the end of the tunnel is probably a burning truck.
When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
..what happens to your own personal Internet?
one really fast way to squirt a video.
wake me up when they do a live test using Jenna Jameson's film "Last Woman Standing".
http://ucsdnews.ucsd.edu/thisweek/2005/oct/10_03_v ideo.asp
10 Gbps is no big deal nowadays.
Anyone who really into supercomputing (or computing for that matter) would not make such liberal use of the phrase 'real time.'
Check out my lame java blog at www.javachopshop.com
Can someone post a link to the video in question please?
I want to see this in all its 125GB glory.
Thanks.
Marge: Does the world really need that much porno?
Homer: {{drool}} One Million Times faster
If Nalgene water bottles are outlawed, only outlaws will have Nalgene water bottles.
Is there some sort of practical use for this, or is this just the academic IT version of a soccer mom driving a hummer to go get her hair done? A huge waste of electricity "just because"? This is a legit question,I am not trying to be snarky at all, *what* is this for? If it is just for fun, like drag racing is totally impractical but fun for the racers, I can see that, but if it is to have some sort of practical reason..what is it? If it is just for fun, can they just own up to it, no matter who paid for it?
Actually it is that high. The faster you get, the higher the overhead percentage.
If the size of the packets scaled with the bandwidth then the overhead percentage would remain constant.
10gbps is around 150,000 packets (9k jumbo packets) per second. 100mbps is 1,500 packets per second (also using jumbo packets).
As you can see the overhead increases with the bandwidth.
Yeah, but STREAMING isn't TCP, its UDP, which is why they get 75%, but over all, you are on the right track.
Any video streaming without MPAA consent is just asking for trouble.
They will argue that setting link like that could be used for other unlawful activities.
In the future I would suggest checking out with MPAA first and using video delivery methods approved by them.
I have a 1 Gb line running between my computers, and the top speed I have observed is around .1 to .2 Gb/s. Where is this very low bottleneck I am hitting? Is this in the HD transfer speed?
She?
Jonathanjk.com
Within a year, they will be streaming the most vile pornography allowed by human morals.
I suggest you read Slashdot
6.0 Mb/s connection? 30% overhead? Are you basing your findings off your cable modem performance? 25-30% overhead is terrible. You're doing something wrong. When testing a couple 10Gb cards through a few different switches, the most overhead I saw was maybe 8-10%, tops.
Also, there are several applications where a 10Gb connection would be used, so I don't think the fact that they used the 7.5 gigs is very impressive. To me it's the fact that they pushed video that played 7.5 Gbps. Now THAT would make for some great HDTV.
No way. Everyone knows Porn >> Games in the scheme of cosmic importance!
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
What rocks is the ability to reliably deliver 7.5 Gb/s Yeah but i can do that by kicking a DVD across the floor, that's well over 7.5 Gb/m/s AND do something useful with it. Oh.
I walk by the Envision Center at Purdue several times a week for class or work (I work at the building where Envision Center is located). Despite being at Purdue for 5 years, I've never actually gone into the Envision Center. They do have large LCD displays in the hallway by the entrance showing 3D molecules rotating, etc... If I recall, millions of dollars went into the creation of the Envision Center. I didn't realize they were doing cool things like this. I'll have to stop by there next week and see what it's all about.
Does someone have a torrent or a coral cache?
75% efficiency IP over IB is actually pretty good. I can get about ~330MB/sec out of an I/O node (storage capability and file system depending of course) after tuning. Seeing upwards of 1GB/sec out of an SDR IB link... that's pretty impressive.
I was at sc06 last week. I am working (as a research assistant) at that National Center for Datamining, University of Illinois at Chicago. We won this bandwidth challenge by transmittting data using the UDT protocol. The results are here https://scinet.supercomp.org/2006/bwc/ (click on the NCDM link).
How many low resolution, high compression youtube videos is this equivalent to?
You really dont need such datarates, even for the newest, best, ect ones.
You simple dont get that much information out of electron detectors.
If you want to push bandwith, you need high sample rates and get "real time" rubbish noisy shit.
And for good statistics, 1-2 Mbit are more than enough. You arent playing Maxwells Daemon, you know, so there is no atom to catch or something....
At least that my opinion, as someone who was also dissapointed the first time he noticed that the 3 million $ SEM only outputs XGA Tiffs with 1024x786, 8bit greyscale, once every 5 seconds if you want nice statistics....
HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
Boilers know, and are good at just about everything!
The MPAA are set to reveal later today their new SLT (Streaming Lawsuit Technology) offering. This should make the extortion process easy and seamless for end-users.
Just because you're paranoid doesn't mean there isn't an invisible demon about to eat your face
I've seen this quite a lot recently here, linking videos from the front page and they are in
Please, this is
605413? Yes, it's a prime.
While trying this out with Real Player the Purdue team still got a buffering warning.
And 640K ought to be enough for anybody.
Good luck taking down Purdue's servers. Nobody is there right now :x ;)
I'm sure the purdue student body regularly finds a good use of 7.5Gb/s transfer, although most of it doesn't even need to go outside of the campus
I smell a disturbance in the Force; as if a 1000 MPAA execs all suddenly crapped their pants at the same time.
Not really. I agree with the part about the 7.5 Gb/s (as long as you are using PCI-e 10 Gb NICs as opposed to PCI-X). You can very easily get about 8.2 Gb/s running from memory to memory single stream...over 9 with multiple streams. What is difficult is getting it from disk that quickly. While it is interesting, it really is a toy experiment when compared to a super computer. Running parallel file transfers on thousands of nodes is where the real challenge lies and parallel file systems just don't work as well at that scale.
Support a great indie game: http://www.abaddon360.com
This is hardly interesting, this isn't an accomplishment of any significance. Not are others claiming this has been done before, its really not very difficult.
The most difficult challenge, I think would be the matters of bus bandwidth and storage. As expected, the article makes note of the storage concerns, and they 'took the easy way out' and mirrored the data across several arrays.
What would be more impressive is if they used native infiniband storage arrays, could access them as a single NAS mount (rather than as JBOD) and even better if they could incorporate writing.
"The Envision Center site hosts a reduced version of the video."
Come on, this is slashdot, at least link to the full thing...
Task Mangler
I bet if they were using WMP to view it, they STILL saw an annoying BUFFERING! BUFFERING! BUFFERING! every couple seconds. Btw what's the point of that transfer speed if your RAM/hard drive can't have data fed into it that fast?
Google's Super Secret Search Algorithm: SELECT @search_results FROM internet WHERE @search_results = 'good'
I recently worked for a notable parallel storage hardware company.
The efficacy of tcp/ip with standard MTU frames is around 91%. You can get that up to 98% with jumbo frames.
But this isn't in a cluster.
They strapped 12 computers together in a cluster. Overhead for what they are doing is much higher. While not perfect, the 75% efficiency of a cluster is still impressive.
We used to obliterate this number with a couple racks of our product every testing cycle to test for regressions, but we spent 120 million dollars to get there.
If they can handle that kind of bandwidth then its pretty much worthwhile :p
Then shouldn't a project this big at least get a once over from the English department?
The line in the video that says: "an 1k stream" should be "a 1k stream".
Personally, I think this is hilarious...
Yeah a commercial broadband connection isn't a great comparison actually the speed tests that I've run have way to many variables in them and are not really representivie of the speeds/efficiencies avaiable in a LAN environment. After I put the post up it it also occurred to me that they are probably not even using TCP/IP, it's probably UDP.
However, a study that I read a while back (sorry no link) shows that MTU size becomes very important at higher data rates. Also, MTU size varies based on the application. Generally a faster network needs to be "tuned" to the proper MTU top achieve optimal performance.
With all that said we have no way of knowing how much optimization they did, but from my personal experience I can see how it would be easy to set up a network with 75% efficency and we don't even know if they were *trying* to use the full link speed!
Commodore64_love: I don't comprehend people who're so frightened of death that they'll bankrupt themselves to stay alive
How could they possibly reach 7.5 gbit with 6 Xserves, if each Xserve only has a 1 gbit ethernet connection?
:-)
This looks to me like some desperate attempt to justify the money they wasted on bad (Apple!) hardware, drugs and hookers
Last year we (SARA, Netherlands) did a 17Gbit/sec sustained (19.5Gbit top) stream over 2x 10Gbit link. The content was realtime generated on a 28 node render-cluster and streamed to a tiled display of 55 (11x5) 23" TFT panels. The image wasn't scaled or anything, just generated at that resolution (17.6k x 6k pixels). The previous year, 2004, we did a 5x3 version of the same thing. Check out www.optiputer.net for the concept and some pics and vids. So, sorry mate, but it's so 2004 :)
</big10joke>
The Independent: Reverend Spooner Arrested in Friar Tuck Incident - ISIHAC, Historical Headlines
Xserve RAID isn't the server, it is a 2 Gb fibre channel attached storage device. It can be attached to MacOS X, Linux, Netware, and Windows Servers. http://www.apple.com/xserve/raid/
Xserve is Apple's Server line, and that's just not what they are talking about. These servers were Dual Opteron systems with 2 1 Gb cards. It doesn't say what OS they are running.