Slashdot Mirror
Better Than Bit Torrent, For Internet2 Users?
FastDownload writes "New technology for doing mulitsource/multithread downloads of ISOs is making Linux users on Internet2 happy. It's called Logistical Networking and is being developed at the University of Tennessee. Though there are some obvious similarities to Bit Torrent, Logistical Networking uses fixed, shared infrastructure
instead of being peer-to-peer, which makes it useful for moving big content even when no peers
are available."
From this page you can see a graphic representation of the Application Layer and Local Layer this program works in (I2). From the description below we can see that this is more like every ISP making local copies of large files available!
Also, a Director for this stuff hints at it being a fee-based in the future. (More documentation here)
Internet2 is such a stupid name. It's just another high-speed network linked to the Internet. We technologically backward countries have similar schemes but don't have the audacity to call them Internet2. In the UK, the high speed academic network is called JANET, for example - and they do quite a bit of research-type things with high speed links. Even has extensive peering with Internet2 IIRC.
LINUX USERS ON INTERNET2 NETWORKS ENJOY THE BENEFITS OF LOGISTICAL NETWORKING
PHOENIX, AZ - November 17, 2003 - Linux users on Internet2 networks are enjoying the benefits of a new approach to high performance content distribution, called Logistical Networking, which will be on display this week in the Internet2 booth at SC2003 in Phoenix, AZ. Developed by a research team from the Logistical Computing and Internetworking (LoCI) Laboratory at the University of Tennessee, Logistical Networking (LN) combines state-of-the-art data transfer technology with storage resources provisioned throughout the network to create a convenient and powerful new paradigm for distributed data management.
To test this technology, the LoCI team has used the 22 terabyte (TB) testbed of LN "depots" deployed across Internet2 networks to create an ad hoc content distribution network for distributing 650 megabyte (MB) CD-images (called "ISO's") of Linux and FreeBSD software. Users are now employing the Logistical Runtime System (LoRS) tools to download these ISO's at speeds of 30 to 80 megabits per second (Mbps)--roughly tens times faster than from traditional HTTP or FTP mirror sites. Downloads for gigabit Ethernet connected users can exceed 150 Mbps.
"We think that this kind of network storage infrastructure paves the way for a new era in content distribution," said Dr. Micah Beck, Co-Director of LoCI Laboratory and the chair of Internet2's Network Storage Special Interest Group. "For example, although using multiple copies and multiple TCP streams to increase transfer speed is similar to what some peer-to-peer systems do, with our fixed but shared infrastructure of well connected nodes, you can scale up the size of the content without sacrificing performance."
What makes this unique combination of flexibility and performance possible is an XML encoded metadata file called an exNode. A content publisher who uploads a file to the testbed of LN depots, which is called the Logistical Backbone (L-Bone), receives an exNode containing metadata that maps the segments of the file's content to L-Bone storage allocations, which are time-limited to make them more shareable. A single exNode can represent content that has been fragmented across multiple depots to accommodate large sizes, replicated to ensure fault tolerance, or both replicated and geographically dispersed to improve accessibility and performance. A single exNode used to distribute a Linux ISO represents eight copies of the ISO's content, which has been broken up into 8-20MB chunks and spread across L-Bone depots nationwide.
Publishing content that has been stored in the L-Bone is as simple as sharing the exNode that represents it. Since exNodes are text files, they can be published via HTTP, sent as e-mail attachments, or passed on a floppy disk. When they are posted on a Web site, as with the exNodes for Linux ISOs, the result is called an exNode Distribution Network, or XDN. To access the content in an XDN, users simply retrieve the relevant exNode from the site, and then use them with the LoRS tools to download the content. The LoRS tools are freely available and easy to set up, have a convenient GUI, and run on Microsoft Windows, Apple OS X, and all common variants of UNIX and Linux. The LoRS tools make fast, mulitsource/multistream downloads routine for Internet2 users when the content is suitably replicated, as in the Linux XDN,
"As compared to some of the other things we're doing with Logistical Networking, like managing the data sets from supercomputer simulations or accelerating remote browsing of massive image databases, putting up an XDN is a pretty simple application that anyone on Internet2 can do," explains Dr. Beck. "And prototype applications like IBPvo show that there are some easy variations on XDN that can automate different parts of the process."
IBPvo is an Internet2-enabled personal video recorder (PVR) service created to show the flexibility of LN technology. Like TiVo or ReplayTV, IBPvo can be scheduled in advance
This is like bit torrent, but not any way close to the redundancy of users. Right now there are only a few servers that host a file. There isn't a need for much more because A. the network doesn't have that many people B. there is so much bandwidth. There is no organized p2p warez network setup. It is in the experimental phase and only used now for research applications.
As stated in previous posts, Internet2 is only accessible by academic institutions and corporations. I'm just a student at a university that is connected to the internet2 and I use LoRS for a smaller cooler application where I download recorded tv shows that I can select to be recorded at a central server.
And you know what? That didn't stop my university from blocking all packets over the bittorrent protocol! Bastards.
I Browse at +4 Flamebait
Open Source Sysadmin
As mentioned earlier, Internet2 is NOT what it sounds like. It's just another high-speed WAN, like the commercial backbones, but because it's *cough* academic only *cough* it's got a lot less traffic and there are no artificial speed bumps.
See when you get an internet connection, of any kind, you usually get wired up for a LOT more bandwidth than you buy, but the ISP caps it to make a market.
On the Internet2 there are different grades of connection, but a huge number of schools are chatting to each other at 155Mbps, full-rate ATM. If you're in a dorm at any of those schools on the I2, chances are some of your regular ho-hum web/p2p traffic flows through the I2.
The real advantages of the I2 project are reduced school ISP costs, as inter-school traffic doesn't have to traverse a commercial line, and much better collaborative access for research, development, and distance-learning.
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
Aww, good for those Internet2 users. All 15 of them.
There are around 130,000 Internet2 (actually Abilene) at my university
Finkployd
LoRS Tools are on Freshmeat
The download link works ok - it seems the slashdotting has merely taken out dynamic HTML generation, not the bandwidth.
Apparently it's under a BSD License - IMHO quite suitable for a publicly funded project. (Flamewar ensues...)
Any sufficiently advanced libertarian utopia is indistinguishable from government.
If you think bittorrent is too adhoc, you've never been to 3dgamers. For every game demo or movie they provide information about, they provide a bittorrent seed. In addition, they do provide direct download mirrors, but I don't even bother anymore.
Another tip: The official bittorrent client isn't that great. You should try Azureus. It's written in Java, which sucks (flame me, I bite back), but even so I love it. In fact it might be the only java program that I like now that I think about it.
LoCI deals with this by having multiple copies of the data spread around (mirroring in your RAID analogy). So if one site goes down, no data will be lost. For instance, "A single exNode used to distribute a Linux ISO represents eight copies of the ISO's content, which has been broken up into 8-20MB chunks and spread across L-Bone depots nationwide."
Unfortunately, most campuses use Packeteer or other packet-shaping devices to analyze the packets to determine the traffic type in order to throttle bandwidth-hogging applications rather than blocking ports explicitly. While port-changing tricks may have worked in the past, even the most incompetent administrator can set up one of these, and no matter how many ports you try, you're not getting around it.
Yep, been there, done that. :)
Our research/education network here in Sweden called SUNET (Swedish University NETwork) runs at 10 Gbit/s in the core and providing all connecting nodes with minimum 2 GB/s trunks, all the major universities are directly connected with 10 GB/s trunks.
--AFAIK, Multicast only works if everyone who wants the content, wants it at the EXACT SAME TIME.
.
== WolfriderV6 == I'm willing to admit that *I just might* be wrong... Are you??