Better Than Bit Torrent, For Internet2 Users?

FastDownload writes "New technology for doing mulitsource/multithread downloads of ISOs is making Linux users on Internet2 happy. It's called Logistical Networking and is being developed at the University of Tennessee. Though there are some obvious similarities to Bit Torrent, Logistical Networking uses fixed, shared infrastructure instead of being peer-to-peer, which makes it useful for moving big content even when no peers are available."

Comments

P2P is NOT Going Anywhere

[Preach the Good Word]

Especially for legal content... bit torrent has made it so that you can get all sorts of legal content like game demos, linux distros, etc. off p2p without having to be on horribly slow ftp servers.

BitTorrent is too ad-hoc

[Space cowboy]

I was considering setting up a download of a database dump for hostip.info using BitTorrent, but it's too awkward to create, and there's no guarantee that there's any saving, as far as I can see (people turn their machine off, and you're stuck waiting for a chunk in the middle). Instead, I let people download the meta-data, and construct the DB - much faster :-)

The idea of fixed nodes is less "cool" I guess, with less of the "dynamic network adapting to the load", but probably more useful...

Simon.

Re:BitTorrent is too ad-hoc

[halr9000]

If you think bittorrent is too adhoc, you've never been to 3dgamers. For every game demo or movie they provide information about, they provide a bittorrent seed. In addition, they do provide direct download mirrors, but I don't even bother anymore.

Another tip: The official bittorrent client isn't that great. You should try Azureus. It's written in Java, which sucks (flame me, I bite back), but even so I love it. In fact it might be the only java program that I like now that I think about it.

Dear Santa

[Anonymous Coward]

Please put me on Internet2. Internet1 is old and busted. I've seen all the pr0n, downloaded all the songs, and I'm tired of getting viagra penis spam 418 times a day.

I'll be a good boy next year, you can believe it.

Thanks in advance,
Johnny

Give it up for the coward!

[Anonymous Coward]

LINUX USERS ON INTERNET2 NETWORKS ENJOY THE BENEFITS OF LOGISTICAL NETWORKING

PHOENIX, AZ - November 17, 2003 - Linux users on Internet2 networks are enjoying the benefits of a new approach to high performance content distribution, called Logistical Networking, which will be on display this week in the Internet2 booth at SC2003 in Phoenix, AZ. Developed by a research team from the Logistical Computing and Internetworking (LoCI) Laboratory at the University of Tennessee, Logistical Networking (LN) combines state-of-the-art data transfer technology with storage resources provisioned throughout the network to create a convenient and powerful new paradigm for distributed data management.

To test this technology, the LoCI team has used the 22 terabyte (TB) testbed of LN "depots" deployed across Internet2 networks to create an ad hoc content distribution network for distributing 650 megabyte (MB) CD-images (called "ISO's") of Linux and FreeBSD software. Users are now employing the Logistical Runtime System (LoRS) tools to download these ISO's at speeds of 30 to 80 megabits per second (Mbps)--roughly tens times faster than from traditional HTTP or FTP mirror sites. Downloads for gigabit Ethernet connected users can exceed 150 Mbps.

"We think that this kind of network storage infrastructure paves the way for a new era in content distribution," said Dr. Micah Beck, Co-Director of LoCI Laboratory and the chair of Internet2's Network Storage Special Interest Group. "For example, although using multiple copies and multiple TCP streams to increase transfer speed is similar to what some peer-to-peer systems do, with our fixed but shared infrastructure of well connected nodes, you can scale up the size of the content without sacrificing performance."

What makes this unique combination of flexibility and performance possible is an XML encoded metadata file called an exNode. A content publisher who uploads a file to the testbed of LN depots, which is called the Logistical Backbone (L-Bone), receives an exNode containing metadata that maps the segments of the file's content to L-Bone storage allocations, which are time-limited to make them more shareable. A single exNode can represent content that has been fragmented across multiple depots to accommodate large sizes, replicated to ensure fault tolerance, or both replicated and geographically dispersed to improve accessibility and performance. A single exNode used to distribute a Linux ISO represents eight copies of the ISO's content, which has been broken up into 8-20MB chunks and spread across L-Bone depots nationwide.

Publishing content that has been stored in the L-Bone is as simple as sharing the exNode that represents it. Since exNodes are text files, they can be published via HTTP, sent as e-mail attachments, or passed on a floppy disk. When they are posted on a Web site, as with the exNodes for Linux ISOs, the result is called an exNode Distribution Network, or XDN. To access the content in an XDN, users simply retrieve the relevant exNode from the site, and then use them with the LoRS tools to download the content. The LoRS tools are freely available and easy to set up, have a convenient GUI, and run on Microsoft Windows, Apple OS X, and all common variants of UNIX and Linux. The LoRS tools make fast, mulitsource/multistream downloads routine for Internet2 users when the content is suitably replicated, as in the Linux XDN,

"As compared to some of the other things we're doing with Logistical Networking, like managing the data sets from supercomputer simulations or accelerating remote browsing of massive image databases, putting up an XDN is a pretty simple application that anyone on Internet2 can do," explains Dr. Beck. "And prototype applications like IBPvo show that there are some easy variations on XDN that can automate different parts of the process."

IBPvo is an Internet2-enabled personal video recorder (PVR) service created to show the flexibility of LN technology. Like TiVo or ReplayTV, IBPvo can be scheduled in advance

mod_torrent is the way to solve this

[belphegore]

The way to go here is to make serving content through bittorrent easier. Ideally, as easy as publishing a resource on a web server (that is, copy a file into a directory and figure out what the URL is). This is the goal of the mod_torrent project. We're building an apache plugin on top of libtorrent which automatically creates torrents in response to http requests, and then begins serving them, in response to conditions on the apache server. Load is low? Fine, service with good-old HTTP the normal way. Load is high? Instead of a direct HTTP download, instead have the HTTP GET respond with an application/torrent file, which then launches bt to grab the content (all automatic). Goodbye slashdot effect.

Aww, how nice.

[Anenga]

New technology for doing mulitsource/multithread downloads of ISOs is making Linux users on Internet2 happy.

Aww, good for those Internet2 users. All 15 of them.

Re:Of course, the important question is....

[MarcQuadra]

As mentioned earlier, Internet2 is NOT what it sounds like. It's just another high-speed WAN, like the commercial backbones, but because it's *cough* academic only *cough* it's got a lot less traffic and there are no artificial speed bumps.

See when you get an internet connection, of any kind, you usually get wired up for a LOT more bandwidth than you buy, but the ISP caps it to make a market.

On the Internet2 there are different grades of connection, but a huge number of schools are chatting to each other at 155Mbps, full-rate ATM. If you're in a dorm at any of those schools on the I2, chances are some of your regular ho-hum web/p2p traffic flows through the I2.

The real advantages of the I2 project are reduced school ISP costs, as inter-school traffic doesn't have to traverse a commercial line, and much better collaborative access for research, development, and distance-learning.