Slashdot Mirror
Better Than Bit Torrent, For Internet2 Users?
FastDownload writes "New technology for doing mulitsource/multithread downloads of ISOs is making Linux users on Internet2 happy. It's called Logistical Networking and is being developed at the University of Tennessee. Though there are some obvious similarities to Bit Torrent, Logistical Networking uses fixed, shared infrastructure
instead of being peer-to-peer, which makes it useful for moving big content even when no peers
are available."
Is there pr0n on this so called internet 2?
I think the Internet2 users should use Kazaa Lite and eMule just like the rest of us. Throw caution to the wind, screw the RIAA.
---
DRM is like antifreeze, to the MPAA/RIAA it's sweet, to the consumers it's poison.
Especially for legal content... bit torrent has made it so that you can get all sorts of legal content like game demos, linux distros, etc. off p2p without having to be on horribly slow ftp servers.
I was considering setting up a download of a database dump for hostip.info using BitTorrent, but it's too awkward to create, and there's no guarantee that there's any saving, as far as I can see (people turn their machine off, and you're stuck waiting for a chunk in the middle). Instead, I let people download the meta-data, and construct the DB - much faster :-)
The idea of fixed nodes is less "cool" I guess, with less of the "dynamic network adapting to the load", but probably more useful...
Simon.
Physicists get Hadrons!
I'll be a good boy next year, you can believe it.
Thanks in advance,
Johnny
From this page you can see a graphic representation of the Application Layer and Local Layer this program works in (I2). From the description below we can see that this is more like every ISP making local copies of large files available!
Also, a Director for this stuff hints at it being a fee-based in the future. (More documentation here)
You'll get is when you pay cubic buttloads of cash to hook yourself up. You think that $45 Cable ISP fee is arbitrarily set?
"Draco dormiens nunquam titillandus."
Internet2 is such a stupid name. It's just another high-speed network linked to the Internet. We technologically backward countries have similar schemes but don't have the audacity to call them Internet2. In the UK, the high speed academic network is called JANET, for example - and they do quite a bit of research-type things with high speed links. Even has extensive peering with Internet2 IIRC.
That is definitely not true. Bit Torrent has a lot of legitimate uses. I used Bit Torrent to download an ISO for clusterknoppix and the multi-disk ISO's for red hat when their servers were swamped. Bit Torrent is a great concept that is not going to be going away. It's sort of like multicasting over the traditional internet structure. I see BT only growing is variations and uses in the future both for good and bad.
Belive in Technology and AMAZE yourself. -- RIP ZDTV/TechTV
You stupid fuck, piracy isn't killing it, its the reason it exists. Do you think you would know what "p2p" stood for if it wasn't for napster and kazaa? File sharing tech has always been driven by piracy, legitimate uses have always come second.
Oh, and as soon as Freenet gets N.G Routing working nicely, BitTorrent will be obsolete [/flamebait] ;-)
Internet.1 is limited only by your mind, those who seek internet.2 will not find it.
I'd hardly call this "better than bittorrent". While the principles may be similar, the target users are entirely different.
Bittorrent is for people who can barely afford to run their one server, and need others to take some of the load off.
This seems to be targeted at people who can set up a whole bunch of servers in a bunch of locations, and just want to use them efficiently to deliver huge content very quickly.
"The worst tyrannies were the ones where a governance required its own logic on every embedded node." - Vernor Vinge
LINUX USERS ON INTERNET2 NETWORKS ENJOY THE BENEFITS OF LOGISTICAL NETWORKING
PHOENIX, AZ - November 17, 2003 - Linux users on Internet2 networks are enjoying the benefits of a new approach to high performance content distribution, called Logistical Networking, which will be on display this week in the Internet2 booth at SC2003 in Phoenix, AZ. Developed by a research team from the Logistical Computing and Internetworking (LoCI) Laboratory at the University of Tennessee, Logistical Networking (LN) combines state-of-the-art data transfer technology with storage resources provisioned throughout the network to create a convenient and powerful new paradigm for distributed data management.
To test this technology, the LoCI team has used the 22 terabyte (TB) testbed of LN "depots" deployed across Internet2 networks to create an ad hoc content distribution network for distributing 650 megabyte (MB) CD-images (called "ISO's") of Linux and FreeBSD software. Users are now employing the Logistical Runtime System (LoRS) tools to download these ISO's at speeds of 30 to 80 megabits per second (Mbps)--roughly tens times faster than from traditional HTTP or FTP mirror sites. Downloads for gigabit Ethernet connected users can exceed 150 Mbps.
"We think that this kind of network storage infrastructure paves the way for a new era in content distribution," said Dr. Micah Beck, Co-Director of LoCI Laboratory and the chair of Internet2's Network Storage Special Interest Group. "For example, although using multiple copies and multiple TCP streams to increase transfer speed is similar to what some peer-to-peer systems do, with our fixed but shared infrastructure of well connected nodes, you can scale up the size of the content without sacrificing performance."
What makes this unique combination of flexibility and performance possible is an XML encoded metadata file called an exNode. A content publisher who uploads a file to the testbed of LN depots, which is called the Logistical Backbone (L-Bone), receives an exNode containing metadata that maps the segments of the file's content to L-Bone storage allocations, which are time-limited to make them more shareable. A single exNode can represent content that has been fragmented across multiple depots to accommodate large sizes, replicated to ensure fault tolerance, or both replicated and geographically dispersed to improve accessibility and performance. A single exNode used to distribute a Linux ISO represents eight copies of the ISO's content, which has been broken up into 8-20MB chunks and spread across L-Bone depots nationwide.
Publishing content that has been stored in the L-Bone is as simple as sharing the exNode that represents it. Since exNodes are text files, they can be published via HTTP, sent as e-mail attachments, or passed on a floppy disk. When they are posted on a Web site, as with the exNodes for Linux ISOs, the result is called an exNode Distribution Network, or XDN. To access the content in an XDN, users simply retrieve the relevant exNode from the site, and then use them with the LoRS tools to download the content. The LoRS tools are freely available and easy to set up, have a convenient GUI, and run on Microsoft Windows, Apple OS X, and all common variants of UNIX and Linux. The LoRS tools make fast, mulitsource/multistream downloads routine for Internet2 users when the content is suitably replicated, as in the Linux XDN,
"As compared to some of the other things we're doing with Logistical Networking, like managing the data sets from supercomputer simulations or accelerating remote browsing of massive image databases, putting up an XDN is a pretty simple application that anyone on Internet2 can do," explains Dr. Beck. "And prototype applications like IBPvo show that there are some easy variations on XDN that can automate different parts of the process."
IBPvo is an Internet2-enabled personal video recorder (PVR) service created to show the flexibility of LN technology. Like TiVo or ReplayTV, IBPvo can be scheduled in advance
They've got the cash,
But we've got the numbers.
--> Insert Funny Sig Here
Shut it, bitch. I bet you think Kirk is better than Picard too.
Unfortunatly, like the original Internet, it won't take all that long for it to be flooded with riff-raff.
I remember a time when I knew literally EVERY English speaking person that used IRC. In the entire world. All 100 or so of them.
That wasn't THAT long ago. Only 15 years or so.
"Champagne for my real friends - and real pain for my sham friends!" http://ericblade.postalboard.com/
It looks more like Freenet or (the defunct) MojoNation to me, except without the constraints imposed by an anonymity guarantee.
Will I retire or break 10K?
This sounds a bit similair to drftpd, Distributed FTP Daemon. Although drftpd does not stripe data at the protocol level, you can use some kind of splitting tool such as RAR with multiple archives to stripe data across nodes. I'd implement striping at protocol level but that would break site-to-site FTP transfers and would require improved protocol and FTP clients. GridFTP might be better for it anyway.
This new thing looks like each site has a piece of your data. If one site loses everything, then wouldn't any file that had even a piece of it in that site be forever lost? Sounds like a distributed RAID0 (stripe) array to me. And everyone knows that reliability of those goes down as you add more stripes....
You think the Internet2 is cool?
Wait until Paris Hilton sex tape #2 hits the P2P networks on OUR internet.
How you gonna DEAL WITH THAT, INTERNET2? HOW? YOU CAN'T TOUCH THIS.
We have secretly replaced these Slashdot mods' sense of humor with a rusty nail. Let's see if they notice!!
Sure it'd be great if Freenet was working nicly, but that's a big if. It's been in development for more than 3 years now and they're still not even close to having a network that's ready for use by the masses. I'm not sure it'll ever get to that point either. One of the golden rules of P2P is that bandwidth is precious, yet Freenet merily uses n times the amount of data being moved in bandwidth passing that data up a chain of nodes to preserve anonimity. This is probably going to forever relagate Freenet to solution-in-search-of-a-problem staus as the vast majority of people arn't that worried about anonimity.
This is like bit torrent, but not any way close to the redundancy of users. Right now there are only a few servers that host a file. There isn't a need for much more because A. the network doesn't have that many people B. there is so much bandwidth. There is no organized p2p warez network setup. It is in the experimental phase and only used now for research applications.
As stated in previous posts, Internet2 is only accessible by academic institutions and corporations. I'm just a student at a university that is connected to the internet2 and I use LoRS for a smaller cooler application where I download recorded tv shows that I can select to be recorded at a central server.
Kirk would kick Picards little French butt all over the vinyard. Then he'd bang all the whores in Paris.
Pennsylvania already tried this type of mega-giant fiber plans, but got screwed by the telecoms. Hopefully the Morons with their genius city planning (seriously, they plan cities better than any one, well on a grid) will do this better.
What they paid: $2.1B + $3B extra by 2015
What they got: crap
What they paid for:
"This capability falls short of Verizon's commitment; Verizon committed to providing two-way broadband capability of at least 45 Mbps," noted Pennsylvania Public Utility Commission member Terrance J. Fitzpatrick in a statement. "The Commission accepted this commitment, and approved the Plan based upon that commitment."
The way to go here is to make serving content through bittorrent easier. Ideally, as easy as publishing a resource on a web server (that is, copy a file into a directory and figure out what the URL is). This is the goal of the mod_torrent project. We're building an apache plugin on top of libtorrent which automatically creates torrents in response to http requests, and then begins serving them, in response to conditions on the apache server. Load is low? Fine, service with good-old HTTP the normal way. Load is high? Instead of a direct HTTP download, instead have the HTTP GET respond with an application/torrent file, which then launches bt to grab the content (all automatic). Goodbye slashdot effect.
Aww, good for those Internet2 users. All 15 of them.
A story about Internet2 networking technology kills Internet1 web server in minutes flat.
Anyone have a torrent link for the Paris Hilton video? (yeah, i still haven't seen it.) thx.
Because I2 wasn't fast enough requiring that everyone on it be in relative proximity to a gigapop.
Who needs mirrors? Can't somebody just seed a copy on bittorrent for Internet2?
As mentioned earlier, Internet2 is NOT what it sounds like. It's just another high-speed WAN, like the commercial backbones, but because it's *cough* academic only *cough* it's got a lot less traffic and there are no artificial speed bumps.
See when you get an internet connection, of any kind, you usually get wired up for a LOT more bandwidth than you buy, but the ISP caps it to make a market.
On the Internet2 there are different grades of connection, but a huge number of schools are chatting to each other at 155Mbps, full-rate ATM. If you're in a dorm at any of those schools on the I2, chances are some of your regular ho-hum web/p2p traffic flows through the I2.
The real advantages of the I2 project are reduced school ISP costs, as inter-school traffic doesn't have to traverse a commercial line, and much better collaborative access for research, development, and distance-learning.
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
Freenet merily uses n times the amount of data being moved in bandwidth passing that data up a chain of nodes to preserve anonimity.
That's the whole point of Freenet! It's not designed to be the easiest way to get the latest DVD rips, it's designed to be a way to communicate 100% anonymously, for example if you're living in a repressive regime, and you need to send a message without getting killed.
LoRS Tools are on Freshmeat
The download link works ok - it seems the slashdotting has merely taken out dynamic HTML generation, not the bandwidth.
Apparently it's under a BSD License - IMHO quite suitable for a publicly funded project. (Flamewar ensues...)
Any sufficiently advanced libertarian utopia is indistinguishable from government.
Yes, if you absolutly positivly need to be anonymous Freenet is the way to go. But the parent was talking about Freenet making Bittorrent obsolete, which is never going to happen because Freenet has to sacrifice too much efficency to maintain anonimity.
here (temporarily)
Bandwidth sponsored by danish research funding.
Any sufficiently advanced libertarian utopia is indistinguishable from government.
imo all three posters are correct; mixing bt into apache should allow a simple process for sharing a number of files (ideally just dropping them into a dir & running some indexer?), whereas one would otherwise need a running bt (seeder) for each shared file (other solutions?) all the time.
seems to be good for 0day stuff and rare files alike.
though making a browser render a page out of a bt tmp-download dir might be possible, surfing exp would probably be nowhere near 'fast'.
That's much better, JANET conjures up a far more exciting mental image, where can I get JANET?
People who only half-assedly care about publishing their material don't need to do this, of course, and your experience with stuff getting aborted in the middle is probably for popular material like anime or perhaps concert recordings where the people who publish it take it down when it's no longer "current". There's no reason that this has to happen for your own garage band's main CDs, even though you might not keep every show on line permanently.
On the other hand, an obvious feature for BitTorrent to add, if it doesn't have this already, is a "De-Publish" option which will let people who have part of the file get the rest of it but won't serve new download sessions to people who haven't already started. That'd be enough to give a day's warning when taking something down. You could either implement it mandatorily in the server or permissively in the client (e.g. make the server enforce it, which is more work, or let the client tell the user that the flag is set for 3.27 hours from now [Continue?][GiveUp?].
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
yes, I've been here long enough to know that one.
I just found it quite amusing in the context. I need to submit that one to fark with IcyHotStuntaz composited in front of some Sunfire K15s all 'glistnin' with chrome, ST fibre and cold-cathode effects.
"If I wanted your input on my pet project, I'd stick my hand up your ass and use you like a sock-puppet." - Muse
This project seems to be about two things
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Slashdot's first reaction to VMware
I'm glad to see the LoCI guys getting some recognition. I've been using their tools for quite some time and they are useful well beyond downloading ISOs. Data movement is an important issue, especially in distributed environments, such as the Internet and grids. The packages that LoCI has released are a very strong solution to the data movement problem.
I'm at a university (Columbia) which is connected to Internet2. It automatically routes (as far as I know) across Internet2 when appropriate, which means to students at the other schools on Internet2. And we all know who P2P's biggest users are... so I've been using BT nonstop to great effect here. Also, Columbia's idiotic bandwidth policies (can't upload >100MB/hr or they administratively sever your ethernet port) don't apply to data across Internet2. So I usually don't get in trouble letting the upstream bandwidth run free on bittorrent. I frequently observe 500kb/s on a single torrent. Not bad for a 10mbit dorm connection (room-to-room speeds max out at around 1000kb/s for comparison).
What's internet2 ???
anyone got a torrent for internet2? ;P
Has anyone checked this out? http://www.two-degrees.com I tried it on a download yesterday of 50 megs and 1 second after starting up I was connected to 7 hosts at full download speed. Torrents have never done that for me ever no matter what network I was downloading from.
Sadly it has no linux version.
This seems like govt. come full-circle to me, in some ways. After all, the Internet as we know it today was built/funded by the govt. (as Arpanet, initially) - and eventually became privatized.
Now, these plans are popping up to create govt. funded high-speed networks? It's just going back to the roots of the net, but on a state or city instead of federal level.
Of course, the telecom companies would be expected to whine and complain - but their selfish motives aside, I think it is legitimate to ask why government feels this is a necessary function for them to perform. Sure, any of us "computer geeks" would love to have a net connection running 100x the current speed - but there's no free lunch here. This eventually means you're going to be paying for it in taxes, and as is the norm for government spending - you can bet they won't shift around existing funds to pay for this. Instead, they'll waste just as much money on other projects as ever, and demand a tax increase to cover this new one....
Unfortunately, most campuses use Packeteer or other packet-shaping devices to analyze the packets to determine the traffic type in order to throttle bandwidth-hogging applications rather than blocking ports explicitly. While port-changing tricks may have worked in the past, even the most incompetent administrator can set up one of these, and no matter how many ports you try, you're not getting around it.
$5 / month hosted VPS on linux = awesome!
Uninspiring, if IPv6 don't make anything new possible aren't there technologies like multicast that could make sense for p2p programs like bittorrent on a sufficiently updated network?
$5 / month hosted VPS on linux = awesome!
Yep, been there, done that. :)
Our research/education network here in Sweden called SUNET (Swedish University NETwork) runs at 10 Gbit/s in the core and providing all connecting nodes with minimum 2 GB/s trunks, all the major universities are directly connected with 10 GB/s trunks.
Multicast is available in IPv4, but not too many ISPs support it natively, and I'm not aware of any of the major ISPs doing multicast peering with each other except to the extent that they connect to the mbone. There has been lots of work on reliable multicast for file distribution, but that's a lot more useful in a controlled environment, e.g. a many-site company distributing software updates or retail price lists over the Internet instead of satellite, than they are in random crowds where people join and leave. Unreliable multicast applications are a lot easier to coordinate over random crowds, e.g. video conferences, where if you miss a few packets you're not going to go back and recover them.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
1. Get To Know your Campus Admin
2. Buy Him/Her $quantity of their favorite $consumable
3. Ask for Local Bittorrent Access
4. Profit!
.
== WolfriderV6 == I'm willing to admit that *I just might* be wrong... Are you??
Because Python isn't Java. All of the Python apps I have used were fast and worked well. I can't say the same of Java apps. :(
However, freenet really does have the potential to be faster than bittorrent due to the way it caches more popular data closer to where the demand is and distributes the load over a great many more systems.
We have 2 local power companies, One of which is cinergy who told me about 6 months ago they were beta testing and had hard plans to offer it in full swing in my area in 6 months, or if needed would offer atleast beta testing (which should be this month) but now they are quiet about it, another power co, IPALCO, has totally dropped moving to internet over power line.
Strange thing is, they have already invested in ALL of the needed hardware and such to run it (i do have an inside source for this info) so...are we really going to see it?
We have seen that living things are too improbable and too beautifully "designed" to have come into existence by chance.
Because Python isn't Java. All of the Python apps I have used were fast and worked well. I can't say the same of Java apps. :( ...and the really funny thing is this is mostly true even though a Java JIT compiler is typically quite a lot faster than the Python interpreter when benchmarked running equivalent code.
(Part of what this implies is that Java code really can be quite a lot faster than Python when it's written right using libraries that don't suck -- but the "libraries that don't suck" bit is important; a Python/GTK interface will be faster than Java/Swing any day, even though the Python interpreter is slower to process the program logic).
BitTorrent is quite nice, though I found that sharing files with it isn't quite easy, since you need to seed the file yourself if no one else has it yet. So if you are keeping up a thousand downloads on your website for example, you'd need to run a thousand bittorent seeds for those.
It would be nice if the bittorrent tracker file could have a regular http or ftp link in it, from which the BT client would start downloading the file in case no seeds are found. I think that would make it alot easier to use for legal file downloads in websites.
What I am trying to do is to get the author of Downloader for X to include BT protocol in his program. As his program already supports HTTP, FTP (with file splitting for multithreaded downloads), scheduling, inter-process communication (X drag-n-drop) and has a decent GUI, if he would add BT it would be just about perfect.
/.ed he may just do it.
Go to his site, take a look, and perhaps drop him a line on his forums asking for BT - maybe if he gets
www.eFax.com are spammers
Would it be possible to make a P2P server hybridized program? Essentially a Server serves content (a peer that never vanishes) and other peers simple serve to speed it up?
Nah, GIFs were all the rage. Especially the new 256 color ones...
"Champagne for my real friends - and real pain for my sham friends!" http://ericblade.postalboard.com/
I would wager that the lines that anyone uses in a 'repressed' country are already tapped.