Slashdot Mirror
Making BitTorrent Clients Prioritize By Geography?
Daengbo writes "While I live in S.Korea and have virtually unlimited bandwidth in and out of the country, not all my Asian friends are so lucky. Many of the SE Asian and African countries have small international pipes. Even when a user has a high-speed local connection, downloads from abroad will trickle in.
Bittorrent clients apparently don't prioritize other users on the same ISP or at least in the same country. Why is that? Is it difficult to manage? If I were to write a plug-in for, say, Deluge, what hurdles would I be likely to come across? If this functionality is available in other clients or through plug-ins, please chime in."
There is already a plugin for Azereus that does this. I downloaded it about a year ago. I'm at work right now or otherwise I would look at my installation and tell you the exact name of it.
IPs could, theoretically, be prioritized based on a database of known general geographies associated with certain digits. Just remember - prioritizing is one thing, but it's a slippery slope to peer exclusion.
uTorrent has a feature called local peer discovery that does that exactly. It was even able to discover other people at my university sharing the files.
There is a plugin for Vuze (formerly Azureus) called Ono which does exactly that. Not sure what the problems they ran into, but as it is a college project I am sure they would be willing to discuss some of it with you. http://www.aqualab.cs.northwestern.edu/projects/Ono.html
One would not even need to prioritize by geographic location: the client could easily give extra priority points by network class: C first, then B, then A, then the rest. The odds of having a very fat pipe to another machine in the same class C are far better than having a fat pipe to a random machine across the planet.
And that would also alleviate the load on backbone links.
http://www.dieblinkenlights.com
That way they can take advantage of the tendency of IP packets to flow downhill.
What you're looking for is an Azureus plugin called "Ono". It prioritized based on router hops. Theoretically, this would make those connected to the same ISP preffered. After that it would make ISP's with direct connections to your ISP preferred. After that, resonably close geographically, ie same country.
... and in the DRM, bind them.
How good is latency or hops as indicator of distance from peer? The idea is that if it takes 5 hops, as opposed to 10, then the peer taking the least hops to get to is the closest.
Jumpstart the tartan drive.
For Vuze, formerly Azureus, there are Ono and P4P, which should do what you're looking for, although for different reasons. Unfortunately, they both rely on people in your region being interested in the same torrents you are, while P4P additionally benefits from an iTracker, an ISP provided tracker that's topology aware (they did some work to prioritize based on ping latency, using that as a distance estimate, but I don't know if it's a fallback mechanism). Due to the iTracker infrastructure and possibly conflicting supporters, there are some privacy concerns.
"Bittorrent clients apparently don't prioritize other users on the same ISP or at least in the same country. Why is that? Is it difficult to manage?"
The reason BitTorrent doesn't prioritize other users on the same ISP or the same country is that it doesn't know which ones are part of the same ISP or the same country. For ISPs, since the introduction of CIDR addresses, ISPs can have multiple blocks of IPs. Can you honestly tell me what all of, say, Comcast's IP blocks are with any degree of certainty?
For countries, you either need to know which IP blocks IANA has allocated to which IP registry or use a geolocation library.
MaxMind's GeoIP seems to be the de facto geolocation library, but they charge money for the "good" version. There is a free version now, but it has some annoying requirements, such as having to include "This product includes GeoLite data created by MaxMind, available from http://maxmind.com/" in all advertising materials and documentation. It also only has a 99.5% accuracy as claimed by its creators, which means the the accuracy is probably considerably lower than they claim. Even if it were 99.5%, that means it's wrong for 1 out of every 200 people.
GLaDOS for President 2016! "Well here we are again. It's always such a pleasure." -- GLaDOS, 2011
I've always wondered why ISPs can't give higher speeds if you stay within their network. You'd get your download faster. You'd use less peering bandwidth, costing the ISP less money. Everybody wins.
Prioritize by network topology is a better way to put it, that just happens to coincide with physical AND political geography in many cases. In the case where you can get 10Mb over a 10-hop connection, or 8Mb over a 3-hop connection, which do you pick? If you pick the latter, there is a good chance that two other users can utilize the other 70% of that 10-hop connection, making total throughput (theoretically) 24Mb.
No. Prioritize by ASN. A smart tracker would get a BGP feed and then hook users together based on locality of network connectivity.
Any other approach is "wrong."
If your ISPs international pipe is flooded then bittorrent will automatically prefer local peers as they'll be the only people who can send you data at a fast enough rate. If you notice local peers who you're not connected to then it's most likely just because they've already reached their connection limit.
Most bittorent clients will connect to many peers and try to saturate your downstream bandwidth. They don't care where in the world those peers are as long as they're fast. If, in your part of the world, local peers are faster then that means you should just automatically connect to more local peers.
Nick
But we American get email all the time offering to make our pipeline bigger!
At Napster I wrote a system to weight peers that were closer to the person searching by using network distance.
It was mostly because universities were complaining and so we weighted everyone on Internet 2 towards each other, but it also worked quite well for service providers like @Home and AOL. Since ISPs don't seem to care as much when their own bandwidth is used, a lot of complaints about our bandwidth consumption disappeared overnight. Indiana state university and someone else helped out if I remember correctly.
It was a rather simple system that used BGP routing tables from a number of routers to build a graph of network connectivity. It wasn't perfect, but it didn't have to be.
That said, with IPv6 weighting is *much* easier because of how the IP space is divided up. You can do a super naive implementation just by prefix.
An Azureus plugin Ono does something similar, though I believe they just look up the IP address for a CDN and weight people that look up the same IP towards each other. It is a decent solution, but it only works for between people who are running the plugin.
The biggest speed issue facing Asia/Australia is the latency of traffic to the rest of the planet. The (Windows) TCP Receive Window is tuned too small for the distances required. If you change the receive window to the maximum, you can get 4x more data in the same period using any client (P2P, browsers, etc...).
Refer to:
http://cable-dsl.navasgroup.com/index.htm#IncreasingWindow
I'm, I'll, can't, it's?
Autonomous System Number. I don't think it helps much either way. You either know what it means or you don't. Also, Border Gateway Protocol.
When I was in my first year at college, we were asked to produce a questionnaire about using ATMs, including the question: "If you could change one thing about your bank's ATMs, what would it be?"
The most popular answer I managed to get was "if the machine's running out of money, they should restrict the cash withdrawal function to customers of this branch".
Does anyone see a parallel here?
There is little to no support for multicast by last-mile ISPs.
It would be nice - ISPs keep bitching about how P2P is eating their bandwidth, but they don't bother implementing multicast which would make P2P use a fraction of the bandwidth it currently does.
Admittedly, in addition to lack of support, IPv4 multicast is pretty "meh" - there aren't many multicast addresses available and I have yet to see a good way of choosing/assigning them on a global network.
retrorocket.o not found, launch anyway?
Extending a business partnership with them, and convincing them that they CAN allow users to serve content without choking their already oversold bandwidth
This is fairly easy to do right now eith US ISPs...it's called a "business account".
They do cost more, but in my case I get 5 static IPs, guaranteed bandwidth, and no interference of any kind with my data (no port blocking, no throttling, and no caps) for $140. Compared to the non-business version with one dynamic IP, port blocking and some throttling (but no caps yet) at $70, it's not a bad deal.
Ono uses statistical data from CDNs to be a little bit smarter about picking peers in certain cases. In most cases the random solution is fine; your client can just randomly pick peers then stick with fast ones and drop slow ones. Ono aims to improve performance in certain cases where that strategy isn't very good.
Just in case anyone reading doesn't notice, Ono aims to find peers that are close to you on the network. That doesn't necessarily mean close to you geographically and so doesn't answer this ask /..
Nick
And I proposed it for two popular BitTorrent clients, only to be told, "we don't need that.."
Simple enough to do. Start with Class C address.. less then 2^24 of them, as some
are reserved, and not routable. Make a bitmap that big, so divide by 8.
Only a two meg file. Then just watch your connections, total by bytes received, then divide by 8. If the result is greater than say 1 kb/sec sustained, then
set the appropriate bit to true. Allow the bitmap to be zeroed if you move or whatever.
After getting the list of peers, prioritize connection attempts towards those
that have a useful sustainable rate. Nothing worse than seeing 80% of my
connections saying connected 30 minutes, 100 KB transferred. Sigh.
While in china, I had a 2 mb connection. But too many Chinese hammering it, so
I could sustain, 2 mbit up, and 1 mbit down. Asymmetric the wrong way..
Seems blindingly obvious to me, yet I still see no clients with this feature.
The law is a weapon of the government, not a protection for the likes of you. Surely you understand that.
I think they've got it all wrong. The software shouldn't be connecting people to nodes in their area. It should be connecting people to nodes in regions that can't sue them for uploading.