The monorail stops at the Las Vegas Hilton (http://www.vegas.com/transportation/monorails.html). It always seemed appropriate to take a monorail to Star Trek: The Experience.
I can't believe that they didn't make a big deal out of it closing - I'm in Vegas often (I went to ST:TE a few months ago) and am into Star Trek, and I didn't hear a thing. You'd think that they'd make a little noise and get Trek fans to go "one last time". Heck, flights to Vegas are cheap enough that I might have gone last weekend to take my kids through ST:TE again - they loved it when they went before. (As did I - I'm not using my kids as an excuse...)
To add some detail, Microsoft's Windows license to PC manufacturers gave them highly preferential pricing if they agreed to pay for a Windows license on every PC shipped, and to not shipping any other OS with their PC's, with the difference in price so high that no OEM could possibly agree to pay the higher price. The restriction was quite extreme - Microsoft blocked several companies from even shipping a BeOS CD in the same box as a "BeOS PC" - I think Fujitsu actually shipped PC's for a little while with no OS, and a form that you could fill out and send them so that they would FedEx you a copy of BeOS. Of course, since they had to pay for Windows anyway, the BeOS PC was not only more complex (you had to order the install CD, then do the OS install) but cost more (since you had to pay for both Windows and BeOS).
By the time the DoJ settlement clarified that this restriction was illegal, BeOS was long dead.
I miss BeOS. On a ThinkPad, BeOS would boot and be running so quickly that if I powered on as I took it from my laptop bag it was ready by the time I put my laptop down and opened the screen. Much faster than Windows coming back from hibernation.
Of course, the old install CD's still work, so if you just need a fast booting OS with a web browser, email, etc., you could probably still run it.
"If I'm paying $60 a month I expect my own freaking lane."
You shouldn't have this expectation, because that's not what you're paying for. If you want a committed 1.5 Mbps (a T1) that costs (for example) $360/month. This gives you guaranteed 1.5 Mbps with a 99.99% SLA. That bandwidth is reserved for your exclusive use, and you can pump data through it 24/7.
What you're paying $30/month for is cheap, shared bandwidth with no guarantees. The reason that it costs so much less is that your ISP isn't reserving capacity in their network exclusively to you, but is running you on a shared infrastructure for your neighborhood, town, state, etc., built out to support normal usage patterns (meaning that users occasionally need their full bandwidth,. but not constantly) So you might get 1.5 Mbps, but you might not, if your ISP has a lot of other traffic right now.
So it's true that money solves the capacity problem. If the ISPs spend 10x on their infrastructure in order to prove everyone with committed bandwidth to the internet, then there would be no congestion problems. But then you'd pay 10x more for internet service. For most people that is a very bad deal.
"Currently, I'm on a bargain-basement Virgin Mobile prepaid plan because I use my mobile phone almost exclusively to arrange rides. This plan costs $160 for two years. If upgrading from a bargain-basement plan designed for occasional voice to a much more expensive data plan would increase my mobile phone bill by an order of magnitude, I'd need a d*** good reason."
Would you consider 24/7 unlimited internet access anywhere you go a "d*** good reason" to pay for a real voice/data plan? If not, what are you doing posting on Slashdot?:-)
Actually, all web browsers download files to your filesystem like crazy - the "web cache". The web cache could have all sorts of evil stuff in in, since it's completely open to the world, so it's in an obscure place in order to make it unlikely that users find and execute them.
Under recent versions of OS X, the OS knows that the file is from the internet, and warns on file open (and offers to show you the page that it came from). This takes place no matter how you got the file (presumable because browser register the info for the file via some API).
I don't recall ever seeing such a warning in Window (Vista, XP, etc., using Firefox, Safari or IE), but since people in this discussion have mentioned such a warning if the file is correctly marked, I wonder if perhaps there's some setting that I need to change?
For FIOS, the ActionTec can be configured with your router as the DMZ host, which effectively puts your router on the internet, though with one layer of NAT/forwarding.
You can also replace the ActionTec with any other router, which gets an address via DHCP. You just have to clone the MAC address or call Verizon to tell them to reconfigure their router to talk to your MAC address. I believe that some of FIOS TV's capabilities depend on the ActionTec router (e.g. VOD).
All very friendly and easy, if you know networking.
However, why would it not be fairly accurate to see that "x ISP is doing a significantly larger than normal amount of resets" = they might have something going on with their resets?
Additionally, its not like comcast or any other ISP wants us to see said data or would let us, so where else can it go? I think that it would be entirely reasonable to say that "customers of x ISP see a significantly higher than normal rate of resets = the ISP might have something going on".
To go further than that, you'd want to do (as another poster suggested) a more detailed data collection and analysis. That could help determine who's doing what.
... the actual reason that ISP's are doing traffic shaping related to p2p is driven by bandwidth consumption exceeding their capacity....
I don't understand. If it's strictly a bandwidth issue, why don't they do traffic shaping for all bandwidth regardless of protocol? They do. There are (AFAIK) four basic strategies for managing bandwidth: - Let lines saturate, then drop random packets. - Prioritize traffic within capacity based on protocol. For example, give the most time sensitive VOIP and streaming protocols highest priotity, then HTTP, then P2P. This seems good in theory, but in practice the distinctions aren't so clear (e.g. p2p streaming), and it opens the door to all sorts of issues. - Rate limit users based on protocol-agnostic rules (e.g. data transfer volumes). - Keep buying capacity to exceed demand.
Each strategy has different strengths and weaknesses. Different ISP's pursue different strategies.
For this, you might pay $60/month for 20 Mbps, or $3/Mbps. You mean $60/month for 3-5Mbps. This is the US, not some other developed, developing, or 3rd world country. The pricing I gave is what I pay for FIOS. Even if you're paying a bit more for your home broadband that I am, that doesn't change the point that consumer capped bandwidth is dirt cheap compared to business-grade committed bandwidth.
I've never heard of any ISP who's monitoring their network for specific content, because it raises all sorts of legal questions.
Perhaps not today, but perhaps you missed this article? Yes, that article about AT&T was all over the news.
The point I was making is that while many people think that traffic shaping has something to do with ISP's not liking specific content, or not liking "piracy," the actual reason that ISP's are doing traffic shaping related to p2p is driven by bandwidth consumption exceeding their capacity, not by content/copyright issues.
I suggest you try not to assume all vuze facts are incorrect. I happen to be in an area where the reset rate was 50-75%, and to ensure accuracy I did nothing more than download a torrent via azureus and then seed it. I'm not sure where you got the idea that I assumed that all Vuze's facts were incorrect. In fact, I'm assuming that all of their facts are correct, because I have no reason to believe otherwise. And because it's pretty obvious how Vuze can count and collect TCP resets. So while it's nice that your testing on your PC showed that their reporting of TCP resets was fairly accurate, that doesn't have anything to do with the issue I raised.
What I pointed out is that capturing reset rates alone can't prove that your ISP is issuing resets, because (as I already explained) the resets could be caused by the other end of the TCP connection or anything along the connection. At most, a high reset rate for many users of a given ISP might indicate that a deeper analysis of traffic patterns might be warranted. But (at least in what's been reported) Vuze hasn't done that deeper analysis, or made the raw data available so that anyone else can do the analysis.
Note that I am not saying that ISP's aren't sending resets, or that Vuze collected bad data. I am only saying that the information that's been released so far (tcp reset rates) doesn't actually prove anything about what ISP's are doing.
... is why ISPs want to be in the business of monitoring their networks for certain content. Aren't they supposed to have common-carrier status (which, AFAIK, is supposed to mean that they're agnostic about and not responsible for the traffic on their networks)? Why do they want to spend money on engineering and PR damage-control for all this if they could just ignore it? They don't. I've never heard of any ISP who's monitoring their network for specific content, because it raises all sorts of legal questions.
The reason that ISP's are starting to manage traffic it is due to capacity issues - changes in user behavior (e.g. viewing high quality video online, p2p) dramatically increase the bandwidth consumption per user, causing demand to exceed available bandwidth.
Given that demand exceeds current supply, and expanding capacity is time consuming and expensive, some ISP's appear to be managing traffic in a protocol-specific way (i.e. deliver time-sensitive VOIP traffic before HTTP page views before P2P seeding), and others appear to be managing traffic in a protocol-agnostic way.
Of course, many ISP's are building out to have capacity that exceeds demand. This is expensive and time consuming (e.g. Comcast has started deploying DOCSIS 3.0, but it'll take years and billions of dollars to upgrade everyone, Verizon has been rolling out fiber to the home, but again it'll be years and billions of dollars before fiber can completely replace DSL). And, in lower population density areas, or parts of the world where people can't pay much for broadband, the cost of providing more capacity exceeds what people are willing to pay, so traffic shaping is the only viable answer.
I've seen some people say "They sold me X bandwidth, and now they're not delivering it". They're confusing two very different types of bandwidth, capped and committed.
Capped bandwidth is cheap, because there are no guarantees other than that you won't get more than a certain amount. This is what home users generally buy. So if you read your ISP's terms, they probably are very clear that you're getting "up to X" performance, but with no committed performance, or even availability. For this, you might pay $60/month for 20 Mbps, or $3/Mbps.
Committed bandwidth is expensive, because the ISP reserves resources so that you can always get all the bandwidth that you're paying for, with financial penalties to the ISP for slowdowns or outages. For this, you might pay $359/month for a T1 line giving you 1.5 Mbps, or $239/Mbps.
What this means is that if you pay for capped bandwidth, you're making the choice of saving a lot of money by buying unreliable bandwidth. If you really, really want committed bandwidth, you can do what web sites and businesses do, and pay for committed bandwidth.
Now, whenever I have a P2P Torrent going a day or more, I know my connection is going to lock up completely anywhere from 20 to 28 hours into the process. The only solution is to hard boot my DSL modem. It then happens again, about once a day, until I stop the torrent.
Coincidence? I think not. I had this happen regularly with my router (linksys). Since home routers are so cheap, I ended up replacing it, and never had it happen again. So I can't say whether the lock-ups were caused by hardware, firmware, etc., but I can say that in my case it wasn't the ISP.
TCP resets can occur for many reasons. All that client software can know and report is that the TCP reset occurred. But, for example, it can't know whether it got a reset because the software on the other end of the connection crashed, or had a bug, or the computer was turned off, or there was some corrupted communications between the two causing the TCP connection to get confused and need to be reset. This is all explained at http://www.tcpipguide.com/free/t_TCPConnectionManagementandProblemHandlingtheConnec.htm (for example).
Vuze's test only counted reset rates, so it can't prove anything about what's going on. At most, it could suggest areas where it might be productive to do more investigation.
Sorry, I thought that we were talking about P4P, not Pando.
Pando's network is a managed p2p network. This gives us the economic advantages of p2p delivery, but with the control and quality of service of a centrally managed system. That being said, Pando doesn't notify your ISP when you connect to a our trackers.
This all has nothing to do with P4P. P4P doesn't notify your ISP, or anyone else, every time you connect to a tracker.
"I can't help but notice your email and sig. I'll tell you what, when the piratebay is part of the working group, I'll consider your technology. until then, I'm afraid I'm not interested."
Yes, I'm the CTO of Pando Networks. Who are you?
To clarify another point, P4P isn't my technology, it's an open standard being worked by by over 50 organizations, including the major P2P companies, many ISP's, universities, and individuals.
In terms of whether your favorite web site supports P4P, well, P4P is an open standard that anyone interested is free to implement. BitTorrent is in the P4P Working Group, and has been since its initial formation. As to whether a specific web site chooses to support P4P, well, that's up to them.
"ok then, lets be crystal clear about this. So your telling me that AT&T, your company's primary participator, and Internet Content Policeman [nytimes.com] did not require your company to develop a means to determine the content of a given P2P connection?"
AT&T has not required Pando to do anything in return for their participation in the working group. Neither has any other ISP.
To clarify who did what in the test, Pando worked with Yale, Verizon and Telefonica to perform the first field test. AT&T is in the P4P Working Group, and contributed valuable data for earlier simulations, but didn't participate in the first field test.
P4P is a mechanism that allows ISP's to provide information to P2P networks to allow them to improve performance and efficiency. P2P companies aren't required to use that information. P4P does not expose ISP internal network structure to P2P's, and does not expose P2P network details to ISP's.
"I do not want my protocol to notify my ISP everytime i connect to a tracker."
"According to the study, redoing the P2P into what they call P4P can reduce the number of 'hops' by an average of 400%."
and am confused. Surely reducing a number by 100% brings it to zero. What does reducing a number by 400% mean?
10 becomes -30? Yeah, I can't say where that number came from. So I'll explain what I do know.
Without P4P the data delivered within the ISP travelled across an average of 5.5 long distance links, and with P4P the data travelled across an average of 0.89 long distance links. The average is so low because a whopping 58% of the data was delivered within the same metro area, and the rest came largely from adjacent metro areas.
The result of this was that for delivering a given volume of data, P4P reduced internal link utilization by 6x (5.5/0.89 is about 6).
P4P is a major privacy killer. based on what I see at the P4P workgroup page, P4P is not a protocol or code that will be inject into existing P2P apps, it is a network management technique and toolkit that the ISPs can use to control existing and future P2P traffic, presumably without knowledge or consent from any of the peers. I'm not sure where you got this - P4P is a mechanism that allows ISP's to provide network map data to existing P2P networks so that the P2P networks can, if they choose, use that information to make smarter peer connections. P4P is entirely optional; if a P2P network doesn't want to implement it, they can keep doing what they are doing now.
Determine, validate, and encourage the adoption of methods for ISPs and P2P software distributors to work together to enable and support consumer service improvements as P2P adoption and resultant traffic evolves while protecting the intellectual property (IP) of participating entities Somehow I just knew IP rights would come up. I'll pay more attention when the pirate bay is in the core group. Until then, I'm not interested, logical as the idea may seem. You're right - this statement is ambiguous, leading to misinterpretation. To clarify, the participating entities in P4P are the ISP's and the P2P networks. Their IP has to do with things like the ISP's network structure and the details of how the P2P network operates or what it's transporting. Thus, the P4P protocol is abstracted so that neither party has to expose more than they are comfortable with.
"the problem with this idea however, is that while the providers have spent billions over the last decade extending and enhancing their distribution tier (to get more customers), the local network is the source of most users congestion"
Using P2P, by definition the total data volume uploaded and downloaded by all users is the same. P4P doesn't increase the amount that users upload or download, so the "edge" network . P4P shortens the distance between the uploader and the downloader, in order to reduce the delivery cost of the data. And while ISP's have spent a fortune building out their core networks, they're continually building out as they add customers, and as customer usage grows, so it's a large ongoing expense.
Of course, caching can also be a part of the answer, which is why all of the p2p caching companies are in the P4P Working Group.
"While I don't doubt your honesty, RIM makes available a fully documented SDK and has done so for years. If the carriers don't want RIM to provide these features, nothing is stopping anyone else from doing so. There are lots of 3rd-party applications available for the blackberry platform."
Well, there's a RIM SDK, but it's terrible (e.g. http://devberry.com/2008/03/06/rim-sdk-a-pre-teen-schoolgirl/). The iPhone SDK is fantastic. Sure, there are some limitations (app's can's access other app's data except through API"s, only one "application" runs at a time), but they're minor compared to the limitations of writing for the RIM, much less BREW, etc.
"Apple gets a monthly tribute from AT&T for every iphone that is active with AT&T. Which is bigger, Apple's margins on iphone sales or AT&T's tribute to Apple?"
"There can be proper vote printing machines. There can be proper vote tabulating machines.
But the same device can never do both properly. The votes must be inspectable by humans between these steps."
This is exactly right. To elaborate, vote printing machines are good, because they can validate input, warn voters when there may be an error (e.g. filling out a ballot but skipping the top race, which is usually not the voter's intent), can provide multi-lingual ballots, and can provide spoken prompts to assist the visually impaired and illiterate.
"Basically, you agree with my argument" yes, I wasn't disagreeing, I was pointing out that content owners recognize that they need to give value to users for using p2p.
That being said, I think that you should keep in mind that usually no cost to users for p2p using otherwise unused uplink bandwidth. Most computers are usually on fixed-cost, always-on internet connections, and are idle 80% of the time, so there are a lot of idle resources available that the user can contribute into a p2p network without any dollar cost.
As for your analysis of the cost to distributors, you're forgetting about the cost to actually deliver the video files, meaning hosting servers, paying for bandwidth, etc. (or paying a CDN to do the same). So while it's true that once the content distributor has encoded a file they don't have to do any more "work" in some sense, there are servers, network connections, etc., that all have to do work to get that file delivered.
So the deal appears to generally be that in terms of what you can get for free (i.e. paid for by ad revenue) you can watch low-quality files via HTTP or streams, and you can download high-quality files via P2P. If you don't value video quality, and don't want to install software, then you should watch the flash stream. If you value video quality, and you are willing to install p2p software, then you should do that.
Or you can buy video downloads. Since people pay for the downloads, that covers the delivery costs.
"wouldn't it be a MASSIVE improvement if the ISPs just gave you a flat list of IPs within your metro area, no routing or anything like that?"
That's an improvement, but if there's information about the structure of the ISP's network you can connect people within their network much more efficiently. For example, Verizon Internet has customers all over the US, Japan, Europe, etc., and it's better to connect people with (for example) the New York metro area to each other first, and avoid moving data through trans-Atlantic and trans-Pacific links. So far in talking with ISP's, these network maps aren't hard to generate, because they use automated systems to configure their routers, and the same data can generate network maps for P4P.
"either you're going to have to collect a huge chunk of routing information so your client can figure out which peers are "close" to you, or a third party is going to have to manage the peering...Neither one of those thrills me, especially since an ISP is pushing the technology, which would make them the obvious third party."
This is a very good point.
P4P has an intermediary between the ISP and the P2P network, called an iTracker, that can be run by a third party. In the field test, Verizon and Telefonica provided the data, Yale ran the iTracker, and Pando was the P2P network. P4P isn't implemented in P2P clients, it's implemented in the P2P trackers (for protocols with a Tracker). For example, in BitTorrent, P4P is a protocol between the Tracker and the iTracker, allowing the Tracker to query the iTracker to determine good IP's to recommend. It's very important that there's an iTracker between the P2P network and the ISP, so that the privacy of both can be protected. That is, the ISP can provide information to the iTracker that they wouldn't want to give a P2P company, and the P2P company just gets connection recommendations. Similarly, the P2P network can provide detailed data to the iTracker in order to get well-tuned connection recommendations, without giving "too much information" to the ISP's.
To clarify another point. P4P doesn't "manage the peering" - it provides guidance to the P2P network about which IP's are near each other to help the P2P network decide which peers to connect. But P4P doesn't control the P2P network - it's just additional information that can be used along with anything else that the P2P network uses.
Slashdot Mirror
The monorail stops at the Las Vegas Hilton (http://www.vegas.com/transportation/monorails.html). It always seemed appropriate to take a monorail to Star Trek: The Experience.
I can't believe that they didn't make a big deal out of it closing - I'm in Vegas often (I went to ST:TE a few months ago) and am into Star Trek, and I didn't hear a thing. You'd think that they'd make a little noise and get Trek fans to go "one last time". Heck, flights to Vegas are cheap enough that I might have gone last weekend to take my kids through ST:TE again - they loved it when they went before. (As did I - I'm not using my kids as an excuse...)
To add some detail, Microsoft's Windows license to PC manufacturers gave them highly preferential pricing if they agreed to pay for a Windows license on every PC shipped, and to not shipping any other OS with their PC's, with the difference in price so high that no OEM could possibly agree to pay the higher price. The restriction was quite extreme - Microsoft blocked several companies from even shipping a BeOS CD in the same box as a "BeOS PC" - I think Fujitsu actually shipped PC's for a little while with no OS, and a form that you could fill out and send them so that they would FedEx you a copy of BeOS. Of course, since they had to pay for Windows anyway, the BeOS PC was not only more complex (you had to order the install CD, then do the OS install) but cost more (since you had to pay for both Windows and BeOS).
By the time the DoJ settlement clarified that this restriction was illegal, BeOS was long dead.
I miss BeOS. On a ThinkPad, BeOS would boot and be running so quickly that if I powered on as I took it from my laptop bag it was ready by the time I put my laptop down and opened the screen. Much faster than Windows coming back from hibernation.
Of course, the old install CD's still work, so if you just need a fast booting OS with a web browser, email, etc., you could probably still run it.
"If I'm paying $60 a month I expect my own freaking lane."
You shouldn't have this expectation, because that's not what you're paying for. If you want a committed 1.5 Mbps (a T1) that costs (for example) $360/month. This gives you guaranteed 1.5 Mbps with a 99.99% SLA. That bandwidth is reserved for your exclusive use, and you can pump data through it 24/7.
What you're paying $30/month for is cheap, shared bandwidth with no guarantees. The reason that it costs so much less is that your ISP isn't reserving capacity in their network exclusively to you, but is running you on a shared infrastructure for your neighborhood, town, state, etc., built out to support normal usage patterns (meaning that users occasionally need their full bandwidth,. but not constantly) So you might get 1.5 Mbps, but you might not, if your ISP has a lot of other traffic right now.
So it's true that money solves the capacity problem. If the ISPs spend 10x on their infrastructure in order to prove everyone with committed bandwidth to the internet, then there would be no congestion problems. But then you'd pay 10x more for internet service. For most people that is a very bad deal.
"Currently, I'm on a bargain-basement Virgin Mobile prepaid plan because I use my mobile phone almost exclusively to arrange rides. This plan costs $160 for two years. If upgrading from a bargain-basement plan designed for occasional voice to a much more expensive data plan would increase my mobile phone bill by an order of magnitude, I'd need a d*** good reason."
:-)
Would you consider 24/7 unlimited internet access anywhere you go a "d*** good reason" to pay for a real voice/data plan? If not, what are you doing posting on Slashdot?
Actually, all web browsers download files to your filesystem like crazy - the "web cache". The web cache could have all sorts of evil stuff in in, since it's completely open to the world, so it's in an obscure place in order to make it unlikely that users find and execute them.
Under recent versions of OS X, the OS knows that the file is from the internet, and warns on file open (and offers to show you the page that it came from). This takes place no matter how you got the file (presumable because browser register the info for the file via some API).
I don't recall ever seeing such a warning in Window (Vista, XP, etc., using Firefox, Safari or IE), but since people in this discussion have mentioned such a warning if the file is correctly marked, I wonder if perhaps there's some setting that I need to change?
For FIOS, the ActionTec can be configured with your router as the DMZ host, which effectively puts your router on the internet, though with one layer of NAT/forwarding.
You can also replace the ActionTec with any other router, which gets an address via DHCP. You just have to clone the MAC address or call Verizon to tell them to reconfigure their router to talk to your MAC address. I believe that some of FIOS TV's capabilities depend on the ActionTec router (e.g. VOD).
All very friendly and easy, if you know networking.
However, why would it not be fairly accurate to see that "x ISP is doing a significantly larger than normal amount of resets" = they might have something going on with their resets?
Additionally, its not like comcast or any other ISP wants us to see said data or would let us, so where else can it go? I think that it would be entirely reasonable to say that "customers of x ISP see a significantly higher than normal rate of resets = the ISP might have something going on".
To go further than that, you'd want to do (as another poster suggested) a more detailed data collection and analysis. That could help determine who's doing what.
I don't understand. If it's strictly a bandwidth issue, why don't they do traffic shaping for all bandwidth regardless of protocol? They do. There are (AFAIK) four basic strategies for managing bandwidth:
- Let lines saturate, then drop random packets.
- Prioritize traffic within capacity based on protocol. For example, give the most time sensitive VOIP and streaming protocols highest priotity, then HTTP, then P2P. This seems good in theory, but in practice the distinctions aren't so clear (e.g. p2p streaming), and it opens the door to all sorts of issues.
- Rate limit users based on protocol-agnostic rules (e.g. data transfer volumes).
- Keep buying capacity to exceed demand.
Each strategy has different strengths and weaknesses. Different ISP's pursue different strategies.
Perhaps not today, but perhaps you missed this article? Yes, that article about AT&T was all over the news.
The point I was making is that while many people think that traffic shaping has something to do with ISP's not liking specific content, or not liking "piracy," the actual reason that ISP's are doing traffic shaping related to p2p is driven by bandwidth consumption exceeding their capacity, not by content/copyright issues.
What I pointed out is that capturing reset rates alone can't prove that your ISP is issuing resets, because (as I already explained) the resets could be caused by the other end of the TCP connection or anything along the connection. At most, a high reset rate for many users of a given ISP might indicate that a deeper analysis of traffic patterns might be warranted. But (at least in what's been reported) Vuze hasn't done that deeper analysis, or made the raw data available so that anyone else can do the analysis.
Note that I am not saying that ISP's aren't sending resets, or that Vuze collected bad data. I am only saying that the information that's been released so far (tcp reset rates) doesn't actually prove anything about what ISP's are doing.
... is why ISPs want to be in the business of monitoring their networks for certain content. Aren't they supposed to have common-carrier status (which, AFAIK, is supposed to mean that they're agnostic about and not responsible for the traffic on their networks)? Why do they want to spend money on engineering and PR damage-control for all this if they could just ignore it? They don't. I've never heard of any ISP who's monitoring their network for specific content, because it raises all sorts of legal questions.The reason that ISP's are starting to manage traffic it is due to capacity issues - changes in user behavior (e.g. viewing high quality video online, p2p) dramatically increase the bandwidth consumption per user, causing demand to exceed available bandwidth.
Given that demand exceeds current supply, and expanding capacity is time consuming and expensive, some ISP's appear to be managing traffic in a protocol-specific way (i.e. deliver time-sensitive VOIP traffic before HTTP page views before P2P seeding), and others appear to be managing traffic in a protocol-agnostic way.
Of course, many ISP's are building out to have capacity that exceeds demand. This is expensive and time consuming (e.g. Comcast has started deploying DOCSIS 3.0, but it'll take years and billions of dollars to upgrade everyone, Verizon has been rolling out fiber to the home, but again it'll be years and billions of dollars before fiber can completely replace DSL). And, in lower population density areas, or parts of the world where people can't pay much for broadband, the cost of providing more capacity exceeds what people are willing to pay, so traffic shaping is the only viable answer.
I've seen some people say "They sold me X bandwidth, and now they're not delivering it". They're confusing two very different types of bandwidth, capped and committed.
Capped bandwidth is cheap, because there are no guarantees other than that you won't get more than a certain amount. This is what home users generally buy. So if you read your ISP's terms, they probably are very clear that you're getting "up to X" performance, but with no committed performance, or even availability. For this, you might pay $60/month for 20 Mbps, or $3/Mbps.
Committed bandwidth is expensive, because the ISP reserves resources so that you can always get all the bandwidth that you're paying for, with financial penalties to the ISP for slowdowns or outages. For this, you might pay $359/month for a T1 line giving you 1.5 Mbps, or $239/Mbps.
What this means is that if you pay for capped bandwidth, you're making the choice of saving a lot of money by buying unreliable bandwidth. If you really, really want committed bandwidth, you can do what web sites and businesses do, and pay for committed bandwidth.
Coincidence? I think not. I had this happen regularly with my router (linksys). Since home routers are so cheap, I ended up replacing it, and never had it happen again. So I can't say whether the lock-ups were caused by hardware, firmware, etc., but I can say that in my case it wasn't the ISP.
TCP resets can occur for many reasons. All that client software can know and report is that the TCP reset occurred. But, for example, it can't know whether it got a reset because the software on the other end of the connection crashed, or had a bug, or the computer was turned off, or there was some corrupted communications between the two causing the TCP connection to get confused and need to be reset. This is all explained at http://www.tcpipguide.com/free/t_TCPConnectionManagementandProblemHandlingtheConnec.htm (for example).
Vuze's test only counted reset rates, so it can't prove anything about what's going on. At most, it could suggest areas where it might be productive to do more investigation.
Sorry, I thought that we were talking about P4P, not Pando.
Pando's network is a managed p2p network. This gives us the economic advantages of p2p delivery, but with the control and quality of service of a centrally managed system. That being said, Pando doesn't notify your ISP when you connect to a our trackers.
This all has nothing to do with P4P. P4P doesn't notify your ISP, or anyone else, every time you connect to a tracker.
"I can't help but notice your email and sig. I'll tell you what, when the piratebay is part of the working group, I'll consider your technology. until then, I'm afraid I'm not interested."
Yes, I'm the CTO of Pando Networks. Who are you?
To clarify another point, P4P isn't my technology, it's an open standard being worked by by over 50 organizations, including the major P2P companies, many ISP's, universities, and individuals.
In terms of whether your favorite web site supports P4P, well, P4P is an open standard that anyone interested is free to implement. BitTorrent is in the P4P Working Group, and has been since its initial formation. As to whether a specific web site chooses to support P4P, well, that's up to them.
"ok then, lets be crystal clear about this. So your telling me that AT&T, your company's primary participator, and Internet Content Policeman [nytimes.com] did not require your company to develop a means to determine the content of a given P2P connection?"
AT&T has not required Pando to do anything in return for their participation in the working group. Neither has any other ISP.
To clarify who did what in the test, Pando worked with Yale, Verizon and Telefonica to perform the first field test. AT&T is in the P4P Working Group, and contributed valuable data for earlier simulations, but didn't participate in the first field test.
P4P is a mechanism that allows ISP's to provide information to P2P networks to allow them to improve performance and efficiency. P2P companies aren't required to use that information. P4P does not expose ISP internal network structure to P2P's, and does not expose P2P network details to ISP's.
"I do not want my protocol to notify my ISP everytime i connect to a tracker."
P4P doesn't do this.
and am confused. Surely reducing a number by 100% brings it to zero. What does reducing a number by 400% mean?
10 becomes -30? Yeah, I can't say where that number came from. So I'll explain what I do know.
Without P4P the data delivered within the ISP travelled across an average of 5.5 long distance links, and with P4P the data travelled across an average of 0.89 long distance links. The average is so low because a whopping 58% of the data was delivered within the same metro area, and the rest came largely from adjacent metro areas.
The result of this was that for delivering a given volume of data, P4P reduced internal link utilization by 6x (5.5/0.89 is about 6).
"the problem with this idea however, is that while the providers have spent billions over the last decade extending and enhancing their distribution tier (to get more customers), the local network is the source of most users congestion"
Using P2P, by definition the total data volume uploaded and downloaded by all users is the same. P4P doesn't increase the amount that users upload or download, so the "edge" network . P4P shortens the distance between the uploader and the downloader, in order to reduce the delivery cost of the data. And while ISP's have spent a fortune building out their core networks, they're continually building out as they add customers, and as customer usage grows, so it's a large ongoing expense.
Of course, caching can also be a part of the answer, which is why all of the p2p caching companies are in the P4P Working Group.
"While I don't doubt your honesty, RIM makes available a fully documented SDK and has done so for years. If the carriers don't want RIM to provide these features, nothing is stopping anyone else from doing so. There are lots of 3rd-party applications available for the blackberry platform."
Well, there's a RIM SDK, but it's terrible (e.g. http://devberry.com/2008/03/06/rim-sdk-a-pre-teen-schoolgirl/). The iPhone SDK is fantastic. Sure, there are some limitations (app's can's access other app's data except through API"s, only one "application" runs at a time), but they're minor compared to the limitations of writing for the RIM, much less BREW, etc.
"Apple gets a monthly tribute from AT&T for every iphone that is active with AT&T. Which is bigger, Apple's margins on iphone sales or AT&T's tribute to Apple?"
Very good point.
"There can be proper vote printing machines.
There can be proper vote tabulating machines.
But the same device can never do both properly.
The votes must be inspectable by humans between these steps."
This is exactly right. To elaborate, vote printing machines are good, because they can validate input, warn voters when there may be an error (e.g. filling out a ballot but skipping the top race, which is usually not the voter's intent), can provide multi-lingual ballots, and can provide spoken prompts to assist the visually impaired and illiterate.
There's an open source system that does exactly this. Please support http://www.openvotingconsortium.com/!
"Basically, you agree with my argument" yes, I wasn't disagreeing, I was pointing out that content owners recognize that they need to give value to users for using p2p.
That being said, I think that you should keep in mind that usually no cost to users for p2p using otherwise unused uplink bandwidth. Most computers are usually on fixed-cost, always-on internet connections, and are idle 80% of the time, so there are a lot of idle resources available that the user can contribute into a p2p network without any dollar cost.
As for your analysis of the cost to distributors, you're forgetting about the cost to actually deliver the video files, meaning hosting servers, paying for bandwidth, etc. (or paying a CDN to do the same). So while it's true that once the content distributor has encoded a file they don't have to do any more "work" in some sense, there are servers, network connections, etc., that all have to do work to get that file delivered.
So the deal appears to generally be that in terms of what you can get for free (i.e. paid for by ad revenue) you can watch low-quality files via HTTP or streams, and you can download high-quality files via P2P. If you don't value video quality, and don't want to install software, then you should watch the flash stream. If you value video quality, and you are willing to install p2p software, then you should do that.
Or you can buy video downloads. Since people pay for the downloads, that covers the delivery costs.
"wouldn't it be a MASSIVE improvement if the ISPs just gave you a flat list of IPs within your metro area, no routing or anything like that?"
That's an improvement, but if there's information about the structure of the ISP's network you can connect people within their network much more efficiently. For example, Verizon Internet has customers all over the US, Japan, Europe, etc., and it's better to connect people with (for example) the New York metro area to each other first, and avoid moving data through trans-Atlantic and trans-Pacific links. So far in talking with ISP's, these network maps aren't hard to generate, because they use automated systems to configure their routers, and the same data can generate network maps for P4P.
"either you're going to have to collect a huge chunk of routing information so your client can figure out which peers are "close" to you, or a third party is going to have to manage the peering...Neither one of those thrills me, especially since an ISP is pushing the technology, which would make them the obvious third party."
This is a very good point.
P4P has an intermediary between the ISP and the P2P network, called an iTracker, that can be run by a third party. In the field test, Verizon and Telefonica provided the data, Yale ran the iTracker, and Pando was the P2P network. P4P isn't implemented in P2P clients, it's implemented in the P2P trackers (for protocols with a Tracker). For example, in BitTorrent, P4P is a protocol between the Tracker and the iTracker, allowing the Tracker to query the iTracker to determine good IP's to recommend. It's very important that there's an iTracker between the P2P network and the ISP, so that the privacy of both can be protected. That is, the ISP can provide information to the iTracker that they wouldn't want to give a P2P company, and the P2P company just gets connection recommendations. Similarly, the P2P network can provide detailed data to the iTracker in order to get well-tuned connection recommendations, without giving "too much information" to the ISP's.
To clarify another point. P4P doesn't "manage the peering" - it provides guidance to the P2P network about which IP's are near each other to help the P2P network decide which peers to connect. But P4P doesn't control the P2P network - it's just additional information that can be used along with anything else that the P2P network uses.