"If they really want to help p2p, then they would expose this topology information to us p2p developers, and let us use it to make all our applications better. What they're likely planning is pushing their own p2p..."
P4P isn't a p2p network. P4P is an open standard that can be implemented by any ISP and any p2p network, and which has been tested so far on BitTorrent (protocol, not company) and Pando software, and the Verizon and Telefonica networks. Participants include all of the major P2P companies and many major ISP's. Participation in the P4P Working Group is open (and free) to any P2P company or ISP. Email marty@dcia.info, laird@pando.com, or doug.pasko@verizon.com if you're interested in joining the working group, or in getting email updates.
"from an endpoint perspective, it's tough to extract network topology and bandwidth"
This is exactly right. There are many interesting strategies for attempting to derive network topology, but they all have weaknesses in various situations, such as when ISP's use OSPF. And observed network behavior can't reflect ISP business policies (e.g. they might prefer to route p2p over cheap links instead of expensive links that are more suited to VOIP and gaming).
It turns out that ISP's are open to the idea of providing precise network information, if done in a way that preserves their security, if it gives them a significant reduction in p2p network traffic. When we asked ISP's at NANOG whether they'd be interested in contributing network data in order to test P4P, about half of the people in the room raised their hands, so there appears (IMO) to be a potential willingness to provide this information.
You're missing the point of P4P. What P4P does is provide information to the P2P network so that it can shorten the distance that data travels through the network. Data that is moved very short distances (e.g. in a local fiber loop) moves much faster than data that is moved long distances (e.g. across the Pacific Ocean). Without P4P, the p2p network doesn't have enough information to determine which IP's are near each other, so on average data moves a very long distance, when the same data was available locally. With P4P, the p2p network can make the local connections. Data that moves short distances only only arrives faster, it consumes less of the ISP's infrastructure.
What we found in the field test is that without P4P, 6% of downloaded data came from the same metro area, and the rest came from further away, or from other ISP's (costing money or consuming scarce/expensive resources). With P4P guidance, 58% of data downloaded came from within the downloader's metro area, dramatically reducing the amount of data being moved across long distance links or from other ISP's.
The software change to implement P4P was pretty simple. Basically it's a Tracker enhancement - it requires no client change. So it's easier to implement than increasing the capacity of an ISP's backbone. Not that I'd want to discourage ISP's from increasing their capacity, of course.:-)
"Why not just do a 'traceroute' to all of the seeds as you discover them, and penalize the ones that are more hops away?"
That would help peers pick between known peers to exchange data with. The problem is that if you're in a large swarm, you'll only know about a small subset of the swarm, and thus almost certainly miss the best peers to connect to. For example, if you're in a swarm with 10,000 peers, and you know about a random 50 peers, you are 99.5% likely not to find out about the closest peer on the first announce (for BitTorrent, which I'll use as the example, since it's well known). The Tracker has global knowledge, so it can tell a peer 100% of the time about the closest peer. Yes, it's true that over time BitTorrent will converge on good data sources, but in large swarms it takes a very long time to connect to and test all peers, so the time that it takes to find a good, nearby data source could well be much longer than the download time,
What we found in the P4P field test is that guided peer connections yielded much faster download speeds almost immediately, because the first peer connection was "close" in the ISP's network, resulting in fast connection and transfer times, and that while the BitTorrent connection logic eventually found good data sources, on average the downloads were over 200% faster (for FTTH users) when the p2p connections were guided.
The P4P information can apply to any P2P protocol. P4P is a protocol, but it's a protocol between the P2P provider's control servers and the ISP, not between clients. Basically, the ISP gives "hints" to the P2P network so that it can figure out which IP's are near each other, and it uses that to guide the p2p connections. For BitTorrent, there's communication between the Tracker and the P4P server. But P4P should in principle be able to apply to any P2P protocol.
"I don't really see how this could be done on a software level... I think thats why they're citing legal content only... it will take some modifications for routing equipment, won't it?"
Nope, one of the appeals of the approach is that it can be implemented at the application level, so there don't need to be any changes to routers, etc. The ISP tells the p2p network how to figure out which IP's are near each other, and the P2P network preferentially connects those IP's rather than using random IP's.
Speaking as the guy that ran the test, I should explain the "hop count" decrease observed in the test in more detail than the article. First, I should clarify that the 'hop' is a long-distance link between metro areas, because that is the resource that is scarce - we ignored router hops, because they aren't meaningful, and generally aren't visible inside ISP infrastructures for security reasons. This means that data that moves within a metro area is zero hops, data pulled from a directly connected area is one 'hop', and so on.
So in the field testt we saw data transmission distance drop from an average of 5.5 'hops' to 0.89 'hops'. This happens because P4P provides network mapping information, allowing the p2p network to encourage localized data transfers. Generic p2p moved only 6.27% of data within a metro area, while p4p intelligence resulted in 57.98% same-metro area data transfer. Thus deliveries are both faster and cheaper.
The reason that Congress set up copyrights was to promote the progress of science and the useful arts. And why would they want to promote that? Because that's good for the country as a whole. If there is more science and "useful arts", our lives are improved. If there were no copyrights or patents, then there would be less progress in the arts and science, because people wouldn't be rewarded for investing in them. And less art, for example, is bad for all of us, including "consumers".
"Why should customers of a commercial P2P distribution service not only pay for content, but also provide extra bandwidth to help other customers download content?"
So far the answer is that if you're willing to contribute resources to the p2p network you can get access to content that you can't otherwise get, or to get it at a higher quality than you can get without p2p. For an example of the latter, if a video publisher can afford to spend $X per delivery, that revenue number limits what they can afford to spend on delivering the programming. They can allow users to "opt out" and get a low quality download by straight HTTP, where the file is small enough that the cost is acceptable, or "opt in" to p2p and get much higher quality video (which has lower cost due to p2p). So in return for being willing to contribute unused uplink bandwidth, the customer gets access to better content.
"If the film studios & record companies can't get hold of a subscriber's details without a court order then what chance does the BBC have. "
The difference is that the film studios and record companies are just producing the content, which others are distributing against their wishes. The BBC is running the service that's delivering the data. This means that they are serving the bits to the users. This means that they know the IP addresses of those users. So they can use Geo-IP mapping to (98% accuracy) determine the geographical location of the users, and not deliver the data to the users outside of the UK. And, of course, the video that the BBC is distributing is DRM-encoded, and DRM systems have implemented geographic permissions for YEARS, because all content distribution contracts have geographic limitations.
"While there's no seeming penalty, civil or criminal, there is a bigger penalty: ongoing confidence in government."
Yes. but that's not a cost paid by the people that violate the law, it's a cost paid by all of us. In terms of the people making decisions to violate the Presidential Records Act, whose declared goal is to destroy people's confidence in the government, this is another reason to ignore the PRA.
The problem, I believe, is that the Presidential Records Act has no enforcement provisions or penalties for non-compliance. Thus, if the White House prefers to ignore it, there's no risk in doing so. So if the value of non-compliance is higher than the value of compliance, which is the case right now, the PRA loses.
This is not simply a case of incompetent IT staff setting up a system badly. The White House had an email system that by all accounts worked very well, archiving everything properly, and it was shut down and the staff let go, and the new system was set up by someone over-ruling their own IT staff in order to make sure that it couldn't work properly. That means that someone made the decision to spend a lot of time and money to eliminate a system that worked properly, to replace it with a system that didn't, over-ruling the recommendations of their own IT staff, which can only have been done intentionally.
What would be ideal would be for the PRA to be given real teeth so that the cost of violating it becomes clearly higher than the cost of not hiding whatever it is you want hidden. Given the extremely high value of keeping embarrassing or illegal behavior secret, the penalty needs to be extremely high as well, as it is for destroying evidence. That is to say, courts should presume that the records that were destroyed were incriminating. Judges take destroying evidence of a crime quite seriously.
P2P "efficiency" can be viewed from several perspectives.
P2P is always "efficient" from the content server's perspective, because data is delivered between peers instead of central servers (or CDN) reducing delivery costs quite a bit.
P2P consumes the content consumer's uplink, so it's "inefficient" in that sense, but generally uplink is an underutilized resource, and contributing uplink lets p2p work, giving access to large files that couldn't otherwise be provided, because the cost of delivery would be prohibitively high.
The issue is more complex for ISP's. For "traditional" P2P (e.g. BitTorrent) the p2p network is "network oblivious" and picks random data sources. This means that even though someone next door might be able to serve you, you will probably download from all over the planet, not from your neighbor. This makes p2p traffic cost ISP's more than CDN delivery (for example). If the P2P network knows about the ISP's network, then data can be delivered much more efficiently.
The P4P Working Group has many major P2P companies (BitTorrent, Pando, Solid State, Grid, LimeWire, Joost, Verlcix, etc.) and ISP's (Verizon, AT&T, Telefonica, etc.) working together to optimize p2p traffic within ISP infrastructures. So far the early results look very good - users get much faster downloads, while transit between ISP's drops dramatically, and the distance that data moves within ISP's becomes much shorter, thus consuming less of the ISP's network infrastructure.
AT&T Bezeq Intl BitTorrent Cisco Systems Grid Networks Joost LimeWire Manatt Oversi Pando Networks PeerApp Solid State Networks Telefonica Group Velocix VeriSign Verizon Vuze University of Toronto Univ of Washington Yale University
P4PWG Observers:
Abacast AHT Intl AjauntySlant Akamai Alcatel Lucent CableLabs Cablevision Comcast Cox Comm Exa Networks Juniper Networks Lariat Network Level 3 Communications Limelight Networks Microsoft MPAA NBC Universal Nokia Princeton University RawFlow RSUC/GweepNet (?) SaskTel Solana Networks Speakeasy Network Stanford University Thomson Time Warner Cable Turner Broadcasting UCLA Universite Catholique de Louvain
For more information, contact: co-chairs Laird Popkin (laird@pando.com), Doug Pasko (doug.pasko@verizon.com), or Martin Lafferty (marty@dcia.info). Participation in the P4P Working Group is free for ISP's and P2P companies.
There's already a port of Java to the iPhone. To run it on a jailbroken iPhone, first install Cydia (http://www.saurik.com/id/1) and then install iPhone/Java.
It even comes with a simple demo Java app that uses the iPhone frameworks!
Admittedly it's pretty primal, and there's a long way from "JVM runs" to being able to run J2ME app's (like, for example, a GUI layer). But it's still really cool!
" if this were Microsoft, who has about the same track record as Apple (but don't tell the fanbois!), they'd have let you know "Microsoft is late again. Someone should sue them, they're clearly trying to screw all the small-time developer houses who were anxiously awaiting an official SDK. Anti-trust, anti-trust!""
Actually, and speaking as a developer for both Windows and Mac (http://www.pando.com, check it out!), ever since Mac OS X came out Apple has a very good track record for hitting deadlines. Releases sometimes get stuck in QA for a few extra weeks (e.g. Apple TV's latest release, and rumor has it iPhone SDK will be a few weeks late), and they did slip 10.5 by a few months, but I don't recall any massive, multi-year development failures, or repeated slipping, in quite a while. Fairly often there are rumored release dates, which Apple doesn't hit, but Apple itself is pretty cautious about announcing future release dates. Also, when compared to Microsoft, Apple's development model is for frequent, smaller releases, which by definition are lower risk than Microsoft's less frequent, larger releases.
The last time Apple seriously missed a deadline that I can think of was the whole Taligent/Pink debacle.
Let's ignore how we feel about the RIAA, and look at this as a technical issue. There's illegal file copying going on, and the RIAA wants to sue people who are doing it. What can they determine?
The RIAA knows that someone at a given IP address at a given time was making songs available to others for free, because they can query the computer at that address and get back a list of songs on that computer, and they can download the songs and listen to them. So they can prove that IP address X had copyrighted files Y at time Z.
So given that's what they can prove, they face two challenges (IMO, IANAL).
First, they have to prove that there was illegal file copying going on, or that simply making files available is illegal. I'm not a lawyer, so I don't know where the second issue stands. But I think that they can prove that illegal file copying was probably going on, because the alternative is extremely unlikely.
As far as copying goes, if they download identical files from multiple people, that pretty much proves that file copying was going on. This is because, surprisingly enough, when two people RIP the same track from the same CD using the same CODEC and settings, they pretty much always generate unique files, because of random variation in the CD read. Heck, if you RIP the same track repeatedly on the same computer you pretty much always generate unique files each time, for the same reason. So if you see multiple people with a song with the same checksum, it's pretty good evidence of copying, though simply finding the same file on multiple computers doesn't prove who copied from whom.
To figure out who's making the copies, the RIAA could monitor the contents of various PC's over time (file sharing networks broadcast a lot of information) so they could watch as the identical file appears on more PC's, which pretty much documents that illegal file copying was going on, and who was going the copying.
So they can make a petty good case that illegal copying was going on. It'd require them to capture a ton of data, of course, so it'd be expensive, but they could do it.
Second, they have to figure out the identify of the person made the illegal copies. Keeping in mind that all they know is an IP address and a time, about all they can do is to ask the ISP (or sue them, etc.) to determine the identify of the owner of that IP address (or who DHCP assigned that IP address to at that time, etc.), and then sue that person.
This runs into the problem that in a household there are often multiple people behind that IP address. So whenever they sue "whoever was using IP address X" they need to translate that into suing a specific person, and fairly often that specific person won't actually have done anything illegal, because someone else in their household did. They look like the same person to the internet, but are different people in real life.
I think (again, IANAL) that this is similar to a car with license plate X being issued a speeding ticket because it was photographed speeding. There's no doubt that there was speeding going on, and the presumption of the law is that the owner of the car with license place X is responsible for the use of the car unless he can prove that someone else was driving it.
The question is whether the owner of an IP address is responsible for the use of that IP address the way the owner of a car is responsible for the use of the car. Either a "yes" or a "no" answer is problematic. If the answer is yes, then it becomes quite dangerous to operate an open Wifi gateway (for example) or internet cafe, because people can use your IP address to do illegal things that you would be liable for. if the answer is no, then it becomes impossible to punish any illegal online behavior that is "anonymous" but can be tied to IP address (e.g. downloading child porn, extortion, etc.) because of the potential ambiguity of the user's identity.
The key issue is who owns the material. Usually in the writing world, the author is not an employee of the publisher, but indepently owns his own work, which he licenses to a publisher in return for the right to publish it, usually paid royalties based on sales. TV show writers used to be paid this way, until years ago the producers changed the rules such that they owned the work (instead of the writers), in return for which they paid negotiated license fees. Now (IMO) they want to retain the ownership of the work, and have fixed instead of negotiated rates, but avoid paying the writers for it to a large degree, by changing the distribution channel from DVD's to the internet.
The appropriate comparison in the software world is to an independent developer who writes a game that he licenses to a publisher. The publisher wants to license the game by paying 4 cents/box sold in stores, with downloads free, and the ability to declare an unlimited number of boxes as "promotional" with no payment. Would you agree to those terms?
Hmm. For the Star Wars Holiday special to have been "jumping the shark" it has to meet the definition of "jumping the shark" and it has to have taken place after the term "jumping the shark" was in use.
in terms of dates, Fonzie jumped the ahark in September 20, 1977. The Star Wars Holiday Special was broadcast on November 17, 1978. So, technically, it could have been, though the JumpTheShark.com web site claims that the term "jumping the shark" was invented in 1985, in which case the 1978 special couldn't have been "jumping the shark".
In terms of substance, the Star Wars Holiday Special is undoubtedly horrible. But to "jump the shark" a TV show needs to be declining, then attempt an "over the top" stunt in order to try to regain popularity. In 1978 Star Wars was incredibly popular, so the special couldn't have been an attempt to recover declining popularity. And, if you've watched it, there was nothing even remotely "over the top" in the special, just pathetically bad.
So I'd say that, both in terms of dates and substance, the Star Wars Holiday Special wasn't when Star Wars "jumped the shark". It was horrible, certainly, but it was a reverse "jump the shark" in that it was a pathetically bad attempt to capitalize on Star Wars' popularity.
Luckily Lucas was smart enough to prevent it from ever being shown again - it was only broadcast once, and never released on videotape or DVD.
I'd have to think that the vast majority of the cost is in digitizing and cataloguing all of the content. Imagine paying armies of people to go through vaults of aging films and tapes, often unlabeled. First you have to physically handle it all, so that you can play and digitize it. And playing it is harder than it sounds - a lot of old material is recorded in formats that can't be played by anything manufactured in decades, so you have to track down a compatible wire recorder, 8mm film setup, etc., and figure out how to get a high quality recording of the original. And since some originals are in bad shape, they can only be played once. Then you have to pay people to watch it all to build an index so that you could find stuff. The idea of doing this on the scale of the BBC archives is stunning!
Compared to that, the cost of putting it online is minimal. I can believe a few $million, to implement a video content management system, transcode everything into online formats, load everything into the CMS, build a web front-end, and actually run the whole thing (hosting, bandwidth, etc.).
ISP's have a real issue, in that p2p changes usage patterns dramatically. In particular, most ISP's assumed that people would use much more downlink than uplink, which is true for web browsing, email, usenet, etc., and built an infrastructure with 10x more downlink capacity than uplink. And they assumed that people would only be online occasionally, so they build enough capacity to allow people to have a great web browsing experience, but not all at once. But p2p users not only use much more bandwidth than others, they use it continuously, and they use as much uplink as downlink, so they're a terrible fit to the ISP's networks.
The ISP's should get beat up about being "cheap" - they all sell business grade service, with 100% available, symmetrical bandwidth, but it costs much more than consumer service.
The challenge for ISP's is to figure out how to sell internet service at prices that people will pay (cheap, flat-rate) but prevent the p2p usage from crushing their networks.
Some ISP's are building out more capacity, and provisioning symmetrical instead of asymetrical infrastructure. Verizon FIOS' 20/20 plan is a good example of this.
Deploying p2p caching servers. These servers listen to the p2p traffic, and when they have the data someone wants, they return it, taking load off of the internet connection, and giving users a better experience. This saves the ISP bandwidth, and gives users great p2p downloads, but caching servers are expensive (they have to have tons of fast disks to cache a reasonable percentage of p2p data).
Deploying traffic shapers. These servers listen to the p2p traffic and interfere with it in various ways, keeping p2p traffic below a set threshold. So p2p traffic can't overwhelm the ISP's capacity, preserving capacity for VOIP, web browsing, etc. There are lawsuits over this.
Making p2p smarter. The P4P Working Group (http://www.dcia.info/documents/P4PWG_Mission_Statement.pdf) is a forum for ISP's and P2P companies to work together to figure out a strategy for everyone to cooperate to make p2p more efficient. The early research shows that if the p2p network takes advantage of a "map" of the ISP's infrastructure it provides 2x the download performance with a 50% reduction in internet bandwidth consumption, which is an elegant "win-win" scenario. There's a technical writeup at http://www.dcia.info/documents/P4P_Overview.pdf. My hope (I'm the co-chair of the P4P WG) is that smarter p2p networks, supporting P4P, combined with caching servers, can lead to p2p being the most efficient way to deliver data to end users, not only for the content publishers but also for the CDN's and ISP's. If you'd like more information about the P4P WG, email marty@dcia.indo.
Long ago I worked in retail computer sales, and I had someone come in and ask to see a particular piece of software. There was only one copy on the shelf, which they asked me to open and show them, so I opened it for them, installed the software, and spent a half hour or so showing them around the software. This was back in the day when most customers didn't know anything about computers, so I ended up teaching people fairly often, so I didn't mind that part. Then, when I asked whether they wanted it, they said that they didn't want the copy that I'd opened for them, because it was a present for someone. When I pointed out that they'd asked me to open the box for them, they stuck to their guns and refused to buy the copy that they'd asked me to open, because it was open.
Yes, I'm not proud to admit it, but in the face of that "Catch 22" I told them that I'd check the inventory in the back, carried the box out to the shrink wrapping machine, re-wrapped it, came back and sold it to them. Luckily they didn't ask where the other box was.
On that front, I had many customers come in, get my recommendations for software, have me give demo's, even have me train them on the basics of the software, then not buy. That's all fine. But then they would come back in, clearly having bought the software mail order, and have the nerve to ask me more questions. I like helping people, but that's just insulting!
If you hack the firmware on the phone, it's pretty obvious that you won't be able to get warranty support if you bring in your phone with the hacked firmware on it. So if you have a physical problem, restore the factory firmware! I've hacked my TiVo, and I kept the original hard drive available to swap in, in case I need a repair, for just this reason. Anyone who doesn't understand this sort of thing shouldn't be hacking their electronics.
Also, all of this talk about phones being 'bricked' is absurd. A device is bricked when the device is so confused (typically by a bad firmware update breaking the firmware loader) so that it can't be recovered from. That's not what's happening on the iPhone. What's happening when people install the firmware update on the iPhone is that it it's restored to the original condition, meaning that if you used 'jailbreak' to run third part apps, the apps are gone (technically still there, but you can't run them), and if you 'unlocked' the phone so that it's activated on another carrier instead of AT&T it'll go back to 'waiting for activation'. In either case, the phone is not 'bricked' as it is functioning fine just as you bought it - it just doesn't do what you hacked it to do.
If you really don't like it, feel free to help find a mechanism for 'jailbreaking' or 'unlocking' the iPhone 1.1.1 firmware, so the game of 'cat and mouse' continues.
This move isn't about blocking Linux iPod users - it's broader than that - it blocks all third-party applications from putting music onto the iPod. The fact that some of those applications run on Linux is probably irrelevant to Apple, because Linux users who don't also have a PC or Mac are probably a very small percentage of Apple's potential sales.
So, why would Apple want to block third-party apps from writing to iPods? Let's speculate:
- Apple might be getting customer support calls from people who corrupt their iPod databases. So by blocking third-party apps, Apple is reducing support costs. - Apple is about to make major changes to iTunes, and to the iPod database format, and needs to keep third-party apps from corrupting the new databases. - Apple wants to be the only way that music gets on iPods for some business reason.
The flaw in this logic is in assuming that MS-DOS is what allowed microcomputers to become mass-market items.
In actuality, the thing that drove microcomputers into the mainstream was the dramatic increase in performance and decrease in price due to hardware advances. The better, cheaper hardware 'tipped the scales' and gave software companies a platform for their products. Of course, without software the hardware wouldn't have sold, but there were many competitive alternatives that were moving computers into the mainstream well before Microsoft was formed. So while Microsoft can take credit for beating its competition, if MS-DOS hadn't been on the market, mainstream computing would have been enabled by one of the numerous and functionally comparable competitors such as Apple's ProDOS, CPM, QDOS (renamed MS-DOS after MS bought it), DR-DOS, GEM, TOS, AmigaDOS, etc.
Most people don't remember this now, but the original PC shipped with three operating systems - PC-DOS, CPM-86 and the UCSD p-System. Any one of them could have made the PC mainstream.
IMO, the real thing that first pushed microcomputers into the mainstream was the Apple II - well before the PC shipped, "normal" people were buying Apple computers to do word processing, run spreasheets, play games, etc. In terms of mainstream adoption, the IBM PC's adoption was mainly driven by IBM legitimizing microcomputers for business use, and in particular by Lotus 123, followed by the emergence of PC clones, none of which was fundamentally based on anything that Microsoft did that wasn't also being done by competitors.
Slashdot Mirror
"If they really want to help p2p, then they would expose this topology information to us p2p developers, and let us use it to make all our applications better. What they're likely planning is pushing their own p2p..."
P4P isn't a p2p network. P4P is an open standard that can be implemented by any ISP and any p2p network, and which has been tested so far on BitTorrent (protocol, not company) and Pando software, and the Verizon and Telefonica networks. Participants include all of the major P2P companies and many major ISP's. Participation in the P4P Working Group is open (and free) to any P2P company or ISP. Email marty@dcia.info, laird@pando.com, or doug.pasko@verizon.com if you're interested in joining the working group, or in getting email updates.
There's more information at http://www.pandonetworks.com/p4p and at http://www.dcia.info/activities/.
"from an endpoint perspective, it's tough to extract network topology and bandwidth"
This is exactly right. There are many interesting strategies for attempting to derive network topology, but they all have weaknesses in various situations, such as when ISP's use OSPF. And observed network behavior can't reflect ISP business policies (e.g. they might prefer to route p2p over cheap links instead of expensive links that are more suited to VOIP and gaming).
It turns out that ISP's are open to the idea of providing precise network information, if done in a way that preserves their security, if it gives them a significant reduction in p2p network traffic. When we asked ISP's at NANOG whether they'd be interested in contributing network data in order to test P4P, about half of the people in the room raised their hands, so there appears (IMO) to be a potential willingness to provide this information.
You're missing the point of P4P. What P4P does is provide information to the P2P network so that it can shorten the distance that data travels through the network. Data that is moved very short distances (e.g. in a local fiber loop) moves much faster than data that is moved long distances (e.g. across the Pacific Ocean). Without P4P, the p2p network doesn't have enough information to determine which IP's are near each other, so on average data moves a very long distance, when the same data was available locally. With P4P, the p2p network can make the local connections. Data that moves short distances only only arrives faster, it consumes less of the ISP's infrastructure.
:-)
What we found in the field test is that without P4P, 6% of downloaded data came from the same metro area, and the rest came from further away, or from other ISP's (costing money or consuming scarce/expensive resources). With P4P guidance, 58% of data downloaded came from within the downloader's metro area, dramatically reducing the amount of data being moved across long distance links or from other ISP's.
The software change to implement P4P was pretty simple. Basically it's a Tracker enhancement - it requires no client change. So it's easier to implement than increasing the capacity of an ISP's backbone. Not that I'd want to discourage ISP's from increasing their capacity, of course.
"Why not just do a 'traceroute' to all of the seeds as you discover them, and penalize the ones that are more hops away?"
That would help peers pick between known peers to exchange data with. The problem is that if you're in a large swarm, you'll only know about a small subset of the swarm, and thus almost certainly miss the best peers to connect to. For example, if you're in a swarm with 10,000 peers, and you know about a random 50 peers, you are 99.5% likely not to find out about the closest peer on the first announce (for BitTorrent, which I'll use as the example, since it's well known). The Tracker has global knowledge, so it can tell a peer 100% of the time about the closest peer. Yes, it's true that over time BitTorrent will converge on good data sources, but in large swarms it takes a very long time to connect to and test all peers, so the time that it takes to find a good, nearby data source could well be much longer than the download time,
What we found in the P4P field test is that guided peer connections yielded much faster download speeds almost immediately, because the first peer connection was "close" in the ISP's network, resulting in fast connection and transfer times, and that while the BitTorrent connection logic eventually found good data sources, on average the downloads were over 200% faster (for FTTH users) when the p2p connections were guided.
The P4P information can apply to any P2P protocol. P4P is a protocol, but it's a protocol between the P2P provider's control servers and the ISP, not between clients. Basically, the ISP gives "hints" to the P2P network so that it can figure out which IP's are near each other, and it uses that to guide the p2p connections. For BitTorrent, there's communication between the Tracker and the P4P server. But P4P should in principle be able to apply to any P2P protocol.
There are some diagrams at http://www.pandonetworks.com/p4p.
"I don't really see how this could be done on a software level... I think thats why they're citing legal content only... it will take some modifications for routing equipment, won't it?"
Nope, one of the appeals of the approach is that it can be implemented at the application level, so there don't need to be any changes to routers, etc. The ISP tells the p2p network how to figure out which IP's are near each other, and the P2P network preferentially connects those IP's rather than using random IP's.
Speaking as the guy that ran the test, I should explain the "hop count" decrease observed in the test in more detail than the article. First, I should clarify that the 'hop' is a long-distance link between metro areas, because that is the resource that is scarce - we ignored router hops, because they aren't meaningful, and generally aren't visible inside ISP infrastructures for security reasons. This means that data that moves within a metro area is zero hops, data pulled from a directly connected area is one 'hop', and so on.
So in the field testt we saw data transmission distance drop from an average of 5.5 'hops' to 0.89 'hops'. This happens because P4P provides network mapping information, allowing the p2p network to encourage localized data transfers. Generic p2p moved only 6.27% of data within a metro area, while p4p intelligence resulted in 57.98% same-metro area data transfer. Thus deliveries are both faster and cheaper.
The reason that Congress set up copyrights was to promote the progress of science and the useful arts. And why would they want to promote that? Because that's good for the country as a whole. If there is more science and "useful arts", our lives are improved. If there were no copyrights or patents, then there would be less progress in the arts and science, because people wouldn't be rewarded for investing in them. And less art, for example, is bad for all of us, including "consumers".
"Why should customers of a commercial P2P distribution service not only pay for content, but also provide extra bandwidth to help other customers download content?"
So far the answer is that if you're willing to contribute resources to the p2p network you can get access to content that you can't otherwise get, or to get it at a higher quality than you can get without p2p. For an example of the latter, if a video publisher can afford to spend $X per delivery, that revenue number limits what they can afford to spend on delivering the programming. They can allow users to "opt out" and get a low quality download by straight HTTP, where the file is small enough that the cost is acceptable, or "opt in" to p2p and get much higher quality video (which has lower cost due to p2p). So in return for being willing to contribute unused uplink bandwidth, the customer gets access to better content.
That's just cool. I love Slashdot!
"If the film studios & record companies can't get hold of a subscriber's details without a court order then what chance does the BBC have. "
The difference is that the film studios and record companies are just producing the content, which others are distributing against their wishes. The BBC is running the service that's delivering the data. This means that they are serving the bits to the users. This means that they know the IP addresses of those users. So they can use Geo-IP mapping to (98% accuracy) determine the geographical location of the users, and not deliver the data to the users outside of the UK. And, of course, the video that the BBC is distributing is DRM-encoded, and DRM systems have implemented geographic permissions for YEARS, because all content distribution contracts have geographic limitations.
"While there's no seeming penalty, civil or criminal, there is a bigger penalty: ongoing confidence in government."
Yes. but that's not a cost paid by the people that violate the law, it's a cost paid by all of us. In terms of the people making decisions to violate the Presidential Records Act, whose declared goal is to destroy people's confidence in the government, this is another reason to ignore the PRA.
The problem, I believe, is that the Presidential Records Act has no enforcement provisions or penalties for non-compliance. Thus, if the White House prefers to ignore it, there's no risk in doing so. So if the value of non-compliance is higher than the value of compliance, which is the case right now, the PRA loses.
This is not simply a case of incompetent IT staff setting up a system badly. The White House had an email system that by all accounts worked very well, archiving everything properly, and it was shut down and the staff let go, and the new system was set up by someone over-ruling their own IT staff in order to make sure that it couldn't work properly. That means that someone made the decision to spend a lot of time and money to eliminate a system that worked properly, to replace it with a system that didn't, over-ruling the recommendations of their own IT staff, which can only have been done intentionally.
What would be ideal would be for the PRA to be given real teeth so that the cost of violating it becomes clearly higher than the cost of not hiding whatever it is you want hidden. Given the extremely high value of keeping embarrassing or illegal behavior secret, the penalty needs to be extremely high as well, as it is for destroying evidence. That is to say, courts should presume that the records that were destroyed were incriminating. Judges take destroying evidence of a crime quite seriously.
P2P "efficiency" can be viewed from several perspectives.
:-)
P2P is always "efficient" from the content server's perspective, because data is delivered between peers instead of central servers (or CDN) reducing delivery costs quite a bit.
P2P consumes the content consumer's uplink, so it's "inefficient" in that sense, but generally uplink is an underutilized resource, and contributing uplink lets p2p work, giving access to large files that couldn't otherwise be provided, because the cost of delivery would be prohibitively high.
The issue is more complex for ISP's. For "traditional" P2P (e.g. BitTorrent) the p2p network is "network oblivious" and picks random data sources. This means that even though someone next door might be able to serve you, you will probably download from all over the planet, not from your neighbor. This makes p2p traffic cost ISP's more than CDN delivery (for example). If the P2P network knows about the ISP's network, then data can be delivered much more efficiently.
The P4P Working Group has many major P2P companies (BitTorrent, Pando, Solid State, Grid, LimeWire, Joost, Verlcix, etc.) and ISP's (Verizon, AT&T, Telefonica, etc.) working together to optimize p2p traffic within ISP infrastructures. So far the early results look very good - users get much faster downloads, while transit between ISP's drops dramatically, and the distance that data moves within ISP's becomes much shorter, thus consuming less of the ISP's network infrastructure.
There's an overview of P4P at http://www.pandonetworks.com/p4p, and a presentation on P4P that was just presented at NANOG at http://nanog.org/mtg-0802/presentations/PopkinPasko_Presentation.pdf. The presentation covers the technology and the numbers in more depth than a post on Slashdot can.
Current membership is P4P Working Group:
AT&T
Bezeq Intl
BitTorrent
Cisco Systems
Grid Networks
Joost
LimeWire
Manatt
Oversi
Pando Networks
PeerApp
Solid State Networks
Telefonica Group
Velocix
VeriSign
Verizon
Vuze
University of Toronto
Univ of Washington
Yale University
P4PWG Observers:
Abacast
AHT Intl
AjauntySlant
Akamai
Alcatel Lucent
CableLabs
Cablevision
Comcast
Cox Comm
Exa Networks
Juniper Networks
Lariat Network
Level 3 Communications
Limelight Networks
Microsoft
MPAA
NBC Universal
Nokia
Princeton University
RawFlow
RSUC/GweepNet (?)
SaskTel
Solana Networks
Speakeasy Network
Stanford University
Thomson
Time Warner Cable
Turner Broadcasting
UCLA
Universite Catholique de Louvain
For more information, contact: co-chairs Laird Popkin (laird@pando.com), Doug Pasko (doug.pasko@verizon.com), or Martin Lafferty (marty@dcia.info). Participation in the P4P Working Group is free for ISP's and P2P companies.
There's already a port of Java to the iPhone. To run it on a jailbroken iPhone, first install Cydia (http://www.saurik.com/id/1) and then install iPhone/Java.
It even comes with a simple demo Java app that uses the iPhone frameworks!
Admittedly it's pretty primal, and there's a long way from "JVM runs" to being able to run J2ME app's (like, for example, a GUI layer). But it's still really cool!
" if this were Microsoft, who has about the same track record as Apple (but don't tell the fanbois!), they'd have let you know "Microsoft is late again. Someone should sue them, they're clearly trying to screw all the small-time developer houses who were anxiously awaiting an official SDK. Anti-trust, anti-trust!""
Actually, and speaking as a developer for both Windows and Mac (http://www.pando.com, check it out!), ever since Mac OS X came out Apple has a very good track record for hitting deadlines. Releases sometimes get stuck in QA for a few extra weeks (e.g. Apple TV's latest release, and rumor has it iPhone SDK will be a few weeks late), and they did slip 10.5 by a few months, but I don't recall any massive, multi-year development failures, or repeated slipping, in quite a while. Fairly often there are rumored release dates, which Apple doesn't hit, but Apple itself is pretty cautious about announcing future release dates. Also, when compared to Microsoft, Apple's development model is for frequent, smaller releases, which by definition are lower risk than Microsoft's less frequent, larger releases.
The last time Apple seriously missed a deadline that I can think of was the whole Taligent/Pink debacle.
Let's ignore how we feel about the RIAA, and look at this as a technical issue. There's illegal file copying going on, and the RIAA wants to sue people who are doing it. What can they determine?
The RIAA knows that someone at a given IP address at a given time was making songs available to others for free, because they can query the computer at that address and get back a list of songs on that computer, and they can download the songs and listen to them. So they can prove that IP address X had copyrighted files Y at time Z.
So given that's what they can prove, they face two challenges (IMO, IANAL).
First, they have to prove that there was illegal file copying going on, or that simply making files available is illegal. I'm not a lawyer, so I don't know where the second issue stands. But I think that they can prove that illegal file copying was probably going on, because the alternative is extremely unlikely.
As far as copying goes, if they download identical files from multiple people, that pretty much proves that file copying was going on. This is because, surprisingly enough, when two people RIP the same track from the same CD using the same CODEC and settings, they pretty much always generate unique files, because of random variation in the CD read. Heck, if you RIP the same track repeatedly on the same computer you pretty much always generate unique files each time, for the same reason. So if you see multiple people with a song with the same checksum, it's pretty good evidence of copying, though simply finding the same file on multiple computers doesn't prove who copied from whom.
To figure out who's making the copies, the RIAA could monitor the contents of various PC's over time (file sharing networks broadcast a lot of information) so they could watch as the identical file appears on more PC's, which pretty much documents that illegal file copying was going on, and who was going the copying.
So they can make a petty good case that illegal copying was going on. It'd require them to capture a ton of data, of course, so it'd be expensive, but they could do it.
Second, they have to figure out the identify of the person made the illegal copies. Keeping in mind that all they know is an IP address and a time, about all they can do is to ask the ISP (or sue them, etc.) to determine the identify of the owner of that IP address (or who DHCP assigned that IP address to at that time, etc.), and then sue that person.
This runs into the problem that in a household there are often multiple people behind that IP address. So whenever they sue "whoever was using IP address X" they need to translate that into suing a specific person, and fairly often that specific person won't actually have done anything illegal, because someone else in their household did. They look like the same person to the internet, but are different people in real life.
I think (again, IANAL) that this is similar to a car with license plate X being issued a speeding ticket because it was photographed speeding. There's no doubt that there was speeding going on, and the presumption of the law is that the owner of the car with license place X is responsible for the use of the car unless he can prove that someone else was driving it.
The question is whether the owner of an IP address is responsible for the use of that IP address the way the owner of a car is responsible for the use of the car. Either a "yes" or a "no" answer is problematic. If the answer is yes, then it becomes quite dangerous to operate an open Wifi gateway (for example) or internet cafe, because people can use your IP address to do illegal things that you would be liable for. if the answer is no, then it becomes impossible to punish any illegal online behavior that is "anonymous" but can be tied to IP address (e.g. downloading child porn, extortion, etc.) because of the potential ambiguity of the user's identity.
Thoughts?
The key issue is who owns the material. Usually in the writing world, the author is not an employee of the publisher, but indepently owns his own work, which he licenses to a publisher in return for the right to publish it, usually paid royalties based on sales. TV show writers used to be paid this way, until years ago the producers changed the rules such that they owned the work (instead of the writers), in return for which they paid negotiated license fees. Now (IMO) they want to retain the ownership of the work, and have fixed instead of negotiated rates, but avoid paying the writers for it to a large degree, by changing the distribution channel from DVD's to the internet.
The appropriate comparison in the software world is to an independent developer who writes a game that he licenses to a publisher. The publisher wants to license the game by paying 4 cents/box sold in stores, with downloads free, and the ability to declare an unlimited number of boxes as "promotional" with no payment. Would you agree to those terms?
Hmm. For the Star Wars Holiday special to have been "jumping the shark" it has to meet the definition of "jumping the shark" and it has to have taken place after the term "jumping the shark" was in use.
in terms of dates, Fonzie jumped the ahark in September 20, 1977. The Star Wars Holiday Special was broadcast on November 17, 1978. So, technically, it could have been, though the JumpTheShark.com web site claims that the term "jumping the shark" was invented in 1985, in which case the 1978 special couldn't have been "jumping the shark".
In terms of substance, the Star Wars Holiday Special is undoubtedly horrible. But to "jump the shark" a TV show needs to be declining, then attempt an "over the top" stunt in order to try to regain popularity. In 1978 Star Wars was incredibly popular, so the special couldn't have been an attempt to recover declining popularity. And, if you've watched it, there was nothing even remotely "over the top" in the special, just pathetically bad.
So I'd say that, both in terms of dates and substance, the Star Wars Holiday Special wasn't when Star Wars "jumped the shark". It was horrible, certainly, but it was a reverse "jump the shark" in that it was a pathetically bad attempt to capitalize on Star Wars' popularity.
Luckily Lucas was smart enough to prevent it from ever being shown again - it was only broadcast once, and never released on videotape or DVD.
I'd have to think that the vast majority of the cost is in digitizing and cataloguing all of the content. Imagine paying armies of people to go through vaults of aging films and tapes, often unlabeled. First you have to physically handle it all, so that you can play and digitize it. And playing it is harder than it sounds - a lot of old material is recorded in formats that can't be played by anything manufactured in decades, so you have to track down a compatible wire recorder, 8mm film setup, etc., and figure out how to get a high quality recording of the original. And since some originals are in bad shape, they can only be played once. Then you have to pay people to watch it all to build an index so that you could find stuff. The idea of doing this on the scale of the BBC archives is stunning!
Compared to that, the cost of putting it online is minimal. I can believe a few $million, to implement a video content management system, transcode everything into online formats, load everything into the CMS, build a web front-end, and actually run the whole thing (hosting, bandwidth, etc.).
ISP's have a real issue, in that p2p changes usage patterns dramatically. In particular, most ISP's assumed that people would use much more downlink than uplink, which is true for web browsing, email, usenet, etc., and built an infrastructure with 10x more downlink capacity than uplink. And they assumed that people would only be online occasionally, so they build enough capacity to allow people to have a great web browsing experience, but not all at once. But p2p users not only use much more bandwidth than others, they use it continuously, and they use as much uplink as downlink, so they're a terrible fit to the ISP's networks.
The ISP's should get beat up about being "cheap" - they all sell business grade service, with 100% available, symmetrical bandwidth, but it costs much more than consumer service.
The challenge for ISP's is to figure out how to sell internet service at prices that people will pay (cheap, flat-rate) but prevent the p2p usage from crushing their networks.
Some ISP's are building out more capacity, and provisioning symmetrical instead of asymetrical infrastructure. Verizon FIOS' 20/20 plan is a good example of this.
Deploying p2p caching servers. These servers listen to the p2p traffic, and when they have the data someone wants, they return it, taking load off of the internet connection, and giving users a better experience. This saves the ISP bandwidth, and gives users great p2p downloads, but caching servers are expensive (they have to have tons of fast disks to cache a reasonable percentage of p2p data).
Deploying traffic shapers. These servers listen to the p2p traffic and interfere with it in various ways, keeping p2p traffic below a set threshold. So p2p traffic can't overwhelm the ISP's capacity, preserving capacity for VOIP, web browsing, etc. There are lawsuits over this.
Making p2p smarter. The P4P Working Group (http://www.dcia.info/documents/P4PWG_Mission_Statement.pdf) is a forum for ISP's and P2P companies to work together to figure out a strategy for everyone to cooperate to make p2p more efficient. The early research shows that if the p2p network takes advantage of a "map" of the ISP's infrastructure it provides 2x the download performance with a 50% reduction in internet bandwidth consumption, which is an elegant "win-win" scenario. There's a technical writeup at http://www.dcia.info/documents/P4P_Overview.pdf. My hope (I'm the co-chair of the P4P WG) is that smarter p2p networks, supporting P4P, combined with caching servers, can lead to p2p being the most efficient way to deliver data to end users, not only for the content publishers but also for the CDN's and ISP's. If you'd like more information about the P4P WG, email marty@dcia.indo.
Long ago I worked in retail computer sales, and I had someone come in and ask to see a particular piece of software. There was only one copy on the shelf, which they asked me to open and show them, so I opened it for them, installed the software, and spent a half hour or so showing them around the software. This was back in the day when most customers didn't know anything about computers, so I ended up teaching people fairly often, so I didn't mind that part. Then, when I asked whether they wanted it, they said that they didn't want the copy that I'd opened for them, because it was a present for someone. When I pointed out that they'd asked me to open the box for them, they stuck to their guns and refused to buy the copy that they'd asked me to open, because it was open.
Yes, I'm not proud to admit it, but in the face of that "Catch 22" I told them that I'd check the inventory in the back, carried the box out to the shrink wrapping machine, re-wrapped it, came back and sold it to them. Luckily they didn't ask where the other box was.
On that front, I had many customers come in, get my recommendations for software, have me give demo's, even have me train them on the basics of the software, then not buy. That's all fine. But then they would come back in, clearly having bought the software mail order, and have the nerve to ask me more questions. I like helping people, but that's just insulting!
This is all stupid.
If you hack the firmware on the phone, it's pretty obvious that you won't be able to get warranty support if you bring in your phone with the hacked firmware on it. So if you have a physical problem, restore the factory firmware! I've hacked my TiVo, and I kept the original hard drive available to swap in, in case I need a repair, for just this reason. Anyone who doesn't understand this sort of thing shouldn't be hacking their electronics.
Also, all of this talk about phones being 'bricked' is absurd. A device is bricked when the device is so confused (typically by a bad firmware update breaking the firmware loader) so that it can't be recovered from. That's not what's happening on the iPhone. What's happening when people install the firmware update on the iPhone is that it it's restored to the original condition, meaning that if you used 'jailbreak' to run third part apps, the apps are gone (technically still there, but you can't run them), and if you 'unlocked' the phone so that it's activated on another carrier instead of AT&T it'll go back to 'waiting for activation'. In either case, the phone is not 'bricked' as it is functioning fine just as you bought it - it just doesn't do what you hacked it to do.
If you really don't like it, feel free to help find a mechanism for 'jailbreaking' or 'unlocking' the iPhone 1.1.1 firmware, so the game of 'cat and mouse' continues.
This move isn't about blocking Linux iPod users - it's broader than that - it blocks all third-party applications from putting music onto the iPod. The fact that some of those applications run on Linux is probably irrelevant to Apple, because Linux users who don't also have a PC or Mac are probably a very small percentage of Apple's potential sales.
So, why would Apple want to block third-party apps from writing to iPods? Let's speculate:
- Apple might be getting customer support calls from people who corrupt their iPod databases. So by blocking third-party apps, Apple is reducing support costs.
- Apple is about to make major changes to iTunes, and to the iPod database format, and needs to keep third-party apps from corrupting the new databases.
- Apple wants to be the only way that music gets on iPods for some business reason.
The flaw in this logic is in assuming that MS-DOS is what allowed microcomputers to become mass-market items.
In actuality, the thing that drove microcomputers into the mainstream was the dramatic increase in performance and decrease in price due to hardware advances. The better, cheaper hardware 'tipped the scales' and gave software companies a platform for their products. Of course, without software the hardware wouldn't have sold, but there were many competitive alternatives that were moving computers into the mainstream well before Microsoft was formed. So while Microsoft can take credit for beating its competition, if MS-DOS hadn't been on the market, mainstream computing would have been enabled by one of the numerous and functionally comparable competitors such as Apple's ProDOS, CPM, QDOS (renamed MS-DOS after MS bought it), DR-DOS, GEM, TOS, AmigaDOS, etc.
Most people don't remember this now, but the original PC shipped with three operating systems - PC-DOS, CPM-86 and the UCSD p-System. Any one of them could have made the PC mainstream.
IMO, the real thing that first pushed microcomputers into the mainstream was the Apple II - well before the PC shipped, "normal" people were buying Apple computers to do word processing, run spreasheets, play games, etc. In terms of mainstream adoption, the IBM PC's adoption was mainly driven by IBM legitimizing microcomputers for business use, and in particular by Lotus 123, followed by the emergence of PC clones, none of which was fundamentally based on anything that Microsoft did that wasn't also being done by competitors.