Those are very conservative numbers. Even the most naive of heuristics elminate a huge fraction of those possible "states". Another huge fraction are simply not realizable given the constraints on how peices are allowed to move.
A naive calculation of the state space size has little to do with computational difficulty.
There *are* many advantages to using HTTP/webserver model for filesharing (and not just for *copyrighted* filesharing, BTW -- filesharing isn't just for piracy!).
But systems such as YouServ are blurring the line by merging the best of P2P and webserving models.
Re:I think project IRIS from MIT is more interesti
on
DNSSEC: Good Enough?
·
· Score: 2, Informative
Time for ICANN to be obsoleted by a nice DARPA funded project from MIT and Berkeley. The guys working on it are pretty bright, and DNS is what distributed hash tables are best for.
Try again. IRIS hasn't proposed a thing that can solve DNS security issues. It might address decentralizing the mostly hierarchical lookup procedure (to address scalability for example), but this would in fact require something like DNSSEC so that DNS records could be verified as legitimate even when provided by untrusted/unauthoritative hosts in a DHT.
Wouldn't working on a improved form of SMTP be a better project?
No. DNS security impacts much more than just e-mail. Note for example that few websites use HTTPS, so DNSSEC would go a long way towards improving web security as well.
Apart from the university setting, who else is out there?
Not many others. IBM Research is still going strong, but it's generally more focused on shorter term research goals than PARC/Bell Labs. But that's probably why it hasn't gone the way of PARC & Bell Labs...
The computation performed by Seti@Home is what Grid researchers refer to as an "embarassingly parallel".
Among many other things, Grid folks hope to solve problems that aren't quite so amenable to divide-and-conquer. But then they had to go base their protocols on the bloated Web services stack, implying a relatively high granularity per compute unit. So we'll see how well that works out!
Though there exists a known algorithm to factorise primes in polynomial time, which would certainly make almost all cryptographic systems obsolete..
Huh? There is a polynomial time algorithm for determining primality (if that's what you mean by "factorise primes"). But this can only *help* cryptographic algorithm relying on hardness of factoring. It renders nothing obsolete because there is no known poly-time algorithm for factorizing numbers in general.
... be aware the IE limits max connections to a single server to 2. So if you open up two big concurrent downloads, IE won't let you access anything else on the site until one of the downloads completes. It doesn't give a warning or anything, it just hangs there waiting for the download to terminate.... A solution is to use a different domain name for your large files to prevent the browser from failing to load your "regular" pages.
Another issue with IE is that it seems to ignore the content-length header. Thus if the connection closes early for whatever reason, IE will treat the download as "complete" even if the length doesn't match content-length. This has caused me more than one corrupt download in the past.
HTTP is as reliable and fast as FTP, but shitty browsers (IE) can still cause problems. This isn't a fundamental limitation of HTTP though.
Having actually seen the SAINT-2003 paper on which I'm assuming this article is based, the approach is indeed related to the standard "aquire all locks in a predetermined order" strategy.
However it's not exactly that. If I remember correctly:-), it's a variant of this strategy that allows a bit more flexibility in aquiring locks (and hence more parallelism) in certain circumstances. These circumstances, from what I recall, are when a service can aquire "any one" of a group of resources in order to get its task done (which is perhaps a reasonable assumption if we're to believe the web services hype of multiple providers, yada yada). In the more constrained case where a service needs to aquire a fixed set of specific resources, it degenerates into the simple order-based deadlock/livelock prevention scheme.
Now from the article he claims to have further refined the technique, so by now there could be more to it. But I believe this is still likely to be the general idea.
Revolutionary? I don't think so, but it's really too early to tell. Indeed as another author posted, lock aquisition has not yet
proven to be a bottleneck in web services, but that could be because the vision of having multiple providers offering multiple (possibly equivalent) services has yet to materialize. Complex web services orchestration is something that is more hype than reality at this point, but that could change.
Downgrade to http1.0 & disabling keepalives is a valid workaround, but it will also make any MS browser visiting the site (when using SSL) perform abysmally. Ever tried to load a page with multiple (small) images from a site without keepalive support? Yikes... I'm really surprised this would be the standard configuration to work around this IE bug, as I think you can more simply work around it by disabling ssl3.0 for MSIE. The fix I use, which probably has the least performance problems (for the client at least), is a 65 second or greater keep-alive timeout, but this does make your webserver a bit more prone to some DOS attacks and also increases its load. Not a problem if you're not a high traffic site though.
..of Microsoft browser networking bugs which make it only work well with IIS. For example, This bug causes IE to fail to properly shutdown SSL connections. IE browsers using SSL conenctions with standard Apache webserver configurations will have all kinds of errors due to this issue. You need to either disable keepalives or increase the keepalive timeout to something outrageous like 2 minutes. This "bug" has been around for ages yet despite IE being in version 6, it is yet to be fixed. My guess is this is actually some kind of "feature" that makes IE work faster with IIS (since the connection never closes, subsequent reqests go faster, assuming the webserver knows how to speak the broken protocol).
I have the 384kbps symmetric DSL service from Speakeasy (which uses Covad's lines). I get just under 40k/s download speeds, but unfortunately my upstream speeds max out at more like 29k/s. Other than this slow upstream issue it's been a fine service. But I must admit since I run a webserver from my machine, this slow upstream speed is making me look at alternatives....
Re:P2O vs the Slashdot effect
on
Putting P2P To Work
·
· Score: 2, Informative
Working on it. See the "distributed content caching" plugin example described in the "peer to peer sharing of web applications" paper on the YouServ project page.
It will let you encode any YouServ URL so that it gets cached and served by an army of available machines instead of just your own machine.
Because it is a centrally coordinated system, fragmentation of the network is not possible and search results are almost always "complete" (that is, if what you are searching for is out there and available at the time of your search, you're almost guaranteed to find it). Check out the paper on the search function for details.
Yes, this means it does exhibit a single point of failure, but as the system is not intended for piracy, porn, evading censorship, or other uses that tend to get systems "shut down", the benefits of centrally coordinating the system outweigh the functionality and performance limitations of a fully decentralized approach.
The difference from this and a completely server-based solution is that there are no central requirements that involve a large amount of resources. The central servers are a pair of old 400MHZ PII boxes, one for DNS, and one for the coordinating services. A one man operation can easily run this system on cheap hardware and a pipe with only moderate bandwidth, and still serve tens if not hundreds of thousands of users.
Groupwise aggregates all content at a central server, whereas in YouServ, all content is stored at and served from end-user machines, hence its designation as p2p.
To be completely precise, YouServ is more of a Napster-like "brokered" form of p2p, as opposed to a purely decentralized architecture like Gnutella. For example, to be completely web compatible, it relies on DNS for peer location, and DNS requires a dedicated server (for a particular namepspace). But almost all the "real work" is still done by the peers.
Kick ass indeed. Unfortunately the CMU deployment of YouServ does not support the p2p search component. But then this limitation is also probably why this deployment hasn't reached a critical mass of users.
I'll leave the reasons behind this feature omission to your imagination.:-)
Re:For a real challenge, try P2P-ing the database
on
Putting P2P To Work
·
· Score: 5, Informative
As a developer of the system, I can tell you that a p2p database is not, nor will it be, one of the goals of YouServ.
The system is intended for personal web hosting, that is, for use by mostly non-technical users for sharing files, creating web logs, guest books, and so on, using their own hardware (and that of their friends). In no way are we trying to provide p2p solutions to business class functions such as serving an online store (though I agree it would be cool if someone did that).
Indeed the existing plugins are simple (my development time has been limited and this component is very new), and at this point are intended only to demonstrate the API features. But even quite simple plugins, if they cooperate across multiple machines, can do some very cool things, a few examples of which are proposed in the paper: sharing files on multiple other p2p networks, distributed content caching ("akamai for free"), cooperative weblogging, and so on. Again, you woudln't use this to build a high-traffic online store, but it does give you many new and easy to use ways to enhance and publish your own (web) content.
There's a lot in YouServ that is not in JXTA. YouServ is an application, JXTA is more about infrastructure with a loose collection of apps atop it, none of which provide all the features offered by YouServ (web hosting with transparent site failover to buddy machines, dynamic dns, integrated authentication system and certificate authority for single sign-on and support of HTTPS, browser compatible access, p2p search over file *content* as well as filenames, and so on...)
Sure you could probably build something like this with JXTA, but nobody has done it yet.
Re:Oh oh! Not compatible? Bahhhh
on
Putting P2P To Work
·
· Score: 3, Informative
That's a pretty minor issue since it only affects HTTPS/SSL (which is used for file uploads to protect auth credentials).
A worse one (for YouServ compatability and anything else reyling on dynamic DNS) is the fact that Mozilla caches IP addresses until the browser is completely restarted. How's that for stupid?
Every YouServ node is granted an SSL certificate from an integrated certificate authority, and uses it for support of HTTPS/SSL. Use of HTTPS guarantees there is no man in the middle.
YouServ is a bit different than traditional P2P apps most people are familiar with. For one, there is accountability: Every one of the files you share has YOUR NAME embedded within the URL required for accessing that file, even if that file resides on another machine at the time of download (e.g. from the ability to replicate your site to other machines). Unless you don't value your job, you'd be highly unlikely to use this system for sharing porn / MP3's / etc, a point made in the paper on the search function.
Two, it's not just for sharing files that are world readable. It's also for sharing stuff with only designated users and groups. Every node is tied together by a single sign-on infrastructure so you can use one and only one password to access secured content on any node, without the possibility of malicious nodes sniffing and stashing your password.
Third, it's at its core web compatible. You access the network with a regular web browser. There
s no need to install spyware riddled clients to get at anything. Each node provides a web accessible search interface for searching the globally shared content or site-specific content. You only install the software if you want to host stuff.
Yes, applets sucked, and yes they were slow and buggy, but why? Because thanks to MS, browsers never incorporated any updates to the original bug-riddled JVM's.
With a monopolistic market share, unless MS upgrades its browser JVM (which everyone must code to), then nobody else need bother. MS knew that upgrading its JVM might actually make Java useful, and give some kind of "advantage" to Sun and other Java proponents. This sort of anti-competitive behavior is the problem. It's quite a bit different than Microsoft refusing to put some niche-geek language like Mesa into its browser.
Applets are plenty fast and reliable with Sun's recent Java plugins. The web would be a much better place if Microsoft hadn't stifled Java's adoption on the client side.
Well, our mutual stupidity aside, I think I agree with you about not trusting a javascript thingy... realistically for this thing to be deployed in a trustable manner I believe it would require some browser / protocol extension (along the lines of HTTPS). Thus just like people trust the little "lock" icon in their browsers (whether they should or not:) there could be another indicator that only privacy-preserved information would be provided to the site.
In more limited application domains this wouldn't be as big a problem.
Maybe it's a pipe dream, but it's still an interesting idea.
>Interesting approach, but useless unless people >actually understand and trust the system.
I don't know... people use HTTPS/SSL all the time. You really think more than.01% of the population understand the subtleties of certificate authorities, public / private key encryption, etc.? Yet still it's far from useless.
As long as people trust that it works, there's no need to actually understand it. I think this trust would be best provided by open source implementations of schemes such as this.
> As another poster observes, if you don't trust > them with the data, why trust them to randomize it?
Duh, because you don't have to -- you have software (which can be open source provided and verified) randomize the data according to the scheme ON YOUR OWN MACHINE.
While randomizing data isn't entirely new trick for ensuring privacy, IF ANYONE HERE would have bothered reading their paper they would realize that this work takes it quite a bit further than before in allowing reconstruction of complex classification models rather than simple aggregate fucntion computations.
Slashdot Mirror
A naive calculation of the state space size has little to do with computational difficulty.
Yup. This is why introducing uncertainty in order to conceal what is being shared should be part of the basic protocol.
Great post...
There *are* many advantages to using HTTP/webserver model for filesharing (and not just for *copyrighted* filesharing, BTW -- filesharing isn't just for piracy!).
But systems such as YouServ are blurring the line by merging the best of P2P and webserving models.
Try again. IRIS hasn't proposed a thing that can solve DNS security issues. It might address decentralizing the mostly hierarchical lookup procedure (to address scalability for example), but this would in fact require something like DNSSEC so that DNS records could be verified as legitimate even when provided by untrusted/unauthoritative hosts in a DHT.
Wouldn't working on a improved form of SMTP be a better project? No. DNS security impacts much more than just e-mail. Note for example that few websites use HTTPS, so DNSSEC would go a long way towards improving web security as well.
Apart from the university setting, who else is out there?
Not many others. IBM Research is still going strong, but it's generally more focused on shorter term research goals than PARC/Bell Labs. But that's probably why it hasn't gone the way of PARC & Bell Labs...
The computation performed by Seti@Home is what Grid researchers refer to as an "embarassingly parallel".
Among many other things, Grid folks hope to solve problems that aren't quite so amenable to divide-and-conquer. But then they had to go base their protocols on the bloated Web services stack, implying a relatively high granularity per compute unit. So we'll see how well that works out!
Huh? There is a polynomial time algorithm for determining primality (if that's what you mean by "factorise primes"). But this can only *help* cryptographic algorithm relying on hardness of factoring. It renders nothing obsolete because there is no known poly-time algorithm for factorizing numbers in general.
... be aware the IE limits max connections to a single server to 2. So if you open up two big concurrent downloads, IE won't let you access anything else on the site until one of the downloads completes. It doesn't give a warning or anything, it just hangs there waiting for the download to terminate.... A solution is to use a different domain name for your large files to prevent the browser from failing to load your "regular" pages. Another issue with IE is that it seems to ignore the content-length header. Thus if the connection closes early for whatever reason, IE will treat the download as "complete" even if the length doesn't match content-length. This has caused me more than one corrupt download in the past.
HTTP is as reliable and fast as FTP, but shitty browsers (IE) can still cause problems. This isn't a fundamental limitation of HTTP though.
Having actually seen the SAINT-2003 paper on which I'm assuming this article is based, the approach is indeed related to the standard "aquire all locks in a predetermined order" strategy. However it's not exactly that. If I remember correctly :-), it's a variant of this strategy that allows a bit more flexibility in aquiring locks (and hence more parallelism) in certain circumstances. These circumstances, from what I recall, are when a service can aquire "any one" of a group of resources in order to get its task done (which is perhaps a reasonable assumption if we're to believe the web services hype of multiple providers, yada yada). In the more constrained case where a service needs to aquire a fixed set of specific resources, it degenerates into the simple order-based deadlock/livelock prevention scheme.
Now from the article he claims to have further refined the technique, so by now there could be more to it. But I believe this is still likely to be the general idea.
Revolutionary? I don't think so, but it's really too early to tell. Indeed as another author posted, lock aquisition has not yet
proven to be a bottleneck in web services, but that could be because the vision of having multiple providers offering multiple (possibly equivalent) services has yet to materialize. Complex web services orchestration is something that is more hype than reality at this point, but that could change.
Downgrade to http1.0 & disabling keepalives is a valid workaround, but it will also make any MS browser visiting the site (when using SSL) perform abysmally. Ever tried to load a page with multiple (small) images from a site without keepalive support? Yikes... I'm really surprised this would be the standard configuration to work around this IE bug, as I think you can more simply work around it by disabling ssl3.0 for MSIE. The fix I use, which probably has the least performance problems (for the client at least), is a 65 second or greater keep-alive timeout, but this does make your webserver a bit more prone to some DOS attacks and also increases its load. Not a problem if you're not a high traffic site though.
..of Microsoft browser networking bugs which make it only work well with IIS. For example, This bug causes IE to fail to properly shutdown SSL connections. IE browsers using SSL conenctions with standard Apache webserver configurations will have all kinds of errors due to this issue. You need to either disable keepalives or increase the keepalive timeout to something outrageous like 2 minutes. This "bug" has been around for ages yet despite IE being in version 6, it is yet to be fixed. My guess is this is actually some kind of "feature" that makes IE work faster with IIS (since the connection never closes, subsequent reqests go faster, assuming the webserver knows how to speak the broken protocol).
I have the 384kbps symmetric DSL service from Speakeasy (which uses Covad's lines). I get just under 40k/s download speeds, but unfortunately my upstream speeds max out at more like 29k/s. Other than this slow upstream issue it's been a fine service. But I must admit since I run a webserver from my machine, this slow upstream speed is making me look at alternatives....
It will let you encode any YouServ URL so that it gets cached and served by an army of available machines instead of just your own machine.
Yes, this means it does exhibit a single point of failure, but as the system is not intended for piracy, porn, evading censorship, or other uses that tend to get systems "shut down", the benefits of centrally coordinating the system outweigh the functionality and performance limitations of a fully decentralized approach.
The difference from this and a completely server-based solution is that there are no central requirements that involve a large amount of resources. The central servers are a pair of old 400MHZ PII boxes, one for DNS, and one for the coordinating services. A one man operation can easily run this system on cheap hardware and a pipe with only moderate bandwidth, and still serve tens if not hundreds of thousands of users.
To be completely precise, YouServ is more of a Napster-like "brokered" form of p2p, as opposed to a purely decentralized architecture like Gnutella. For example, to be completely web compatible, it relies on DNS for peer location, and DNS requires a dedicated server (for a particular namepspace). But almost all the "real work" is still done by the peers.
I'll leave the reasons behind this feature omission to your imagination. :-)
The system is intended for personal web hosting, that is, for use by mostly non-technical users for sharing files, creating web logs, guest books, and so on, using their own hardware (and that of their friends). In no way are we trying to provide p2p solutions to business class functions such as serving an online store (though I agree it would be cool if someone did that).
Indeed the existing plugins are simple (my development time has been limited and this component is very new), and at this point are intended only to demonstrate the API features. But even quite simple plugins, if they cooperate across multiple machines, can do some very cool things, a few examples of which are proposed in the paper: sharing files on multiple other p2p networks, distributed content caching ("akamai for free"), cooperative weblogging, and so on. Again, you woudln't use this to build a high-traffic online store, but it does give you many new and easy to use ways to enhance and publish your own (web) content.
Sure you could probably build something like this with JXTA, but nobody has done it yet.
A worse one (for YouServ compatability and anything else reyling on dynamic DNS) is the fact that Mozilla caches IP addresses until the browser is completely restarted. How's that for stupid?
Please vote to have this issue fixed right here.
YouServ is a bit different than traditional P2P apps most people are familiar with. For one, there is accountability: Every one of the files you share has YOUR NAME embedded within the URL required for accessing that file, even if that file resides on another machine at the time of download (e.g. from the ability to replicate your site to other machines). Unless you don't value your job, you'd be highly unlikely to use this system for sharing porn / MP3's / etc, a point made in the paper on the search function.
Two, it's not just for sharing files that are world readable. It's also for sharing stuff with only designated users and groups. Every node is tied together by a single sign-on infrastructure so you can use one and only one password to access secured content on any node, without the possibility of malicious nodes sniffing and stashing your password.
Third, it's at its core web compatible. You access the network with a regular web browser. There s no need to install spyware riddled clients to get at anything. Each node provides a web accessible search interface for searching the globally shared content or site-specific content. You only install the software if you want to host stuff.
With a monopolistic market share, unless MS upgrades its browser JVM (which everyone must code to), then nobody else need bother. MS knew that upgrading its JVM might actually make Java useful, and give some kind of "advantage" to Sun and other Java proponents. This sort of anti-competitive behavior is the problem. It's quite a bit different than Microsoft refusing to put some niche-geek language like Mesa into its browser.
Applets are plenty fast and reliable with Sun's recent Java plugins. The web would be a much better place if Microsoft hadn't stifled Java's adoption on the client side.
In more limited application domains this wouldn't be as big a problem.
Maybe it's a pipe dream, but it's still an interesting idea.
>actually understand and trust the system.
I don't know... people use HTTPS/SSL all the time. You really think more than .01% of the population understand the subtleties of certificate authorities, public / private key encryption, etc.? Yet still it's far from useless.
As long as people trust that it works, there's no need to actually understand it. I think this trust would be best provided by open source implementations of schemes such as this.
> them with the data, why trust them to randomize it?
Duh, because you don't have to -- you have software (which can be open source provided and verified) randomize the data according to the scheme ON YOUR OWN MACHINE.
While randomizing data isn't entirely new trick for ensuring privacy, IF ANYONE HERE would have bothered reading their paper they would realize that this work takes it quite a bit further than before in allowing reconstruction of complex classification models rather than simple aggregate fucntion computations.