Slashdot Mirror
Making Freenet Find Stuff Faster
Sanity writes "Many probably saw the recent announcement of Freenet 0.5.2. This release represented a vast amount of work - primarily in reducing Freenet's CPU and memory requirements. However, streamlining Freenet's current functionality isn't all we've been working on. I just finished an article that describes the most fundamental improvement to Freenet's core algorithm since its original design over three years ago, it is called "Next Generation Routing" and has the potential to dramatically increase the speed with which Freenet retrieves information. It could even make Freenet faster than the World Wide Web in many circumstances, all without compromizing anonymity and while remaining immune to the /. effect."
Freenet is an awesome idea, and very rapidly becoming one that is neccesary to ensure your protection. Although it is a double edged sword (It can help both good, and bad people), I think it's one that is neccesary. And, if it becomes speedier than the web at large, it'd be just freaking awesome. Now, no one needs to fear censorship, nor do they need to fear the government shoving them into a database.
Now if only I could get it running on my Mac OS X box...
Other than the fact that most infringers do not like to use Freenet because it is too clunky for them to get their quick hit of free music, it is no more of a threat than any of the popular P2P services.
Translation: "Oh Lord, I hope Freenet is inherently unable to have robust search functions, because if it ever develops these, we're hosed. But in the meantime, we can dismiss this software as being a big POS."
Now, less than two weeks after the interview, it seems the one aspect of Freenet that Oppenheim wanted to write off at is on the brink of being fixed.
I'm generally "Interesting," "Insightful," and even "Funny" here. What the hell happens to me at parties?
What I find interesting about this algorithm is that it is applied individually by each node; there seems to be no need for nodes to share data over some complicated protocol as in many distributed systems. Yet (I think we can believe Clarke) this change improves response time through the system as a whole. It's a validation of the basic Freenet model of systems acting alone but providing a service greater than the sum of its parts.
has anyone ever tried peekabooty, esp. under wine? The reflections on open source development the developer(s) feature on their website sound kinda depressed..but then again, the honesty factor speaks for them. Are there any deep flaws in the idea? I personally like the simplicity of their design, but since I'm not a design guru, I may be utterly wrong.
Makes you wonder if Freenet gained popularity over the web whether all "official" transactions would be web-based, leaving Freenet to misc. web sites that are completely information/communication based. The reason I wonder is because if someone gets their login/password stolen from some random service on Freenet which they invested mucho time in, how will anyone else know the difference? That would really irk me.. (Yes, I know the web is vulnerable to this as well, but at least it requires a user have an IP address -- whether or not it's actually legit.)
They thought it would be cool to design a censorproof network. They weren't interested in supporting what was already in development, namely Freenet, after all - where is the publicity in being part of someone else's project?
The only problem was that they dramatically underestimated the difficulty of pulling it off - the result? Peekabooty was, is, and probably always will be, vaporware. The design they do have is a primitive HTTP proxy network last time I looked, and it doesn't solve any of the difficult problems of circumventing censorship (just ask them how the poor little Chinese dissidents are supposed to find their HTTP proxies).
Amuzing, after draining the concept of a censor-proof network for all it was worth (without actually building one) - they then did their best to drain publicity from their failure to build it!
Freenet answers those questions, and has done so since its original design in 1999.
the nice thing about the current ng routing scheme is that there's plenty of room for research on how to tune it even further.
/me wanders if embedding fortran in java makes sense ;))
Note: if you haven't read the article, this won't make much sense to you.
For one, the number of reference points doesn't have to be fixed; if/when memory and cpu power allows us, we could have variable number of reference points per node. This opens the door to other decisions, such as whether we encourage clustering reference points. If yes, we add new ref points closer to others. If not, we remove a ref point the density within some keyspace interval gets too big. Another option is to add a new ref point whenever the n previous estimates turn out to be more than x% correct, and remove one if otherwise.
Another direction to go into is curve fitting. If cpu power allows us, we could use various techniques of polynomial or Fourrier interpolation within the existing reference points to draw more accurate curve of time vs. keyspace.
Don't go silently into that peaceful night
Freenet is NOT immune to the /. effect currently. Every time /. runs a Freenet related story, loads of new users seem to get on the Freenet and it just collapses, meaning response times go way up and many freesites become unreachable. I'm sure NGRouting will take care of this, but it's not honest to say it will help Freenet "remain immune" to the /. effect, because it's not immune.
"Oppression and harassment is a small price to pay to live in the land of the free." -- Montgomery Burns.
No, you are referring to the startup time of the VM. Once started, the memory pages will have settled and the response gets better and better. The same thing happens when you use the menu's the first time. Once the classes have loaded, the program has fine responsiveness. Actually, you can preload classes with Java, but not too many developers use that particular feature (it will add to the startup time anyway).
IMHO, the Java VM should be loaded at startup, and a single VM should be used to launch multiple applications. When used like that (together with an efficent GUI startup process) much of the gripe against Java applications should be gone. Obviously the firewalling between programs should be maintained. Alas, this is not currently so.
To come back to freenet: it doesn't incorporate a GUI in normal use (using the web interface is not the same as launching a Swing application) and for networking speed: the speed of the connection will be the bottle neck, not the Java application.
It must be said that the current implementation will scare away programmers that are looking towards efficency. For most programs you should n't though. Look at the architecture before trying to get something more efficient by changing languages.
ps. for an ample showcase of efficiency, try Eclipse from IBM. Check the features first before posting though.
Possibly, yes. But any country in which the courts rule forwarding requests to illegal content to be illegal will also have made public HTTP proxies (like anonymizer.com) illegal. At that point a different system will be necessary, something where it would be impossible to distinguish publisher/retriever from 'innocent' bystander. There are possibilities for such systems, but they're going to be even less efficient/fast/simple, so let's hope they aren't necessary.
The text in the FAQ is mine, I don't think Ian is claiming freenet will get better throughput/latency for browsing random websites, however freenet can be faster for downloads from websites, websites with flash animations or big applets, those kinds of things. (At current the anonymity filter (a piece of code that filters potentially anonymity-compromising parts from freenet websites) will remove all plugins and applets, but we're talking middle to long-term future here).
:-))
Basicly, freenet latency is bad, freenet throughput is good. (and freenet reliability is different
You're part of the problem! The reason Freenet sucks for a little while after each release is that there's a huge influx of empty datastores joining the network. The network bounces back pretty quickly, as data gets passed around and as routing tables hone themselves, the network gets a lot better.
Then a day or two later, you and 90% of the other slashdotters drop off, and leave holes in everyone's routing tables. All the contribution that your nodes were just starting to make, gets undone. All the copies of content that got replicated into your datastores vanish. All the routing optimizations that were just sorting themselves out get broken again.
Tourists hurt the network. If you're judging Freenet based on it's performance the day after a slashdotting, you're not getting a full or fair picture. Come back and stay a while! Let your node run for a week and I think you'll be impressed.
When they say Freenet is slashdot-resistant, they refer to content within the network. Any piece of data, be it a single file or a whole freesite, will simply propagate more as more people request it. The network itself definitely labors a bit as empty datastores dillute it. The best way to improve Freenet's performance is to encourage those tourists to stick around, so they and the network will benefit the most.