Slashdot Mirror
SQL Over FreeNet
Nym writes: "It seems that another component, SQL has been added to the many layers of Freenet. Check out FreeSQL Project, and start porting your SQL apps today." The possible implications of this sort of utility are amazing.
← Back to Stories (view on slashdot.org)
Yeah, and also imagine getting a Buzzword and another Buzzword there... Now that free-speech-concerned people are interested about Freenet, with things like these we could get politicians and The Management interested about Freenet!
"Yes, you can get Freenet with a red and green browser - and blue, if you pay a bit extra!" =)
Does this mean you would just start over any time you wanted to change your db schema?
Is the content of freenet read only?
Data are inserted into Freenet using CHK (content hash keys). The basic unit of data is a file (of any size); when you insert a file, its hash value is computed, and that is used as an index for retrieval, as well as for encryption. Other forms of Freenet keys redirect to the CHK.
If you insert file F1 into Freenet, with CHK K1, it's there to stay. You could have a human-readable KSK that points to it (e.g., freenet:KSK@/file/F1) so that it's easier to retrieve. But there's no way to delete it.
If you modify file F1 to create file F2, and then insert it into Freenet, you end up with a new CHK K2, which is the hash of F2. But if you try to insert F2 using your old KSK (freenet:KSK@/file/F1), it won't work -- you'll get a keyspace collision.
So in short, there's no simple way to update information that's been posted to Freenet. It will eventually die out if nobody ever reads it; but in accordance with Murphy's Law, if you made an error, it will haunt you forever.
(This is not to say that it's impossible to have updated content on Freenet! It's just that you have to go through some fairly involved convolutions to do so. The most common technique in use today, AFAIK, is a date-based redirect (DBR). Essentially, there's some black magic in the Freenet client that lets a key of the form freenet:MSK@SSK@... redirect to a different key every day, or every week, or every hour, or however often you choose. Then you set up a crontab job to publish a new index of your data periodically, just before the deadline.)
How this affects FreeSQL, I couldn't say.
"Of course the real bottleneck is the P2P aspect of the application itself."
The fact that freenet runs on Java is completely irrelevant from the point of view of network performance since the bottleneck is not execution speed or memory usage but network throughput and IO. Implementing it in for instance C would have no other effect than seriously slowing development (assuming you use the same software architecture).
Applications like limewire (IMHO one of the best gnutella clients available) prove that Java is a very good platform for developing p2p software. My limewire gnutella client has no trouble processing the trafic generated by 13000 hosts and there's plenty of processor time left for other apps. And if you are wondering: I'm on a 450 Mhz pII with 128 Mb of ram running win2k.
I've tried out freenet as well. Performance currently sucks but that's not a Java issue but an issue of the freenet architecture in combination with the problem of lots of transient nodes and limited available content.
Jilles
I hear too many assumptions being made. Yes this is a SQL interface. That does not imply that there is a full featured database behind it. It only means that you can use this popular language to retrieve data.
So what if FreeNet does not follow all tenents of a mature Relational Database model. Just because there is a SQL interface doesn't mean you have to run a business on it.
I really hate that. How hard is it to put a blurb of what you are doing on your sourceforge homepage? That's what it is there for!
How we know is more important than what we know.
FreeSQL adds no new functionality to Freenet itself. All search indexes and data are stored anonymously, distributedly, and securely on Freenet as always. However, once this data is assembled on your local node, you can run queries against it and use that to find yet more indexes and data to request, and then run queries against that, etc. Your privacy is not risked in any way.
However, one of the prime uses (we think) for FreeSQL will be to convert existing web-based apps to Freenet. In this case, the data will be inserted into freenet as a result of your accessing one or more web pages, so just like Slashdot can invade your privacy, so could other sites.
However, the advantage of this approach is that, because there could be many Slash-like sites using the same freenet "database" there is no possibility for a central site to censor any comments/stories/posts. And once your post gets onto Freenet, so long as your privacy wasn't violated while you were posting, your privacy cannot be violated further.
Can your IM do this?
I've been listening to naysayers until now. From this point forward, I'm interested.
-- @rjamestaylor on Ello
That said, it's pretty dumb. It seems to be all client-side joins, for example. If you send SELECT * FROM actors WHERE firstname = "Natalie" AND lastname = "Portman", your client gets back all hits on "Natalie" and "Portman", then pulls out those with both. That's horribly inefficient if any term of the search is likely to return lots of hits.
There's a lot of work ahead before this is useful.
Doesn't this flat out contradict some of the principles of relational database design? Maintaining a *single* version of any data, for instance? With Freenet, you would get duplication and redundency inherently, but who's to tell you which version of whatever table you are using is the correct one when queries start returning duplicate, slightly temporally shifted information?
It seems that you'd be wasting the power of freenet if you tried to build a relational database on top of it. The mathematics and goals involved in each model are almost wholly alien to eachother, unless my understanding is flawed.
This would bet THE ideal thing to port slashdot to. Store the comments on freenet, and leave the metadata like mod points and such at a centrally controlled site. No more being forced to pull a comment due to lawyers babbling, and no more problems with old content that people still want going away, etc. etc.
a thought anyway.
"There's no secret. You just press the accelerator to the floor and keep turning left." -- Bill Vukovich
And in other news, recent developments seen on Sourceforge include SQL over carrier pigeon and SQL over pnuematic tube. Combined with holographic storage, room temperature fusion, and a beowulf cluster of Ximian-driven network degraders, this promises to be the future of computing.
Getting a bit more real, Espra looks to be shaping up as a tasty music-sharing layer for freenet. It's a GUI-licious as a fresh dreamsicle.
UPDATE freenet SET freenet.content = 'natalieportman.jpg';
1,293,190,901,598 rows updated
======================================
======================================
Writers get in shape by pumping irony.
...is the problem I've always had with the internet, albeit reduced. However, it is merely changed.
;), or it got hit hard with lawyer-torpedoes and had to be scuttled. It's gotten to the point where I've simply taken to storing anything I like on my own hard drive, because I've found that I can't trust any enjoyable site to still be there in a month.
.NET - with .NET you'll have the worst of both worlds. I *want* to believe, but I'm too depressed over how the web turned out I guess.
The problem is unstable content. It annoys me when I find some great webcomic or website or FTP server that really makes me sit up and say, "Wow. This is one of the reasons I get online in the first place.", and then the site up and vanishes. Either there wasn't enough funding, or it became unpopular (or rather, with my tastes, it *started* unpopular and became moreso over time
The only things that are permanent on the internet as far as I can see are things supported by well funded institutions, or based on something really popular. I don't have very mainstream tastes, and the sort of content the web is trying to push these days just doesn't get me excited.
Freenet, I at first thought, would be an improvement. The third reason for content vanishment, lawyer attacks, would be removed. However, the longer I look at it the less certain I am of its usefulness. It is designed very carefully as a system to host only the most popular data. I'm beginning to think this is a dreadful weakness.
For one thing, the quality of Freenet users will directly impact the quality of the content, in a way the web can only dream of. While it's annoying that the quality stuff on the web seems drowned out by all the AOLer-Britney-Spears-fan pages, it's at least still THERE. If those were the people using Freenet, however, there would be nothing BUT.
Ah, but you say, surely Freenet will be used by other hackish types like me? Well, maybe, but I don't share the same views and preferences with some demographic majority of hackerdom. I think in the end the popularity system of Freenet will cause enjoyable content to be unfindable for a minority of people, of which I have little doubt I shall be a part. And eventually as Freenet grows the tastes and likes of the majority of its users will come more into line with the mainstream (ie., become more likely to delete content I enjoy).
The other great argument for Freenet is that it is a great forum, say, to speak out on about important issues. Say, a chinese citizen could speak out about his government's crimes. However, what good is that? How will anyone hear about the document before it gets deleted due to lack of requests? How will the voice of the minority ever be heard in a system where only the content approved by the majority is maintained?
I also find it annoying that, just like the web, there is no attempt to determine whether a piece of work is valuable IN AND OF ITSELF, and is such a masterwork that even if you don't personally like it it must be maintained indefinitely for posterity. I personally would rather Freenet have a system whereby one could rank files by their value. I would also propose that a Freenet user would have the one-time ability (if it could be made unabuseable that is), to give up his rating priviledges to mark a file as permanent and never to be deleted.
I see a great amount of history, art, literature, and achievement being transferred to digital storage. How much of it will remain undeleted in 30 years? 50 years? I'm certainly hoping someone can reply and tell me I've got it all wrong and show me that Freenet has a fix for these problems. After all, it's gotta be better than
Freenet, hmm. What was that saying about Democracy, eh? I guess what I'm saying is that Freenet is great, but it still does not deal with the ancient problem of those whose outlooks are in the minority getting screwed.
-Kasreyn
Kasreyn: Cheerfully playing the part of Devil's Advocate to hairtrigger
Freenet is doing some really amazing things along the lines of creating an alternate network of information that evades censorship and promotes personal anonymity and security. What it's not doing is finding ways for that network to be faster, both in terms of execution speed and adoption. Mod me down for posting as a troll, but I'd much rather Freenet run as an optimized executable rather than Java classes. While Java proponents go on and on about how fast Java can be under the right circumstances, the overhead in terms of processor cycles and RAM required to run an app like Freenet on top of a JVM is still pretty staggering. Of course the real bottleneck is the P2P aspect of the application itself. The current key system has no hope of ever being seriously adopted by the mainstream and an upload/propogation of a smallish webpage with a single on Freeweb ran about 35 hours over my cable connection. Are we seriously considering adding SQL calls to this morass? C'mon guys. Work on basic functionality like increased key-search speeds and simplifying the key system for grandma to use before you start adding additional features.
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
I remember when they announced Fproxy. This would allow you to view HTML pages stored in Freenet on any web browser.
Stupid, I thought, but somebody was bound to try it. Now it is very popular and most people who have only heard of Freenet think Freenet is a place to store your porno web pages where his wife won't find them.
By the way. FreeSQL was reported on Snarfoo (a website in Freenet). I'll bet you wouldn't be reading it on Slashdot today if it weren't announced on a Freenet based website three days ago.
Now a lot of people seem to think that because of Fproxy, Freenet is a replacement for the Web. When people complain about speed they must consider it is an encrypted channel. SSL is a bit slower isn't it? You don't know where the documents are coming from, of course unpopular documents may take a minute to find. Once a few people request your documents they spread like wildfire. Cheesey freesites (websites in freenet) get looked at then they fall out of Freenet.
Freenet is not Geocities. It (please) won't ever be Geocities. If you want to have midi files playing in the background while people look at your stained glass collection, keep it on the web where it will sit and rot forever.
People complain about the complex keys that are impossible to remember. There are also simple KSK keys like 'slashdot'. These aren't secure but they serve their purpose. You put a file in Freenet named 'flowbie', tell whoever to get it then you forget about it.
Like someone mentioned in another thread, look at the URL for this post. You aren't going to remember that, you are going to your bookmark, or to a well known link and then to the complex URL. It is the same way in Freenet. You go to your default page then click away to get where you want to go.
Do you use telnet with your first name as the password stored in a plain text file or do you use ssh?
I love the Britney Spears example that keeps popping up. It is perfect. She won't be hot someday soon. Then guess what? She drops out of Freenet. Just what it was designed for. Then the next hottie will propogate.
The guy who wrote FreeSQL did it because he could. Sound familiar? He never asked anyone to port your SQL apps to Freenet. It is for doing simple stuff in Freenet and will help to create more apps that can store and retrieve data with a simple syntax. People will create a Slashdot in Freenet and why not use something like FreeSQL? Slashdot stories aren't accessed often after the first day are they? Authors do have the ability to re-insert the data if they want to if it falls out of Freenet.
No search engines in Freenet? Go try it. There are some great web based ones, though they are bound to get shut down. www.freegle.com is great.
In the last two releases of Freenet they introduced in-Freenet key indexes. While a little clunky, they work. With the recent introduction of Freenet Client Protocol and XML-RPC there will be non-Java clients popping up all over the place. To examine a couple of key indexes:
java Freenet.client.KeyIndexClient -list freegle
java Freenet.client.KeyIndexClient -list snarfoo
These 'start over' at 12:00am GMT so they won't be too lengthy. 3rd party programs are even better:
fngrab.py -x freegle
fngrab.py -x snarfoo
FreeSQL is newsworthy. CmdrTaco doesn't know why though.
Freenet is not permanent storage. It is a non-permantent database - why not have a simple database interface to it?
- Is this even newsworthy? There is no code, not even a homepage, nothing to suggest this isn't vapourware (of which there is no shortage on Sourceforge)
- Freenet isn't very well suited to this task because it forgets unpopular information, which is just what you don't want a database to do
Note that the lack of permenant data is not a flaw in Freenet, in this regard Freenet is more like a radio station than a library, or indeed, a database. Before anyone says "but why not just add permenant data to freenet" know that someone suggests this on the freenet mailing lists about once every two days - it isn't possible while staying true to Freenet's other goals.--
I love it. Forget all that ACID crap, especially the D part, let's put a database interface on top of a storage system that is designed to drop data practically at random. Brilliant. I can't imagine why nobody thought of this before.
Slashdot - News for Herds. Stuff that Splatters.
Those who tried out FreeNet know that it's slow, and it's expectable.
However, SQL is kind of very load-demanding accessing method. Anyone who could put an excess amount of loading to the network by running an open end SQL with sorting(just an example):
SELECT * FROM FREENET.BIG_TABLE WHERE 1=1 OR 2=2 ORDER BY 1
A similar carelessly written statement of above crashed an enterprise database once because excessive TEMP spaced was requested for open end sorting.
How do they deal with the stability/performance in favour of accessibility?