Slashdot Mirror

I wait until they include YaCy

Being a snitch for the state is worse. The list of forbidden words can only grow.

The fact is, you can't depend on any single company subject to any jurisdiction. As long as information is treated as contraband, we need to compensate.

If it exists on the net, they should find it (except for criminally illegal content like child pr0n).

So which is it then? Either they censor, or they don't.

To get around this problem, Yacy could use a hell of a lot more users...

but to where?

Yacy. If more people use it, it can only get better. And it's very resistant to censorship.

Great idea, I used it a few weeks back but the results were horrible. Yes, it would be fantastic if more people used it.

but to where?

Yacy. If more people use it, it can only get better. And it's very resistant to censorship.

YO! I got your torrent to find your torrents to torrent your torrents! You won't avoid takedowns any other way!

We need a flagship open search engine

You already have one, and the censors will have a terrible time trying to meddle with it, but it needs lots of users to be effective.

bla bla bla it doesn't store your personal information...

Yeah, I gotta bridge for ya

The moment they become statistically noticeable everything changes. Your best hope for (kinda) private unfiltered searches is Yacy

Can it block YaCy? I mean, c'mon! We have to get around this shit somehow!

Can it block YaCy? I mean, c'mon! We have to get around this shit somehow!

I think that YaCy might make you very happy: http://yacy.net/

Yes, they could hurt domestic business, but Europe doesn't have that kind of clout outside its borders. I mean, yeah, Google will cave, but that's only because they have a physical presence they don't want to give up. Doesn't really matter, we won't get unfiltered searches without a distributed search engine anyway. There's no other way around arbitrary and capricious law written by feudal bureaucrats.

First of all, exactly.

Second, we need to take a step back and consider how we got in this situation in the first place. Consider all the shit wrong here:

• If this sort of unauthorized access to a computer system is a felony -- and under the CFAA, it is -- then we shouldn't be laughing it off in this case just because a corporation did it; the FBI should be raiding BK's headquarters and arresting executives.
• This sort of thing shouldn't be a felony in the first place; the CFAA is a bad law.
• Google shouldn't be allowed to abuse its near-monopoly by censoring, tampering with, or editorializing search results.
• Google shouldn't be able to censor the results because we should all be using something we control ourselves (like YaCy) instead of handing so much control of the Internet over to Google in the first place.
• Devices with voice recognition should process it locally, not send everyone's private utterances to third-parties (which are inherently untrustworthy).
• People should not want to infect their homes with always-listening surveillance devices anyway!

In theory, you could use YaCy and adjust the algorithm yourself. Self-hosting my search is still on my "to-do" list (not my "done" list), though.

Yacy is making an effort... decentralized P2P is the way. It should be more difficult to shut down, but it could be easier to pollute. After that, your ISP will be the next obstacle to overcome.

Is YaCy worth using yet?

I agree about the communications. I have an extreme dislike for centralized services such as google, facebook and most others simply because they can do whatever the hell they want and may or may not tell you how they are manipulating content and or speech...

IMO
The world needs most popular peer to peer search and social media service that is maintained by random peers and or social media by your network of 'friends' perhaps..

http://www.yacy.net/en/
Don't know of a social media one...

That's a neat idea! However, I'm a little dismayed that you think users need to be "allowed" to do this or that there "should be" a right to do it, instead of realizing that users don't need anybody's permission and that right already exists and always has existed.

All we need is easy-to-use software to implement it. I think building that kind of functionality into things like Web of Trust and YaCy (if they don't implement it already) is a good place to start.

I would hope this would provide a little incentive to find alternatives to Google. The Yacy project seems like a good start. Or maybe it's too kludgey like Freenet. Either way, decentralization and ad hoc networks are our only hope for an open and secure internet.

What's the end-user alternative?

What we really need is to make a concerted effort towards replacing all these centralized web services with distributed equivalents:

• Google Search -> YaCy
• Gmail, Google Drive, etc. -> OwnCloud
• Google Maps -> OpenStreetMap
• Hangouts -> XMPP
• Youtube -> ???
• Facebook -> Diaspora
• Etc...

What we need is a concerted effort to develop Free Software distributed/federated/p2p alternatives to all the major centralized services. Not just social networks, but even things like search.

Sorry, you can only prosecute for action, never speech. The actor is the sole guilty party, no matter what he claims as motivation, whether it's tabloid hearsay or anything else. Censorship still remains the prime evil. Its only purpose is expediency and for the protection of powerful people and institutions. We need to kill off the whole discussion by making censorship impossible with decentralized, P2P type services. That would be the end of the problem and we can move on.

But you sure as hell don't have any right to prohibit it from coming up on Google. I'll filter my own searches thank you.

Personally, I'm hoping for more widespread circumvention of the whole issue.

Is it time to join the YaCy crowd yet?

http://yacy.net/

That's right. Create an alternative. As for Google, well fuck them too!

I was looking into the YaCY P2P search engine ( http://yacy.net/âZ ), but I have not given it a go yet. Have you tried that one? What did you find about it that was bad?

A post in a thread a few days ago gave a good list. (I'd link back to it, but I can't find it.)

• RequestPolicy
• NoScript
• RefControl
• Ghostery
• HTTPS-Everywhere
• BetterPrivacy
• Cookie Monster

I didn't list Lightbeam because while it is good at visualizing tracking, it doesn't actually stop it.

I also currently use

• AdBlock Plus
• Self-Destructing Cookies
• DuckDuckGo search provider

I'm also looking into running a YaCy server so that I don't depend on centralized (and therefore inherently trackable, even if some say they don't) search engines at all.

We (as in, we users of the Internet) should not be so reliant on a single entity's web services, just as we (as computer users) are not reliant on a single entity's OS. Guess what, you can participate in a decentralised web search engine right away, with project YaCy, by running a node on your computer(s). There are very few nodes at the moment given the potential, and the search will only get better as more people join.

I don't think that with such censorship tightening around the search engines we can stand still and watch. It is not just child porn, there is strong censorship around movies and other "protected" material. When they start censoring "terrorism" and after that censoring legitimate political opposition in the name of anti-terrorism, it will be too late.
I think, that Google and other filtered search engines should be replaced by peer to peer search engines, like yacy. However, such search engines are not usable yet. For example yacy is written in java so it is slow and resource consuming. I'm looking forward to see some good solution that is actually working - I would deploy it on my server immediately.

Google can end-to-end encrypt Drive and Docs, they can't do that with email, social networking they host, and searches. For email, anything you send or receive goes out in plain text unless you and the recipient use PGP, and they need to read your mail to do spam filtering. For social networking, as long as they host it they need to hold the encryption and decryption keys for when you share with other users. And their search technology works off of plain text communication, I believe (and could be wildly wrong) that it's possible to have an encrypted search engine with encrypted keyword search, but I believe in that case the end user has to encrypt the data before uploading it to the hosting provider. Since Google collects the data to encrypt, not the end users, that won't work.

I want the internet to work the way you're describing, but I don't see it happening soon. Here's hoping that http://yacy.net/en/ (distributed search engine) and http://secushare.org/ (what looks to me like the best hope for true distributed social networking, but it's in its infancy) take off.

What implications (if any) does this have for Yacy?

http://yacy.net/en/ (the distributed search engine)

What do you think of decentralized search engines like YaCy? Maybe you have ideas of their potential effects on privacy, information findability, and the natural environment?

I'd like to point out that all search engines are ad supported

Most, but not all. See http://yacy.net/

http://yacy.net/en/index.html

Already exists http://yacy.net/en/

You know, this looks very interesting actually. I wouldn't be so bold as to say that it's "taking on Google", but it's a great idea. For example, right now it looks like yacy.net is down (maybe because of us, btw) - but you can just install it on your own Linux-based machine and you can still search the network, heck there's even an apt repository.

From there you get your own search engine, which you can even use to only search your LAN or private network. You can not only use that to search the distributed database, but also crawl your own sites, to improve the results for your community.

Think about it: no more profiling of your searches, no more dependency on a central authority for results. More importantly, no more single point of failure for the collective knowledge of mankind. Seems to me like a good goal.

That is a pretty good start, I would say. Of course we shouldn't expect the results to be better or even close to Google's, but if we all jump in and help this project, this could become something decent.

Or we could just sit back in our usual armchair specialist posture and say it's all crap. This is slashdot after all...

It has already learned from my search.

The whole "portal only as an afterthought demo" seems to me a huge flaw as well. You think your average person is going to install this on their computer just so they can do web searches? Not-going-to-happen. People who want to run it, will. People who don't or don't know how, won't. They're the 99.99%. They need a portal. Clients should automatically be putting themselves in the portal-switching queue.

As for the capabilities, I just tried it out. The results are *extremely* few and very poor. "Dog" gets five hits, for example. You'd almost think it was a joke. Hopefully this was a load problem or a problem due to a lack of scaling in the system thusfar, and not a design flaw.

At least their frontend doesn't seem designed with injection in mind. Start off a search with ' (such as 'Test) and watch what happens to the peer listed at the bottom of the page. I doubt that particular issue is exploitable, but if this a habit of one of their coders...

From TFA:

It is fully decentralized, all users of the search engine network are equal, the network does not store user search requests and it is not possible for anyone to censor the content of the shared index.

However, that seems to be all the information there is on the process... doesn't quite assuage the ol' paranoia circuits, does it?

The network stores everything.

Will one client be able to view the queries of its peers?

If yes, how is that an improvement? If no, how does it work?

From TFA:

It is fully decentralized, all users of the search engine network are equal, the network does not store user search requests and it is not possible for anyone to censor the content of the shared index.

However, that seems to be all the information there is on the process... doesn't quite assuage the ol' paranoia circuits, does it?

A commenter further up pointed out http://yacy.net/

YaCy

We need to have a project that aims to unite all the privacy projects out there to make something good come out of it, using the power of the crowd with free software in a privacy respecting matter but in a much more powerful way that can actually serve people...

Here are some projects or ideas that deserves to be noticed:

An openID with privacy features:
http://openprivacy.org/

P2P social networks / research:
http://www.movim.eu/
http://www.peerson.net/

P2P search:
http://yacy.net/

P2P SIP:
http://www.blyon.com/blog/index.php/2009/06/22/p2p-sip-uri-dialing/

Encryption:
http://code.google.com/p/cryptsetup/

P2P encrypted networks:
http://www.i2p2.de/
http://freenetproject.org/

Augmented reality / group mapping:
http://www.openillusionist.org.uk/documentation/doku.php?id=site:home
http://www.biomapping.net/

Mesh:
http://robin-mesh.wik.is/

I envision a setup where our cell phones or little home servers (open ones, like the n900 or better) can connect to each other via mesh, have open social infrastrcture running on them routed over an I2P layer so nobody knows who is talking to who and you have total control as to who/when/what is seen by your peers.

These setup have cameras that can use such network to create massive collaborative networks to document a situation or location. Be it a manifestation where you relay real time camera from all angles with sound level maps and other sensors to augmented reality group interaction and other crazy ideas.

This is more broad that what is discussed here as it touches all OSI layers and ask for a shift toward a p2p infrastructure at all level respecting and working for the user and independance from middle man as much as we can.
Of course a distributed DNS might have to be worked on too. I think these research are fundamental to the survival of freedom online as we knew it ...

You can have a look at YaCy

How does this compare to http://www.yacy.net/yacy/ ?

Let the user become the crawler- and do not eliminate the search giants (just don't rely on them completely). Already I sort of operate like a (slow) crawler with my queues of links to read, bookmarks (be weary- big load) and indexing those very interesting or important pages, sharing related tidbits, etc. Just feels like the natural extension, though I am sure that many people will want to stick with traditional GUIs and "back/forward" habits. There is also some interesting discussion in ATLAS-L re: future search infrastructures. So, in the spirit of promoting development in this area, linkage:

* Grub article (now defunct)- was distributed peer-to-peer crawler. (see also)
* Boitho, another distributed crawler
* YaCy- another peer-to-peer crawler
* How to build a web spider
* C++ web crawler lib
* LibWWW (perl)
* W3C's WebBot
* The Internet Archive's Heritrix crawler
* WebSPHINX- customizable crawler

Somehow, this is like an extension of surfraw. I imagine that soon enough we will start up an open source crawler-browsing hybrid software package, though have been surprised that nothing like it has popped up yet- it's (usually) the way of the programmer to make sure that he has the ability to do what the giants are doing. Maybe we have all been collectively blinded by graphical web browsers (IE, Firefox, Opera, etc.) and "click-click-click" thinkware?

ah, yes, the thing i am planning is similar to what you describe, in that it involves many ideas that anonymizing networks like tor or freenet only implement a few of. it's called banana (there was a project on sourceforge i and a friend started with the same name, but it sat dormant for almost a year, and i've started from scratch in my free time about a month ago, and _lots_ to do, and i'm basically abandoning the piece of crap i left on sf.)

however, i disagree on some points you had; personally, i think it should be entirely pseudonymous -- in other words, there would be no 'logging in', unless one is logging into a server reached over the network (which, incidentally, would be anonymous).

other things that would be cool: clouds, using kademlia (dht) with 512 bits to find nodes adjacent to a server followed by distance-vector routing to actually get to it, lots of hash verification... lots of ideas here.

one thing about it though, is that keeping it decentralized is important; no one place to shut it down. they system, therefore, can't reliably do a karma system on the quality of 'uploads'; this would have to be decentralized. quite frankly, i've got enough on my inchoate coding plate to do before attempting karma. karma can be done by servers on the overlay network. similar issue with a search engine. let things be indexed by hashes, not human-readable names. if it's human readable, then it's not as easy to do and isn't indexable in the same way.

the purpose of my overlay network would be to supply all the usefulness of a (second) internet (tcp-like connections) as well as freenet / bittorrent swarms. freenet is a big dht, which is fine for small files but not so good for larger ones. i think that onion routing isn't useful in banana, but similar ideas there.

so, take 2,3,4,5,6,10 out (and the 'log in' part of 7), and you're close to my idea, except that i add in the useful feature of tor's hidden services; let people run servers that are accessible over the overlay network which are literally just like having a host on the usual internet.

basically, i think your desires are too p2p oriented; you want something like napster, while i want something like the internet in general + arbitrarily hosted data. someone could build a napster on top of banana, but that's too focused. incidentally, if you want to learn more about distributed search engines, you should check out yacy:

http://www.yacy.net/yacy/

I'll emphasize a few points:

-Google does have a quasi-monopoly, in the sense that a lot of (most?) other search engines contract with Google for search results, and none have come to the same quality as Google, hence if you want an accurate search result you "google" for it

-I'm arguing that the employment market rewards cheating (I think most management text books would agree, but that is another subject). You can't really cheat with a successful business model (although you can steal people's ideas)

-and I said "ideally"... ideals are hard to come by and not always practical. In essence Google is not being net-neutral by what it is doing (discriminating against certain types of businesses). It is an academic point. I could always run my own search spider or use http://www.yacy.net/yacy/Download.html if I wanted more neutrality... but those options of course require more work

Your points are well taken. I myself will continue to use Google.

There's at least one prototype open-source search engine. It's called YaCy (pronounced like "ya see"), and it works without a centralized server. They call it a p2p web search. Honestly I don't see how it could function without any central server at all, but I don't know much about the nuts and bolts of p2p.

There's a lot to like about this kind of search (transparency, uncensorability) but queries are said to take much longer than they do at google. I can't give a personal account; I tried to install it but I couldn't get it to work.