Scroogle Has Been Blocked

An anonymous reader writes "Scroogle, the secure third-party Google search interface, has been blocked by Google. Scroogle was an SSL-based search proxy that enabled one to search for and receive Google results over an SSL connection in a pseudo-anonymous manner."

Comments

Scroogle

[sopssa]

While I would love to see a good rant towards Google and while I also myself use Scroogle, the summary isn't really being truthful. Google hasn't blocked anything, they just changed the page that Scroogle scrapes and they're throwing a hissy fit about it.

From the Scroogle announcement:

We regret to announce that our Google scraper may have to be permanently retired, thanks to a change at Google.

That interface was at www.google.com/ie but on May 10, 2010 they took it down and inserted a redirect to /toolbar/ie8/sidebar.html. It used to have a search box, and the results it showed were generic during that entire time.

Now that interface is gone. It is not possible to continue Scroogle unless we have a simple interface that is stable. Google's main consumer-oriented interface that they want everyone to use is too complex, and changes too frequently, to make our scraping operation possible.

Google changing something isn't exactly "blocking" a third party service. Even more so, it's just a few lines of code to get the results from main Google search too. All the search results and links have approciate html ID's associated to them and it's been the same for years already.

I have no idea why Scroogle is bitching about this.

Oh well. I changed to use ixquick, which also has the added benefit of being located in the Germany rather than US and a lot better and useful interface.

-sopssa

Re:Scroogle

[longacre]

What's the benefit of being in Germany?

Re:Scroogle

[DarkKnightRadick]

I love Startpage. As a metasearch engine, it's pretty darn good.

Re:Scroogle

[Registered Coward v2]

What's the benefit of being in Germany?

Maybe:

Ve hav vays of making you benefit

or,

Ve no nutting, nutting,

Re:Scroogle

[sopssa]

From the FAQ:

European Privacy Seal
On July 14th 2008 Ixquick received the first European Privacy Seal from European Data Protection Supervisor Mr. Peter Hustinx. The Seal officially confirms the privacy promises we make to our users. It makes Ixquick the first and only EU-approved search engine. Both EU Commissioner Viviane Reding and Dr.Thilo Weichert, German Privacy Commissioner complemented Ixquick on its privacy achievements.
You can find the press release here.

Since I am in EU, it also means US can't just randomly get data that doesn't belong to them, ie. for people from other countries. Frankly, EU and European countries take privacy a lot more seriously, for historical reasons too.

Re:Scroogle

[Anonymous Coward]

real beer, excellent food, beautiful landscape

Re:Scroogle

[Jer]

What's more, the link they were scraping off of [www.google.com/ie] seems to be related to Google's support of Internet Explorer. Since it's been replaced with a "go get IE 8" page, it's probably been dumped to encourage people to dump their older versions of IE and get something newer.

Re:Scroogle

[Anonymous Coward]

It is not located in the US and therefore would require more hoops for the US DoJ to get information from.

Re:Scroogle

[cmiller173]

You sir are absolutely correct. Anyone who writes an application based on screen scraping should expect changes to happen and not act surprised when they do. Besides doesn't Google have a freaking search API? http://code.google.com/apis/ajaxsearch/

Re:Scroogle

[bsDaemon]

and less hoops for INTERPOL, German Federal police, etc. Also, there is then no protection, even in theory, from NSA or CIA operations as that becomes "over-seas" and thus fertile ground. I guess it all depends on who you're trying to slightly inconvenience, though.

Re:Scroogle

[Anonymous Coward]

Now CmdrTaco can't safely search thai hooker blood removal!

Re:Scroogle

[JustinOpinion]

Indeed. Scroogle sounds like a good idea... but it's a service that exists parasitically to Google proper. I'm not trying to imply anything unethical by using the word "parasite", but this really is a situation where Scroogle uses Google's capabilities/services without contributing anything back to Google. This is fine to the extent that Google tolerates it. But they are under no obligation to make accommodations to keep these third-party services running smoothly. TFA says "It's not as if Google needs the money" which seems rather uncharitable given that Google has put up with Scroogle's operations for many years now without any complaints or blocking attempts (that I'm aware of). And Google does need some money (they would have to shut down if everyone used their services through Scroogle...).

Scroogle needs to either adjust their service to keep up with Google's changes, or make a business case to Google for why it is in their best interest to provide a stable interface/API for third-party redistributors like them. The implication in TFA that they are somehow entitled to this interface/API/access is really silly.

Re:Scroogle

[Anonymous Coward]

Scroogle isn't throwing a hissy fit...the person who made this summary is.

Re:Scroogle

[Anonymous Coward]

A couple fun facts about Scroogle:
1. Since Scroogle hit multiple Google IPs, it used to be possible to search the same keywords 5 or 6 times in a row and see the variation in page rank. Great for web site owners to see how they ranked.
2. Scroogle dot COM is NSFW at all, so when telling people about Scroogle it was usually CRUCIAL to emphasize the dot ORG part of the domain. At a previous job I made the mistake of telling my boss about it without emphasizing the dot ORG part and, well... he got an eyeful of the wrong type of "org"...

Re:Scroogle

[Jer]

The linked article does use the title "Scroogle has been blocked" when, really, they haven't been blocked at all. They're free to change their code to work with the various other methods of accessing Google - like perhaps using the publicly available API that Google provides. Since I've never used the API I'm not sure exactly what technical limitations it imposes that make screen scraping a better alternative to the API for privacy concerns. Anyone have an idea why they would need to use a screen scraper to anonymize connections instead of using the API?

Re:Scroogle

[ravenspear]

Well, if you are a US citizen, adding in a foreign country anywhere in the chain of evidence makes intelligence gathering much more difficult.

If the feds want to use Google against me, it's rather easy if all my searching is on Google.com. The FBI (A US agency) can just send a NSL to my ISP (A US company) which will be obliged to turn over my IP address, which they can send to Google (A US company) in another NSL and oblige them to turn over my search history identified by my IP address (located within the US).

Re:Scroogle

[Anonymous Coward]

While I would love to see a good rant towards Google and while I also myself use Scroogle, the summary isn't really being truthful. Google hasn't blocked anything, they just changed the page that Scroogle scrapes and they're throwing a hissy fit about it.

From the Scroogle announcement:

We regret to announce that our Google scraper may have to be permanently retired, thanks to a change at Google.

That interface was at www.google.com/ie but on May 10, 2010 they took it down and inserted a redirect to /toolbar/ie8/sidebar.html. It used to have a search box, and the results it showed were generic during that entire time.

Now that interface is gone. It is not possible to continue Scroogle unless we have a simple interface that is stable. Google's main consumer-oriented interface that they want everyone to use is too complex, and changes too frequently, to make our scraping operation possible.

Google changing something isn't exactly "blocking" a third party service. Even more so, it's just a few lines of code to get the results from main Google search too. All the search results and links have approciate html ID's associated to them and it's been the same for years already.

I have no idea why Scroogle is bitching about this.

Oh well. I changed to use ixquick, which also has the added benefit of being located in the Germany rather than US and a lot better and useful interface.

-sopssa

If Microsoft and Google are any indication, there is no large corporation Sopssa won't defend and make an apology for.

Re:Scroogle

[somersault]

You could try following the links, dumbass

https://www.european-privacy-seal.eu/awarded-seals

Now, it doesn't mean there will never be a data breach (by a disgruntled employee or whatever), but the fact that they have actually bothered to get a certification says to me that these guys are more committed to data privacy than your average website that collects and distribute personal information.

Re:Scroogle

[davester666]

But then they only have the last 18 months of your search history, and your browsing history through google analytics, and all your email [no NSL necessary for that!], and any documents in google docs.

Actually, the email part is scary. Evidently, the law is that email on a server is freely accessible by federal law enforcement, without a warrant or even a 'we think he knows somebody who may have parked next to a fire hydrant' if it's been stored for more than six months...

Re:Scroogle

[cynyr]

doesn't http://code.google.com/apis/ajaxsearch/ count? Also i'm fairly sure that it shouldn't be that hard to get the links out of the normal google search(but it's been a while since i looked at the HTML of the results. yep,

a href=foo class=1> name /a> (sry i had to mangle it a bit, but the idea should be clear)

The results are the only links to have a class of 1. so a simple matter of parsing it as XML should work.

Re:Scroogle

[Anonymous Coward]

So, kinda' like how filesharers don't actually give anything back to the artists they claim have been cheated by the record companies....

Ah, there is a certain glory in being an AC and modded out of existence. A martyr kind of thing....

Re:Scroogle

[Anonymous Coward]

I love Startpage. As a metasearch engine, it's pretty darn good.

As do I. I've been using it since Alex Jones started promoting it, and I have no intention to switch back to Google.

Re:Scroogle

[theshowmecanuck]

After GW Bush opened Pandora's can of worms with respect to secret police searches and constitutional bypass operations on the heart of U.S. justice an freedom, there is a strong perception that there is NO protection from government intrusiveness in the U.S. any more. So may are now feeling that a little privacy is a lot better than none. That said, it matters whether you trust your government and form of government or not. If you do, then logically it doesn't matter if they can spy on you in the night, come search you, or put you up against a wall, because they are trustworthy and wouldn't do anything wrong. But the government is made up of people, and people of course never do any wrong. [/sarcasm]

Re:Scroogle

[geekoid]

The US can't do that in the US either. Just an FYI.

" Frankly, EU and European countries take privacy a lot more seriously,"
Care to back that up? I mean when you can take time away from being on public video, told what you can and can not say, carrying papers,

IT would be more correct to say it treats privacy different;which makes sense because what it considered ''privacy' is different. For example, what you do in public can be considered 'private' in some countries.

Of course it's such a patch work in the EU, it's almost nonsense to say to use the EU as a generally statement concerning privacy.

Re:Scroogle

[Anonymous Coward]

but it's a service that exists parasitically to Google proper. I'm not trying to imply anything unethical by using the word "parasite", but this really is a situation where Scroogle uses Google's capabilities/services without contributing anything back to Google.

The word you're looking for is commensalism. Although I think in this case it is closer to parasitic since it does use some of Google's resources without giving back much or any value to Google itself.

Re:Scroogle

[Chewbacon]

They don't really appear to be bitching about it so much as asking what's going on. They are planning to contact Google to see WTF up. I think they are suspicious that Google is taking the contra-Don't Be Evil route, which wouldn't surprise me either. I still love google.

Re:Scroogle

[logjon]

Not sure which US you live in, but here in the US I'm a citizen of, the government has unfettered access to communications, digital and otherwise. The patriot act took the last of American privacy, and with a hearty chuckle, wiped its ass with the remainder of the fourth amendment.

Re:Scroogle

[insufflate10mg]

Why can't they just scrape off of the original, classic Google search page. Isn't that the simplest possible interface Google could possibly provide? It consists of a textbox and 2 buttons.

Re:Scroogle

[logjon]

Trusting any government is a bad idea.

Re:Scroogle

[Anonymous Coward]

Except filesharers represent the largest customer base.

http://arstechnica.com/tech-policy/news/2010/05/file-sharers-are-content-industrys-largest-customers.ars

Re:Scroogle

[dAzED1]

...carrying papers...

Haven't been listening to the news about AZ lately, eh?

Re:Scroogle

[_xeno_]

Because if they did that, they'd be forced to abide by the search Terms of Service. And they appear to be violating Section 1.4.

By using the generic web robot approach, they're allowed to scrape Google based on the same concepts that allow Google to scrape third party web pages in the first place.

From Google's robots.txt:
User-agent: *
[snip]
Disallow: /ie?

Well, OK, so they're not obeying robots.txt in the first place. But ignoring that one pesky fact, uh...

Re:Scroogle

[negRo_slim]

What's the benefit of being in Germany?

I may be mistaken but I believe they have stronger privacy laws.

Re:Scroogle

[joeszilagyi]

And that's why you host your own e-mail on your own server with your own encryption.

Re:Scroogle

[Anonymous Coward]

The US can't do that in the US either. Just an FYI.

Horseshit.

The US passed laws basically saying that any data which lands on a US server (or one controlled by a US company) is open to full inspection by them, regardless of the laws in the country of origin.

Also, in this court case, they established that:

to solve this case, the court resorted to the Restatement (Third) of Foreign Relations Law 442 (1987). Section 442(1)(a) states that a court or U.S. agency may, when authorized by statute or court rule, to order a person to produce documents, objects, or other information relevant to an action or investigation, even if the information or person in possession of that information is outside the U.S.

It covers how they might balance US wishes with foreign laws, but more or less leaves it open to interpretation. If they want it bad enough, they'll decide they have the right to compel it.

Expect this to get worse as the US continues to decide that its self interest trumps the sovereignty of other countries.

It's easy to see how a lot of anti-US sentiment can happen when your politicians act like bullying assholes to the rest of the world.

Re:Scroogle

[icebraining]

And if you live in the UK, they can arrest you for not providing the key.

Re:Scroogle

[Smauler]

The Data Protection Act in the UK was as a result of the Data Protection Directive from the EU. This severely limits what people and/or companies and/or governmental agencies (with some exceptions to the last) can do with information about you. I'm not aware of legislation as strong as this in the US, but I may be wrong.

I do agree that different countries treat privacy differently - I personally believe that anything I do in public is basically that - public. I won't ever carry papers in my own country, so if somehow the ID card in the UK goes through (looking very very unlikely at the moment), I'll just lose mine every time I get a new one, and reapply. Some people don't have a problem with such things, but I do. The EU is a very diverse place, but that data protection directive means that all EU countries have similar laws with regards to data protection AFAIK.

Re:Scroogle

[DarkKnightRadick]

Unfortunately the results are not good enough for me to stop using Google. ):

Re:Scroogle

[icebraining]

Except each search query Scroogle did cost some cents to Google, while a filesharer copying a song/album/movie to another person costs the record companies exactly zero.

They may had no right to copy it, and maybe they shouldn't have done it, but claiming each P2P copy costs real money to the companies is ludicrous. All the costs (bandwidth and electricity) are paid by the two sharers.

Re:Scroogle

[MikeBabcock]

As in they don't have a leg to stand on, yeah.

"Hey we're scraping a page you told us not to scrape as a robot and you moved it" - "We have a public API" - "We don't wanna follow your licensing terms"

Re:Scroogle

[Anonymous Coward]

They make great stuff.

Re:Scroogle

[slaad]

From the FAQ:

European Privacy Seal On July 14th 2008 Ixquick received the first European Privacy Seal from European Data Protection Supervisor Mr. Peter Hustinx. The Seal officially confirms the privacy promises we make to our users. It makes Ixquick the first and only EU-approved search engine. Both EU Commissioner Viviane Reding and Dr.Thilo Weichert, German Privacy Commissioner complemented Ixquick on its privacy achievements. You can find the press release here.

Since I am in EU, it also means US can't just randomly get data that doesn't belong to them, ie. for people from other countries. Frankly, EU and European countries take privacy a lot more seriously, for historical reasons too.

I'm pretty sure the NSA doesn't care about the European Privacy Seal.

Re:Scroogle

[Myopic]

That could be, and if so it would be significant, but the benefit might simply come from the server being in pretty much *any* other country. I imagine it's a lot more footwork for a police department to get information from foreign servers versus domestic ones. It's one more barrier in front of your privacy. No barrier is perfect, but any barrier is better than none (if privacy is your goal).

Re:Scroogle

[Myopic]

Nice. How's the weather there? The rest of us are stuck here in the United States of Reality.

Re:Scroogle

[icebraining]

Using the API means agreeing with the ToS, which prohibits that kind of usage. Since using the API requires signing up for a key, they couldn't say they never needed to read it to use the service.

IANAL, but i think having to click "I agree with the TOS/EULA" is different, legally, from being able to use it without accepting anything.

in ProCD v. Zeidenberg, the license was ruled enforceable because it was necessary for the customer to assent to the terms of the agreement by clicking on an "I Agree" button in order to install the software. In Specht v. Netscape Communications Corp., however, the licensee was able to download and install the software without first being required to review and positively assent to the terms of the agreement, and so the license was held to be unenforceable.

Re:Scroogle

[Anonymuous Coward]

I'll just lose mine every time I get a new one, and reapply.

And pay a hefty fine every time.

Re:Scroogle

[clone53421]

It’s not a fully-automated tool, therefore not a robot. It scrapes the page only once at a real user’s direct request.

As such it is no different from a browser (which also scrapes – downloads, parses, and translates into useful format – a page once at a user’s direct request)...

or, for that matter, any different from the IE search interface that the /ie path was meant to support!

Re:Scroogle

[slaad]

Since I am in EU, it also means US can't just randomly get data that doesn't belong to them, ie. for people from other countries. Frankly, EU and European countries take privacy a lot more seriously, for historical reasons too.

I'm pretty sure that the NSA doesn't much care about the European Privacy Seal.

Re:Scroogle

[insertwackynamehere]

Why is everyone so willfully ignorant to theft? By downloading media without paying for it, there is no tangible product that costs anyone any money except for bandwidth, but that is not what pisses off record companies. What pisses them off is that they release a product for sale with a value attached to it. This value factors in attempts at profits and attempts at covering the cost of production/paying the artist/etc. When music is illegally downloaded, the company loses money because someone now owns the song without paying for it. Now maybe that person wouldn't have paid in the first place and unlike shoplifting, there was no physical loss or diminishing of goods. However, the fact is that the person downloading the song is still taking something of value without purchasing it. This is theft. And media can have intrinsic value..it costs money to make good music and songs, money spent with the expectation of high returns in sales. If everyone downloads music, then the free market will indeed win as many Slashdotters smugly wish for, hoping to prove the things people make for them that they like aren't actually worth anything. However this free market win will be a ceasing of production for movies and songs..no one can profit so no need to fill the void.

Re:Scroogle

[Mister Whirly]

What's the benefit of being in Germany?

Because obviously nothing evil could ever come from Germany.

Re:Scroogle

[Anonymous Coward]

They are bitching about it because they are crazy anti-google wacko hypocrites.

They devote lots of space to google bashing while using ONLY google's search engine. Nice eh?

Re:Scroogle

[al0ha]

Actually IxQuick is located in The Netherlands which is certainly not Germany and darn well good that it is not because Holland does actually concern itself with privacy protection, Germany on the other hand, not so much.

Re:Scroogle

[Antiocheian]

by sopssa (1498795)

[...]

-sopssa

I still don't believe it's you.

(BTW, is this some new trend in Slashdot? I've noticed others are doing it too)

Re:Scroogle

[logjon]

Is that the one where the supreme court didn't rule that violations of the fourth amendment are okay as long as they're really, really important?

Re:Scroogle

[Anonymous Coward]

WTF you call sauerkraut and sausages excellent food?? thanks but NO THANKS.

I have lived in Germany for almost 2 years and I would say that German food is definitely NOT an "advantage".

Beer I agree, the best shit I have tried! Of course, the food is better than the UK but... I will change any time a brötwurst for a good portion of Smoked BBQ Ribs! (Real American!)
p.s. I am not American, I am a Mexican :)

Re:Scroogle

[Anonymous Coward]

It sure is convenient forgetting that the Patriot Act had expiration dates, and all of the bad parts of it have since expired or been repealed.

The parts that remain are the parts that fixed what Clinton broke when he blocked the different agencies from being able to talk to each other.

You do know that one of marijuana's side effects is paranoia, right?

Re:Scroogle

[Myopic]

Yes exactly. It's the one where the Supreme Court rules that violations of any amendment are okay as long as they're really, really important. That, again, is called reality. Here in reality, we know that sometimes rules need to be broken.

In the case you cite, though, I don't like the way that rule is broken. I wish it were different, here in reality. There are lots of things I wish were different, here in reality, but I don't pretend that limited exceptions to constitutional provisions amount to (quoting you) "unfettered access to communications". In this case, the operative word is "unfettered". Here in reality, the government has *fettered* access to communications, although sometimes I wish they were slightly more fettered.

Re:Scroogle

[logjon]

I'm sure they're being very responsible with all the data they can secretly seize and nobody can ever question them about.

Re:Scroogle

[soliptic]

Thanks for the information. It is simply pathetic that half the time I open a story on slashdot, the first comment serves to point out the headline and/or summary are essentially complete horseshit. Fair enough on a pure UGC site like reddit, but this place has editors - paid editors. Really, though, it seems that calling them "editors" is about as accurate as saying Google have "blocked" Scroogle.

Re:Scroogle

[oztiks]

Can't they use Googles Custom Search http://www.google.com/cse/ pay the stagger $100 year fee to not have ads and go back to normal?

Re:Scroogle

[Beezlebub33]

Why is everyone so willfully ignorant to theft? By downloading media without paying for it, there is no tangible product that costs anyone any money except for bandwidth, but that is not what pisses off record companies. What pisses them off is that they release a product for sale with a value attached to it. This value factors in attempts at profits and attempts at covering the cost of production/paying the artist/etc. When music is illegally downloaded, the company loses money because someone now owns the song without paying for it. Now maybe that person wouldn't have paid in the first place and unlike shoplifting, there was no physical loss or diminishing of goods. However, the fact is that the person downloading the song is still taking something of value without purchasing it. This is theft. And media can have intrinsic value..it costs money to make good music and songs, money spent with the expectation of high returns in sales. If everyone downloads music, then the free market will indeed win as many Slashdotters smugly wish for, hoping to prove the things people make for them that they like aren't actually worth anything. However this free market win will be a ceasing of production for movies and songs..no one can profit so no need to fill the void.

No, it is not theft. Theft is depriving the owner of the use of something. Downloading songs is not theft because you have not deprived the owner of the use of it. It is copyright infringement. It's also illegal.

And in response to a likely response, no, it is not the same thing. You might as well call it rape or murder, if you are going to abuse terms. You are not entitled to redefine words.

"That student was raping the music company!" sounds even better, right?

Re:Scroogle

[Anonymous Coward]

Why are some so willfully ignorant of the difference between sharing and theft?

I believe that theft is taking property that isn't yours. To me, and many others, sharing art simply doesn't qualify as theft unless you so pervert the concept of property as to destroy the concept of public domain.

Intellectual property is simply a contradiction in terms.

Re:Scroogle

[Myopic]

Sorry, my reptilian alien overlords won't let me comment on that conspiracy.

Re:Scroogle

[Anonymous Coward]

Applications like these do not have to follow robots.txt since they're powered by user action. robots.txt is for completely automated spiders.

Re:Scroogle

[logjon]

It may be hard for you to comprehend, but some people aren't as eager as others to give up rights for a warm fuzzy sense of safety.

Re:Scroogle

[Obfuscant]

Haven't been listening to the news about AZ lately, eh?

Don't understand the news about AZ lately, eh?

Re:Scroogle

[Anonymous Coward]

The parts that remain are the parts that fixed what Clinton broke when he blocked the different agencies from being able to talk to each other.

Citation Needed.

Re:Scroogle

[Jugalator]

Yes, Scroggle has obviously been using a hacky way of getting their results. /ie was always an odd IE support page at Google. I can understand that it changed now that Google has made a thorough site refresh.

Re:Scroogle

[wurp]

+1, MC Frontalot in your sig

Re:Scroogle

[lehphyro]

WTF is NSA?

Re:Scroogle

[teslafreak]

It may be difficult for you to comprehend that if the government cared as much about your data as you suspect, you likely wouldn't be able to type these things (as you would be either in prison or dead).

Re:Scroogle

[clone53421]

Yes, and contains utterly nothing of value.

In fact it is immensely easy to “interface” with that page. You just download http://www.google.com/search?q=insert+search+terms+here.

The problem is, that HTML code is meant to be displayed by a browser in human-readable form. You have no idea what the HTML code of that webpage will look like in order to have a machine interpret the results, and even if you figure out how to machine-parse the results they could completely change their HTML tomorrow and your interface would break.

For instance, this is an example of a simple scraper.

Perform any Google search, then paste this code into the Address bar and press Enter:

javascript:var a=["</a></h3>","<em>","</em>","<b>","</b>","<h3"];try{var e=document.all[0];for(i=0;i<a.length;i++)a[i]=a[i].toUpperCase();}catch(e){};ap=Array.prototype;ap.skip=function(){this.shift();return this;};ap.splitAll=function(k){for(i=0;i<this.length;i++)this[i]=this[i].split(k);return this;};ap.joinAll=function(k){for(i=0;i<this.length;i++)this[i]=this[i].join(k);return this;};ap.shiftAll=function(){for(i=0;i<this.length;i++)this[i]=this[i].shift();return this;};ap.popAll=function(){for(i=0;i<this.length;i++)this[i]=this[i].pop();return this;};ap.collate=function(j){for(i=0;i<this.length;i++)this[i]=[this[i],j[i]];return this;};String.prototype.replaceAll=function(k,j){return this.split(k).join(j);};alert("\""+document.body.innerHTML.split(",'','','res','").skip().splitAll(a[0]).shiftAll().splitAll("\">").popAll().collate(document.body.innerHTML.split(a[5]+" class=").skip().splitAll(a[0]).shiftAll().splitAll("href=\"").popAll().splitAll("\"").shiftAll()).joinAll("\"\nurl: ").join("\n\n\"").replaceAll(a[1],"").replaceAll(a[2],"").replaceAll(a[3],"").replaceAll(a[4],""));

It works. But it is extremely sensitive to the exact HTML of the search results page, and I have no idea whether it would work tomorrow. Google might decide to change the HTML template for their search results and the scraper would most likely need to be changed.

Re:Scroogle

[operagost]

Considering how weaselly Congress is these days, I'd prefer that they get less ferreted.

Re:Scroogle

[operagost]

Hey, neither does the President. Despite the clause that explicitly prohibits using ethnicity as a determining factor, he's claimed that Mexicans are being rounded up from ice cream parlors as we speak.

Re:Scroogle

[logjon]

I never said that they're out to get me. I think you should re-read my posts, and try responding without putting words in my mouth.

Re:Scroogle

[operagost]

I'm glad that Eurasia has certified that we are safe from the prying eyes of Oceania.

Re:Scroogle

[Anonymous Coward]

a filesharer copying a song/album/movie another person costs the record companies exactly zero.

This must be the most stupid comment moderated +4 Insightful this year on Slashdot...

(Hint: Downloading removes the incentive to buy it.)

Re:Scroogle

[operagost]

Sounds like Pennsylvania. We're full of Germans too.

Re:Scroogle

[geminidomino]

WTF is NSA?

Expansion Error: (NSA) No Such Agency

Re:Scroogle

[geminidomino]

(Hint: Downloading removes the incentive to buy it.)

ProTip: No, watching/playing a crappy movie/game removes the incentive to buy it.

Re:Scroogle

[AndersOSU]

ethnicity can't be the sole determining factor. Ethnicity can be considered.

Anyone who thinks that the AZ law won't lead to profiling is deluding themselves.

Re:Scroogle

[JoeF]

Frankly, EU and European countries take privacy a lot more seriously, for historical reasons too.

Hmm, have you ever heard of the GEZ in Germany??? They are worse than anything in the US.

Re:Scroogle

[Goaway]

I have no idea why Scroogle is bitching about this.

Mostly because the creator of Scroogle is kind of a complete nutcase, and he just spotted a great chance to throw a hissy fit about how evil Google is.

Re:Scroogle

[clone53421]

So what you are predicting is that we will learn that lots of legal Hispanic Americans will be detained and/or harassed with demands for them to prove their citizenship.

Ok, we’ll see if you’re right.

Personally, I predict that we’ll learn that lots of illegal Hispanics were stopped for unrelated things, found to be illegally here, and sent home. That’s just my prediction, though. Oh, and I also predict that your prediction will turn out to be incorrect.

Re:Scroogle

[quickOnTheUptake]

Correct me if I'm wrong, but I think they send US traffic to servers in the US, specifically Cali: The second to last hop is:
gige-g1-1.core2.fmt1.he.net (72.52.92.246) which comes back as Freemont, CA.
The proxy itself was 74.82.58.2. This doesn't have any location info on record, but it is licensed to Hurricane Electric, Inc. which seems to be based in Fremont, CA.
Unfortunately.

Re:Scroogle

[AndersOSU]

We won't have a good sense of how many legal Hispanics were forced into producing proof of citizenship/residency because the police will never report that number.

They'll hold a little parade every time they deport someone though.

I predict that there will be on outpouring of stories on liberal and immigration issue blogs of people being harassed. I further predict that you will dismiss this as anecdotal, politically biased, and not representative.

Re:Scroogle

[dAzED1]

Let's look at what I was replying to:

I mean when you can take time away from being on public video, told what you can and can not say, carrying papers

Now, how am I taking the current AZ situation out of context? Is racial profiling not occurring, with people being told to show papers? In fact, that's exactly what is occurring. Which means, it is no longer valid to use Europe's habit of asking for papers as an indication that we have more liberties here - since that is now occurring here.

Note also that I didn't really indicate how I felt about the new law, either - I just said that it makes it silly to say we're different in that regard, since we're no longer different.

Thoreau, after being jailed for refusing to pay taxes due to his stance against the Mexican war (oddly appropriate...), wrote a dialogue containing these lines:

Under a government which imprisons any unjustly, the true place for a just man is also in prison. The proper place to-day, the only place which Massachusetts has provided for her freer and less desponding spirits, is in her prisons, to be put out and locked out of the State by her own act, as they have already put themselves out by their principles. It is there that the fugitive slave, and the Mexican prisoner on parole, and the Indian come to plead the wrongs of his race, should find them; on that separate, but more free and honorable ground, where the State places those who are not with her, but against her,--the only house in a slave-state in which a free man can abide with honor.

So a person with dark skin is walking along, and a police officer spots them and asks them for their papers. Note that can be the primary cause for the contact; the person doesn't have to have been doing anything wrong, they merely need to be a victim of racism. If the person doesn't have paperwork on them, they are jailed, and have to prove they are allowed to be here. Would I run that risk, being white? Nope. And thus, the issue that many people have with the new law.

Re:Scroogle

[dAzED1]

...and sent home.

Because the place where they sleep at night, have their clothes, their stuff, the place they go to after work, the place where their phone rings, and potentially even their wife and kids live...that's not their "home."

Yes, a person's worth as a human being can definitively be determined by where they were born. Does it ever blow your mind when you consider that George Washington wasn't a natural-born citizen?

They're willing to do things you're not willing to do. Why does one person's attempts to better themselves, and experience the liberties here in the US (which I served to help protect, in the USMC), bother you so?

Re:Scroogle

[icebraining]

(Hint: Downloading removes the incentive to buy it.)

That's not a cost. That's a possible loss of profit. Loss of profit is not a cost. Just like if I open a new competing car manufacturer company, the cost the other companies incur per car produced doesn't suddenly raise.

Re:Scroogle

[Anonymous Coward]

So...because the idea of a 'home' makes you feel all warm and fuzzy, we can't enforce our laws?

Re:Scroogle

[Anonymous Coward]

Doesn't help IP producers that they always flat-out lie about everything they say in regards to the matter.

Take the recent announcement from the BSA, if they cut piracy ("costing" upto 51 billion dollars) by 10% that would result in 500,000 jobs etc. etc.

And the other number - what was it ... - that music piracy costs the industry 200 billion dollars?

I grant you that somewhere in that 251 billion dollars there was some money to be had, that wasn't spent.

But these companies outright lie saying it is a loss of 251 billion dollars to the economy of the country the music/software was made in. Which is not true, you'll find that 251 billion dollars was spent *elsewhere* in the economy.
And they are all crying poor? How many music publishers have you seen fold over the last 10 years due to "a loss of 200 billion dollars". It's none, because they don't pay the music performers in the first place. They "Loan" them the entire cost of producing and printing an album, in return for perpetual ownership and distribution rights of their work, if they don't pay off their "loan" - using their part of the profits - they get nothing!. In effect, music publishers own 90% of the value of music when they loan artists the money to produce it. (they also charge the artists to produce it, and ensure the artists pay them for the privelidge).

Its like a bank loaning you money for an investment property, then renting it out for you.
You get a loan of 100k to buy the house.
The Bank ends up owning 90% of the house. (and thus gets 90% of the rent).
The Bank pays off (for you) the loan out of your 10% of the rent.
Say you get 1000 dollars per month rent.
So the Bank kindly pays off the loan to buy the house at 100 dollars per month.
1000 months later, the house is yours! .. well, not exactly, 1000 months later the house is 90% owned by the bank, and you get 100 dollars per month from your 10%...

Re:Scroogle

[ooshna]

Secretly seizes? I for one good sir do not believe the government does anything in secret..... wait.... hold on... I thought I heard something on my phone. Good thing you need a court order to wiretap... what was I talking about again?

Re:Scroogle

[ooshna]

Don't you love how imaginary lines on a map determine so much about you and what you can or cannot do.

Re:Scroogle

[ooshna]

Damn Germany deports quick they got you after only two years? In the US if you don't get caught on the way in your golden.

Re:Scroogle

[clone53421]

You know what’s harassment? You know what’s a violation of civil rights? Stopping 620+ drivers and forcing them to allow you to sample the very breath from their lungs just to find the 20 who were driving while intoxicated.

Some illegal alien gets stopped for an actual infraction, or picked up after committing a crime? Send them back to their country of origin.

Re:Scroogle

[jonadab]

> What's the benefit of being in Germany?

Ordinarily, I would say using a service located in the US would be at least as good as using one located in Germany.

But in this case, Google happens to be in the US, so if the interface you use to access Google is *not* in the US (Germany, for instance, is not in the US), that means it's in a distinct jurisdiction. Consequently, if anybody nefarious wanted to do dastardly things that require finding out what you're searching for, they'd have to operate in at least two countries. So that would be at least a little bit harder for them, at least potentially. Of course, it also matters where *you* are. If *you* are in Germany, then that pretty much negates the advantage of using a Germany-based service. OTOH, if you are in the US, the Germany-based service is even more advantageous.

That all probably sounds a bit paranoid, but to the sort of person who wants an SSL-protected web search interface, such things might seem relevant. Personally the only thing I really care about SSL for is secure shell (SSH), but YMMV.

Re:Scroogle

[AndersOSU]

I think traffic checkpoints are of dubious constitutionality, but at least they're egalitarian infringements.

Obviously, the bulk of of people required to prove their citizenship under the AZ law are going to be those suspected of some sort of wrong doing (probably mostly traffic stops.) That, to me, is not particularly troubling.

What is troubling is that I suspect that at the same DUI checkpoint that you're outraged about they will let all the white people go after blowing, but demand, "papers please" from all the brown people - both drivers and passengers. Of course, they weren't selected solely because of their ethnic background, there were also 4 people in a car on a "known human trafficking route" (never mind that this particular trafficking route would have originated at the bar across the street from the checkpoint). Or some other canned "reasonable and articulable suspicion."

Re:Scroogle

[clone53421]

Because the place where they sleep at night, have their clothes, their stuff, the place they go to after work, the place where their phone rings, and potentially even their wife and kids live...that's not their "home."

Squatters rights, eh? Damn right it’s not their home. They came by it illegally.

Yes, a person's worth as a human being can definitively be determined by where they were born. Does it ever blow your mind when you consider that George Washington wasn't a natural-born citizen?

A fine straw man indeed.

A person’s ethics can be effectively determined by whether they abide by the law, as it justifiably and accurately applies to them. They can also be determined by how a person acts when the law unjustly applies to them, and whether or not they will pay the penalties imposed by it. I’m not saying you have to be a Ghandi or Thoreau and turn yourself in. I’m just saying that if you are caught violating a law you should be subject to the law even if you think it’s an incorrect law.

They're willing to do things you're not willing to do.

Incorrect. I am willing to do those things. Just not at the prices they are willing to do them for.

I guarantee that if every illegal worker disappeared tomorrow, after the major upheaval and turmoil subsided the labour market would bring itself to a wage that American workers were willing to work for and employers were willing to pay. It wouldn’t be a pretty process and it wouldn’t be an ideal way to reach a solution... it’d be a major fiasco and an incredibly terrible mess, but it would eventually find a new equilibrium.

Why does one person's attempts to better themselves, and experience the liberties here in the US (which I served to help protect, in the USMC), bother you so?

I thank you for your service. I must ask, though: Why does a mass flaunting of this so-called liberty by boldly transgressing the laws of the land that you fought to help protect not bother you?

Re:Scroogle

[clone53421]

What is troubling is that I suspect that at the same DUI checkpoint that you're outraged about they will let all the white people go after blowing, but demand, "papers please" from all the brown people - both drivers and passengers.

I’ll be just as upset over such gross violations of our rights as I would over the DUI checkpoints themselves. Not for the people who are breaking the law: they just got caught, and they deserve what punishment they get. I’ll be outraged for the people who were obeying the law who were unconstitutionally harassed.

Re:Scroogle

[cunina]

They're willing to do things you're not willing to do.

Like violate this country's immigration laws, for example. Otherwise, I'll do whatever they're doing, for the right price. They're just willing to work more cheaply than I. And the net effect of that is to import poverty and widen the income gap in this country. Does it give you a warm fuzzy feeling to know you're on the same side as ConAgra and Wal-Mart?

Re:Scroogle

[spanky the monk]

The fact is, that it costs ~$0 to reproduce a song. So it's intrinsic value is roughly ZERO! I work for an hour to earn the cost of a CD, then go to record company and say I want album X. Record company preses a button and ZAP, a fresh copy of album X. That'll be $30, please come again. So what exactly am I paying for? Charging money for abstractions makes no sense, economically. The reason people don't want to pay for copies, and don't feel bad about "stealing" them, is because they're actually not worth anything.

Re:Scroogle

[atomic777]

Perhaps I'm oversimplifying, but the approach to privacy in the EU and US is fundamentally different. EU privacy laws involve explicit regulation of what can/cannot be done in many specific scenarios, while the US approach is much more 'laissez-faire'.

An important example of this is the EU Data Protection Directive requirement that personal data not be transferred to here-be-dragons, privacy-unsafe countries -- including the US. Some countries like Canada and Argentina brought their privacy legislation into compliance with the directive, while the US compromised and offered a voluntary "Safe Harbour" program that allows a US company to assert their compliance if they wish to do so.

The simple fact that US companies must work to achieve compliance with EU privacy laws, while i'm not aware of any similar problem for European companies in the US, would seem to me a good indication of their relative strength.

Re:Scroogle

[atomic777]

Yes, even though there is the "Safe Harbour" program for US-based companies to comply with the EU directive, it is unclear what laws are given priority.

On a side note, this might be why Canada decided to fully comply with the directive, presumably to encourage transatlantic businesses to locate data centers there. With strong privacy protection, a cold climate and plenty of renewable electricity, it is a wonder to me why there isn't a much stronger push for data centers...

Re:Scroogle

[mikiN]

Contrast this to:

NSA: No Strings Attached
NSA: Nukes Still Armed
NSA: Nixon Sold America
NSA: Nerd's Sudoku Alternative ...ad nauseum

Re:Scroogle

[DocHoncho]

Actually it's Weird Al from "White and Nerdy"

http://www.azlyrics.com/lyrics/weirdalyankovic/whitenerdy.html
http://www.youtube.com/watch?v=N9qYF9DZPdw

Great song.

Re:Scroogle

[Anonymous Coward]

Copyright infringement is not theft. Please look up the definitions of the two. The vast majority of downloaders are either downloading to try out content before deciding if they want to buy it or are financially unable to afford the product and would not buy it regardless.
If 'everyone downloaded' creative industries would not collapse, but industry middlemen would. New and creative forms of content distribution such as the humble indie bundle would continue to be just as viable as they are now in such an environment.
Creative content industries need to stop attacking those who enjoy their products and punishing those who do buy their products by giving them inferior DRM-laden versions. People will continue to largely download such content as long as the 'pirated version' is vastly superior to the paid version.

Re:Scroogle

[Anonymous Coward]

Actually I just went to scroogle.org and it worked without a hitch.

Re:Scroogle

[PReDiToR]

No kidding.

The cost of a replacement passport
An adult passport costs £77.50 for standard service and £112.50 for the one-week Fast Track service. A child passport costs £49 for standard service and £96.50 for the one-week Fast Track service. The Premium one-day service is not available for replacement passports.

And for Driving Licence:
Apply by phone
You can apply by phone if one or both parts of your photocard licence have been lost or stolen and none of your details have changed. DVLA accepts the following credit or debit cards - MasterCard, Visa, Eurocard or Maestro and the fee is £20.00.

Re:Scroogle

[kiwimate]

Copyright infringement is not theft. Please look up the definitions of the two.

Okay, I'll do that.

Theft: the act of stealing

Stealing: 1 a : to take or appropriate without right or leave and with intent to keep or make use of wrongfully b : to take away by force or unjust means c : to take surreptitiously or without permission d : to appropriate to oneself or beyond one's proper share : make oneself the focus of

Copyright infringement is taking something without permission. Equals stealing. Equals theft. And by the way, why does it make things ANY better when slashdotters self-righteously proclaim "it's not theft or stealing, it's copyright infringement"? Nice one, all you're doing is muddying the argument over semantics; as is frequently pointed out, it's still illegal.

Next time you decide to pontificate, make sure you know what you're getting into.

Re:Scroogle

[Achromatic1978]

... according to filesharers, who may just be biased, and may just have a vested interest in trying to minimize the perception or reality of a negative effect upon an industry from said filesharing, you mean?

Re:Scroogle

[Anonymous Coward]

Frankly, EU and European countries take privacy a lot more seriously

Really? I'd say it is the other way around. Take a look at all the laws invading privacy in Sweden as a good example.

Re:Scroogle

[Anonymous Coward]

EU banks are sending to USA the information about your bank transactions, right now. (Cross-country transactions at least).

Re:Scroogle

[CharlyFoxtrot]

told what you can and can not say

Only hate speech (inciting discrimination/intolerance and pro-nazi propaganda) is forbidden in the EU; to suggest this somehow limits our freedom is like suggesting you aren't completely free because you cannot own a slave. A freedom is not a freedom if it comes at the cost of the rights of others, your rights end where someone else's begin (in this case the right not to be discriminated against.)

Re:Scroogle

[Caetel]

It's different mainly because Google doesn't get the ad views (and potential hits/revenue) which is the reason they offer a search engine in the first place.

Re:Scroogle

[bram]

Too bad ixquick's search results suck.
Not as much as the other search engines but they're far away from google's.

A plus to using ixquick compared to SEs like http://www.ecosia.org/ is that they are ssl capable so you can use https://www.ixquick.com/ so nobody but them sees your search queries.

Re:Scroogle

[Anonymous Coward]

Network Error (dns_unresolved_hostname)

Your requested host "www.gez.de" could not be resolved by DNS.

Re:Scroogle

[Anonymous Coward]

Copying is not taking. Please look up the definitions of the two.

Re:Scroogle

[winkyeah]

I wonder how many people created a new tab and went to the .COM version to check it out. LOL

Re:Scroogle

[clone53421]

Google doesn’t have any ads on their search results page.

And yes, I made sure to use a browser that didn’t have Adblock to double check before I said that.

Re:Scroogle

[AmiMoJo]

You are not "taking". You are copying when you download.

Re:Scroogle

[Anonymous Coward]

Scroogle.org is searchable again.

Re:Scroogle

[Anonymous Coward]

beer is substantially better as well, so are the cars, only thing they can't do it play quake, so America probably has more brown and more crates.

Re:Scroogle

[Kattspya]

Now you need to look up the definitions of the word copy and take and realize the difference between them. Good luck.

Re:Scroogle

[Anonymous Coward]

Isn't there any other cost related to the Internet infrastructure that isn't paid neither in bills, nor in taxes by the two parties? Well, probably not...

They didn't block it...

[stagg]

If you RTFA you'll notice that Google didn't block Scroogle, they just upgraded without consideration to its functionality. As soon as someone can explain why Google WOULD have Scroogle on a dependency chart we can all put our conspiracy hats back on.

Re:They didn't block it...

[enaso1970]

They WOULD because if you change some letters in Roswell and add one, you get Scroogle! Isn't it obvious???! It should be to the aliens who run Google. Clearly the government has you in their clutches - only an upgrade to TinFoilHat 2.3 will save you.

Re:They didn't block it...

[Monkeedude1212]

Are you trying to tell me that a Private Corporation such as Google doesn't regularily consider the functionality of other companies who slam it on a regular basis, such as Scroogle?

Re:They didn't block it...

[Anonymous Coward]

Google is a Public Corporation.

Re:They didn't block it...

[Anonymous Coward]

Scroogle was a company?
Uh, no. And there were of no consideration to Google.

Re:They didn't block it...

[Anonymous Coward]

WHAT? When did 2.0 come out? I'm still on TinFoilHat 1.4...

Re:They didn't block it...

[HasselhoffThePaladin]

WHAT? When did 2.0 come out? I'm still on TinFoilHat 1.4...

See, I knew you were going to say that.

Re:They didn't block it...

[icebraining]

Scroogle is owned by Public Information Research, Inc. Their board of directors is in Scroogle: http://www.scroogle.org/staffsc.html

Re:They didn't block it...

[Anonymous Coward]

Scroogle is simply too big to fail!

Re:They didn't block it...

[Anonymous Coward]

And how, exactly, does scroogle slam google?

The Summary Lies!

[dancingmilk]

What a horrible summary. Google didn't block anything, they just changed the page that Scroogle scrapes off of. Scroogle claims that they need a "simple" interface to scrape off of. Sounds to me like they are too lazy to adjust their service.

Re:The Summary Lies!

[animeshpathak]

Agreed. And why can't they use the Google AJAX search API?

Re:The Summary Lies!

[clone53421]

Sounds to me like they are too lazy to adjust their service.

...at the whim of an update schedule that is irregular, unannounced, and liable to massive changes that would break scraping.

Re:The Summary Lies!

[idontgno]

break scraping.

Scraping is inherently unreliable. Particularly if you're scraping without the data source's permission or cooperation. It's what you do with the bottom of the barrel.

If you want reliable, you won't be doing any scraping. If you're doing scraping, don't get bent out of shape with it suddenly stops working. By choosing a scraping solution, you've committed yourself to intermittent service and a continual race to keep up with target interface changes.

Of you can use the provided API? Yes, it has limitations. But one of them isn't "brittle, unreliable, and subject to complete failure without notice".

Re:The Summary Lies!

[clone53421]

I’m not defending their choice to scrape... just pointing out that whether or not scraping was a good idea, their justification for not scraping (any longer) is perfectly reasonable.

If you’re asking “why are they scraping in the first place”... well, they found an interface to scrape from that they thought would never change... and surprisingly enough, it did.

Re:The Summary Lies!

[natehoy]

Scroogle has the absolute right to a refund for any and all money that they have paid Google because Google isn't living up to the contract where Scroogle pays Google for a stable connec... wait, what was that? Oh, I see. Never mind.

Scroogle may be providing a service that people value, but they are still using Google to do it, and not paying Google for that access. Google is tolerating this, which is all well and good, but they are under absolutely no obligation to make sure the connection is unchanged. Sites change all the time, and anyone who employs scraping technology as part of their technological solution should not be surprised when they do.

Re:The Summary Lies!

[schon]

just pointing out that whether or not scraping was a good idea, their justification for not scraping (any longer) is perfectly reasonable.

The decision to not scrape is reasonable. What is not reasonable is the outrage and blame placed on Google.

they found an interface to scrape from that they thought would never change

So in other words, they're stupid.

Re:The Summary Lies!

[clone53421]

I really can’t say I disagree.

I’m just saying it’s not based on quite as much laziness as you made it seem. Yes, they’re lazy, but scraping would’ve been the wrong solution anyway: it’s too much work.

Re:The Summary Lies!

[Skal Tura]

The default search there is now is easy to scrape, very easy infact despite the attempts to obfuscate the page.

There is this basicly:

div id: ires -- contains the results
li id: imagebox class g --- image results
li class g -- normal result

within li class: g there is: h3 class r - subject/title, and div class s -- result meta.

It's very easy to scrape. Infact anyone who has done a serious scraper with proper methods in the past, could probably scrape the results with just 30-60minutes of work properly, and from that doing the proxying page is easy.

Infact, the new Google interface sucks so bad i might do one myself, just to have a simpler interface, for myself.

The Google AJAX API is far from API to get the search results, quite contrary: It's pretty much just embedding in a even less scrapeable manner.

Re:The Summary Lies!

[dancingmilk]

Agreed. I really think that the folks over at Scroogle are trying to paint Google in a bad light on this when its their own software that is inadequate. If you are doing scraping and are going to throw a hissy fit whenever the page you scrape changes you might want to try something other than scraping...

A good days worth of work would fix this for Scroogle. I give them a day only because its clear they don't know what they are doing :)

Re:The Summary Lies!

[mzs]

Read the terms of use for why they would not want to. Then read the docs to see that even they wished to not honor the troublesome clauses in the terms of use for them, they could not do what they wanted to anyway with the API.

Re:The Summary Lies!

[BitwiseX]

What a horrible summary.

What a horrible "article" with a horrible title.
As many have already pointed out, Google has no obligation to Scroogle. Do we really think they are the only ones affected by this? I doubt it. They only reason they aren't screaming at Google is because they understand that when you scrape another person's or company's website, you are at their mercy.
Anybody who's ever done any kind of scraping, web or otherwise, knows this.

God bless RSS.

Optimize Google Firefox Extension

[areusche]

I have been using this extension for Firefox called Optimize Google. http://www.optimizegoogle.com/ It has the ability to disable click tracking and Google's ad services as well as a bunch of other features. Whether it works is up for debate. Personally I would not have used Scroogle since they can still track my usage and I don't really care if Google knows what I've been searching.

Re:Optimize Google Firefox Extension

[Anonymous Coward]

I have been using this extension for Firefox called Optimize Google. http://www.optimizegoogle.com/ [optimizegoogle.com] It has the ability to disable click tracking and Google's ad services as well as a bunch of other features. Whether it works is up for debate.

You use it, but don't know if it works?

Re:Optimize Google Firefox Extension

[0100010001010011]

The point isn't that Google or Scroogle can see what you're searching for, with SSL no one in between can see.

Say you search for "How to kill your wife and hide the body". With Google, every ISP that transfers packets between you and them has a record of it. With Scroogle only Scroogle knows what you searched for. (Not sure if they keep logs to subpoena).

Re:Optimize Google Firefox Extension

[Anonymous Coward]

You do realize that the program has other functions built into it?

Re:Optimize Google Firefox Extension

[Anonymous Coward]

Guess what, most ISPs do not save the data for later review. If they did many would need multiple petabytes per week. They might record what IP connected to What IP, but that is also unlikely.

Re:Optimize Google Firefox Extension

[clone53421]

Not sure if they keep logs to subpoena

“We don’t use cookies, we don’t save search terms, and logs are deleted within 48 hours.” – graphic on their homepage.

Re:Optimize Google Firefox Extension

[AntiDragon]

Yes, but once you have that list of results you're wide open to snooping again - those URLs you visit can be snooped the same way a Google search string can be.

The rest of the web is as as open and interceptable as a plain Google search. Far better to do your dodgy web activities from an anonymous location or via something like Tor.

In either case, this is a site that's piggybacking on another company's services for free. Google haven't deliberately blocked them and whatever my personal opinion of the company may be, they are under no obligation to check if their updates break these kinds of sites.

Re:Optimize Google Firefox Extension

[geekoid]

OR so they claim.. ba ba BAaaaaa

Re:Optimize Google Firefox Extension

[clone53421]

If you can prove otherwise, I’m sure that a few people from Scroogle would be hauled into court and possibly jailed.

Proving it might actually result in you being jailed too, though, so good luck with that.

Duck Duck Go

[Anonymous Coward]

I used to use Scroogle for privacy reasons, but switched to Duck Duck Go a few weeks ago. It is quickly becoming a great privacy-respecting alternative to Google and often gives more relevant results than Google.

Re:Duck Duck Go

[Anonymous Coward]

I too use Duck Duck Go.

Re:Duck Duck Go

[compumike]

I too have been trying Duck Duck Go (link to encrypted version) for the last several weeks and have been impressed.

Furthermore, check out their privacy policy, as well as a recent blog post about search privacy that explains why it "might be the most private place to search the Internet". No IPs logged, no cookies, no contractors.

There are also a large set of convenient "bang commands" such as searching "!slashdot foo".

And finally, searching over (encrypted) HTTPS just works "out of the box".

Give it a try for a few weeks!

Re:Duck Duck Go

[sexconker]

Goose!

Re:Duck Duck Go

[sproot]

It also seems to need javascript to work properly, unless you add it as a search plugin to ff.
Haven't seen a good explanation for that yet, scroogle managed perfectly well without it.

Re:Duck Duck Go

[Anonymous Coward]

And their FAQ quietly ignores why they need javascript. F'em.

Re:Duck Duck Go

[base3]

Yup -- not turning on JavsScript for that--no reason it should need it.

Re:Duck Duck Go

[Anonymous Coward]

I used to use Scroogle for privacy reasons, but switched to Duck Duck Go a few weeks ago. It is quickly becoming a great privacy-respecting alternative to Google and often gives more relevant results than Google.

Sounded good. Checked it out. Saw "This site requires JavaScript" and some links didn't work. Google works without javascript or cookies....

Re:Duck Duck Go

[Anonymous Coward]

And GP seems to be eerily complimentary of them. I don't like it.

All it looks like is some animation or font smoothing, but I followed an earlier poster's advice and am now on ixquick.

Re:Duck Duck Go

[Anonymous Coward]

I used to use Scroogle for privacy reasons, but switched to Duck Duck Go a few weeks ago. It is quickly becoming a great privacy-respecting alternative to Google and often gives more relevant results than Google.

Duck Duck Go is Amazing!!!!!!!! Thank you Mr. Duck!

Umm

[mindstrm]

When google wants them to stop, they'll be hearing from lawyers........ not just finding that google changed their page layout.

Re:Umm

[Anonymous Coward]

Dude, paying for a line of code is far cheaper than paying for a line of legal text. If that works, there's no reason to pursue anything further. If it doesn't work, nobody cares anyway since it doesn't actually change anything. The deprecation of IE pre-v8 is far more important to Google's eyes anyway.

Real API

[hey]

Doesn't Google have a real search API they can use?
Rather than using a kludge like google.com/ie (yuck)

Re:Real API

[jandrese]

In fact they do. It's not clear why Scroogle has such a hard on for screen scraping.

Re:Real API

[daveime]

It's not clear why Scroogle has such a hard on for screen scraping.

Perhaps because the API will probably block them after the first 10,000 hits and Google will ask them to contribute something back ?

Re:Real API

[mzs]

It used to (the SOAP one) the AJAX and REST ones do limit you to the first four sets of 8 results each though. Also a referer needs to be sent. A bunch of the terms and conditions would be troublesome to scroogle as well.

Re:Real API

[egcagrac0]

Terms of Use section 1.4:

You will not, and will not permit users or other third parties to: * incorporate Google Search Results as the primary content on your Property or any page on your Property;

That pretty much means "no, can't has, not yours" in this case.

Ah, Don't be evil?

[GNUALMAFUERTE]

They are being Evil. They have a perpetual obligation to keep every single feature in a time-freeze so that third parties can use them as they see fit!

Ah, wait, no they don't.

There is an assload of meta-search engines out there. Scroogle seems to be the only one that has been affected. That's because they were saving bandwidth, processor usage, and programmer's time by using the same fucking simple interface for the last 5 years. So, they've been using an old interface that existed for the SOLE PURPOSE of being compatible with shitty old IE versions .... now that google pulls it out, they bitch about it? Come on ...

Here is what I hate: Everyone is complaining about the privacy concerns with many services, but nobody stops using them! Everyone feels they have the right for every service to work they way they want it to. Guess what, you don't. You don't like google? Stop using it!. I don't like microsoft. I Don't like anything from them. So, I don't use ANYTHING FROM THEM. Not their software, nor their services, nothing. On the other hand, we have people cracking their software and complaining when they are evil. They ARE evil? stay the hell away from it.

I'm really tired of this privacy-concerns constant circle-jerking. Stop using the shit you don't like. Simple, huh?

Re:Ah, Don't be evil?

[Thundersnatch]

They are being Evil. They have a perpetual obligation to keep every single feature in a time-freeze so that third parties can use them as they see fit! Ah, wait, no they don't.

Wait a minute. This is Slashdot, and the collective hive-mind knows that Microsoft does "have a perpetual obligation to keep every single feature in a time-freeze so that third parties can use them as they see fit". So does Oracle, Apple, HP, or any $big_succesful_company. Why doesn't Google?

Re:Ah, Don't be evil?

[Adrian Lopez]

Stop using the shit you don't like. Simple, huh?

I do that sometimes, but if I followed your advice to the letter I wouldn't be able to do much at all. It's hard to find big corporations that are truly benign.

Re:Ah, Don't be evil?

[sorak]

Can you write that in an XML format so I can autopost it to my facebook, myspace, and twitter pages? </ducks>

Re:Ah, Don't be evil?

[Smauler]

Here is what I hate: Everyone is complaining about the privacy concerns with many services, but nobody stops using them! Everyone feels they have the right for every service to work they way they want it to. Guess what, you don't. You don't like google? Stop using it!. I don't like microsoft. I Don't like anything from them. So, I don't use ANYTHING FROM THEM. Not their software, nor their services, nothing. On the other hand, we have people cracking their software and complaining when they are evil. They ARE evil? stay the hell away from it.

The trouble is that other user's use of locked in privacy invading software _does_ infringe upon you. If everyone is using a standard app that decides not to communicate with anything else, you're forced to use that app or not communicate with other people. That's the point of shouting about it - the less people who use privacy invasive applications, the easier it is to use your own non-privacy invasive application and still communicate.

I do agree absolutely with your sentiments, but the aforementioned ramifications will sometimes produce a world I don't want to live in... so I complain loudly about it.

Re:Ah, Don't be evil?

[Myopic]

Yeah, sometimes the market approach works, sometimes it doesn't.

"Don't like air pollution? Stop breathing!"
"Don't like lead on your child's toys? Personally test each toy for lead yourself!"
"Don't like buying food without ingredients lists? Starve to death!"
"Don't like to be monitored by camera every time you go into public? Live your entire life inside your home!"
"Don't like loud motorcycles? Cut off your ears!"
"Don't like corrupt politicians? Move to another country!"
"Don't like paying Social Security taxes? Don't have a job!"
"Don't like being denied insurance for a pre-existing condition? Stop being a cripple!"
"Don't like having your medical records posted to the internet? Stop going to the doctor!"

Without accusing you of being one, I would like to state my exasperation with the idiocy of unregulated-market extremists. In the case of Google privacy practices, though, perhaps a market solution is better than a regulatory solution -- or perhaps not.

Wait, in fact I'd like to know your opinion about that last one. If the solution to privacy concerns with websites is to stop using the website, would you suggest a similar approach for medical records? Such as, make sure to go to a doctor with an acceptable medical-record policy? Or do you think medical records are above the threshold where legislation becomes reasonable? (I think most people are in the latter group, as am I.)

Re:Ah, Don't be evil?

[GNUALMAFUERTE]

When people say "My needs aren't being met", I always quote George Carlin on the issue: "Drop some fucking needs! Drop some fucking needs".

Re:Ah, Don't be evil?

[YodaYid]

What about Google Analytics, which spies on you even if you try to avoid actual Google websites studiously?

Need for anonymous search engine

[Enderandrew]

Is this the same company that started anonymizing search logs sooner and refused to hand over search data to the US?

Is there a reason why you NEED a more anonymous search engine? And can you trust the other party you're going through isn't logging your search inquiries?

Ultimately it comes down to who you trust more. I just don't understand why no one trusts Google when they have the cleanest track record out there.

Re:Need for anonymous search engine

[Turzyx]

I just don't understand why no one trusts Google when they have the cleanest track record out there

I think it is down to how many companies have burned their users in the past. Last.fm, Facebook etc. were all 'trusted' by users until their moral compasses went haywire. I think distrusting content providers* is healthy, if only from a security and privacy perspective.

*Yes I realise Google searches don't technically provide content

Re:Need for anonymous search engine

[geekoid]

And yet, some people will trust a site based on the pure presentation of that site not trusting google.

As if some in between site is more or less likely to sell your data.

Re:Need for anonymous search engine

[Enderandrew]

I just think Google is smarter than that. Facebook is king of the internet right now, but they are alienating their users at every turn.

They forget that Myspace was once the largest social networking company out there. Facebook could fall to second banana just as easily the second a better alternative shows up.

Google however is smart enough to realize that their entire business model is about getting users to use their products so they can serve up contextual ads. If they piss their users off, it destroys their business model.

Their April Fool's prank was to name themselves Topeka for a day. They were basically saying "we love you too". In fact, they've directly said it to me.

They had award nominations for open source contributers. I nominated Andrew Morton, not realizing at the time that Google was paying him to work on Linux full time. I got an email back from Google explaing that it made Morton ineligible for the award.

I responded saying that it was just one more reason I loved Google, because they pay guys like Andrew Morton. I literally got an email back from Google saying "We love you too."

The world is full of shitty companies that don't care about their customers/users. They just don't get it. I really think Google is the exception to the rule.

Re:Need for anonymous search engine

[causality]

world is full of shitty companies that don't care about their customers/users. They just don't get it. I really think Google is the exception to the rule.

Or they hire the very best PR people and pay them well. (/cynacism)

Re:Need for anonymous search engine

[RobVB]

I literally got an email back from Google saying "We love you too."

Is it wrong to be a little jealous?

Re:Need for anonymous search engine

[aaaaaaargh!]

I just don't understand why no one trusts Google when they have the cleanest track record out there.

This has nothing to do with trust and also has nothing to do with paranoia or wanting to hide something, as is sometimes suggested (not by you, to be fair). Sure, Google has a clean track record but the mere potential for abuse also counts, not just how likely you consider it or whether it has happened in the past. The more data a company has the less pleasant are the consequences when that data is abused. Google is one of the largest or perhaps even the largest data collectors in the world. (I guess the NSA tops them, but who knows.)

Therefore it's good not to give too much data to Google. This applies to any service or company, be it on the net or elsewhere. It's generally a good idea to use alternate services from time to time, use anonymizers, proxies, fake information, etc. The goal is not to have total anonymity (who cares about that) but to limit the bad consequences for you personally and for others if the company in question decides to become evil and sell your data to the highest bidder, put trojans, spyware, or riskware on your machine, or give away personal data to government agencies without a subpoena, etc. Sounds like reasonable common sense to me.

Re:Need for anonymous search engine

[Myopic]

Agreed. Thus far, Google has remained above the trust threshold. Nevertheless, you must recognize that Google is in the best position of any company to turn around and become the opposite kind of company. Remain vigilant, but there is no need to see phantoms where there are none.

Re:Need for anonymous search engine

[alexo]

Google however is smart enough to realize that their entire business model is about getting users to use their products so they can serve up contextual ads. If they piss their users off, it destroys their business model.

It might have been that way in the past but I think that Google already reached the "too big to care" stage.
Recently I keep having more and more problems with the way Google treats me as a user.

It started with their toolbar that, after version 3, started breaking features right and left (both toolbar features and native Firefox ones).
It continued with Google Maps that removed the ability to save favourite locations without enabling web search history.
It culminated with Google "archiving" all the discussion groups relating to their products, which proves that they are not interested in any dialogue with the "users".

Re:Need for anonymous search engine

[Al Dimond]

Facebook's moral compass never *went* haywire. It was haywire from the beginning. It had to act nice to its users for a while to gain a following, but behind the scenes Mark Zuckerberg has always been pure evil.

Kinda like Flash on the iPad

[numbsafari]

This is exactly why Apple doesn't want third-party UI systems on the iPad. They make their own business decision to improve their UI (something Google has needed for some time and something Google needs in order to survive against the likes of Bing) and now they are getting bad press because some lazy programmer can't figure out how to scrape their search results.

Intermediation is not a right. Businesses should have the right to engage with their customers without third-parties trying to intervene.

Re:Kinda like Flash on the iPad

[CannonballHead]

This is exactly why Apple doesn't want third-party UI systems on the iPad.

Because the only website I visit that might give the iPhone issues are those with Flash... all other websites work perfectly?

Re:Kinda like Flash on the iPad

[Urkki]

This is exactly why Apple doesn't want third-party UI systems on the iPad.

Because the only website I visit that might give the iPhone issues are those with Flash... all other websites work perfectly?

I'm asking because I really don't know, and I'm interested: Do Google Documents and Google Wave work perfectly on iPhone?

And yes, you're talking about websites you visit, and you might not be visiting Google Docs or Wave, no need to point that out, I'm not disputing that, but those are websites I visit regularly so I'm interested :-)

Re:Kinda like Flash on the iPad

[CannonballHead]

I'm not really sure. I don't own an iPad or iPhone. Fortunately. ;)

Google's forum doesn't give one very much hope, though.

I've heard some funny things about the mobile version of Safari, too... I don't know if it is true or not, but one person said things like tinyMCE didn't work, which meant bloggers had to put in actual coding and weren't able to use the javascript WYSIWYG editors.

Distributed privacy?

[LoudMusic]

Others have already stated this is not actually a block, but if it had been ...

I wonder if a distributed proxy would work. Run a client on your computer that puts you into a pool. Point your browser to localhost web server where it provides a search interface and submit your query. The client randomly picks another host in the network where your request is carried out and returns your results.

After a couple hundred thousand users go online the amount of mixed requests muddles the data so much that it's all useless to Google.

Re:Distributed privacy?

[gorzek]

That's pretty much what Tor does, only it can be used for any kind of traffic, not just searching.

Re:Distributed privacy?

[slyborg]

Done.
http://www.googlesharing.net/

Why should i trust scroogle more than google?

[Anonymous Coward]

I find it interesting that so many people trust scroogle more than google. Google is easily categorized, as a company which makes money by displaying ads. Scroogle however...

Re:Why should i trust scroogle more than google?

[0xdeadbeef]

Trustworthy people usually don't have such a whiny sense of entitlement. They've probably been a honeypot all along.

Re:Why should i trust scroogle more than google?

[natehoy]

I was about to post the same thing, then I saw your post...

Frankly, I'm amused that someone would distrust Google enough to use any third party to anonymize searches for them. Don't get me wrong, Google collects more than I'm comfortable with, but at least they are a well-known and pretty well-regulated entity. Whenever an unknown comes in and tells me they are going to protect me from the boogeyman for free, I start looking for the catch. When I can't find one, I get nervous. Why, exactly, does Scroogle exist? Is it to protect me from Google? Great, so now quis custodiet ipsos custodes? Do I have a reason to trust Scroogle more than I do Google?

I mean, think about it - Scroogle has access to the exact same information Google would have had you used Google. The only difference is that, instead of giving that information to a company that discloses how they use it and why they want it and is under massive international scrutiny and gets pounced on every time they appear to break their promise, you are now giving the information to a much smaller company that promises not to use it at all, but you don't know how they are making money or what their true motivation is for offering the service, nor do you know if they are keeping said promises.

How long is Scroogle maintaining your search history? Who are they going to sell it to? Where is their privacy policy? I sure as hell don't see one on their site.

Google is the devil I can see. I know precisely what Bezos et al want from me - they want to know as much about me as possible so they can show me ads they think I'll click on and buy shit. In order to tempt me to cough up said information, they offer me world-class tools to do all manner of things. For free. I get that there's a price tag attached. And there are plenty of governments and organizations out there watching to make sure Google doesn't take more from me than they promised. It's a Faustian bargain, but the devil's only agreed to take part of my soul, and can be held to his promises.

Scroogle? No, all I see is a list of names, and no promises or real claims about anything, except that their proxies will protect me from Google. What protects me from them?

Re:Why should i trust scroogle more than google?

[slyborg]

>Scroogle has access to the exact same information Google would have had you used Google

Wrong. The reason people that don't just walk around cluelessly believing everything people tell them are concerned about Google is that it continuously is trying to be the Panopticon. It's not just your search history, that's a tiny part of what they do now. It's Google Analytics, it's watching what you're watching on YouTube, their (pathetic) attempt to muscle into social networking with Buzz, the emails they have full access to via Gmail, etc. It's not because they want to "improve your search experience". They want a full profile on you so they can sell you as a package to advertisers.

Even setting aside concerns about oppressive governments getting their hands on this data, do you really want advertisers to have this data in detail? For example, Amazon already uses differential pricing. If a retailer knows you are super keen on a particular genre, they may provide you with *higher* pricing than other people because they are reasonably sure you'll pay. I don't want to hand over negotiating leverage to a party that already has way more information than I do.

Re:Why should i trust scroogle more than google?

[base3]

Amazon already uses differential pricing. If a retailer knows you are super keen on a particular genre, they may provide you with *higher* pricing than other people because they are reasonably sure you'll pay.

I was thinking maybe I'd seen this with one item (a shaving brush) but wrote it off as coincidence. I'll have to use a fresh browser, change my MAC for a new DHCP address, and try again.

Re:Why should i trust scroogle more than google?

[natehoy]

Right. I get that Google has a larger view of data. That's not my point. My point is that, for the purposes of that and all of your other searches, you have to reveal the same data (what you are searching for, what is your IP address, etc) than you would to Google.

And, let's be honest, what's to stop Scroogle from turning around a year from now and selling that data to Google outright, AND to everyone else who offers them money? They have your IP address, the dates you accessed their service, and what you searched for. That information could easily be combined with anyone's "panopticon" in this day and age of nearly-fixed IP addresses (remember, Scroogle has your actual IP address in their logs - their proxy does not protect you from them).

Scroogle has no published privacy policy I could see. They have made only one promise - to anonymize your access to Google during the course of the search.

Scroogle can't gather that full profile, but you are giving them enough information to be very saleable for use as part of a profile like that.

I'm not saying "trust Google blindly." You, as an individual, can and absolutely should make a decision on whom you trust. I know that my usage of Google is being sold to advertisers. I understand the nature of the bargain I've made with the Google Devil.

But, for your own safety, don't distrust Google then blindly turn to a lesser-known company and just trust them because they are not Google. What level of trust has Scroogle offered? What promises have they made? Who have they contracted with to make sure their willingness to keep their promises has been kept?

I'm not saying "Google Good, Scroogle Bad". I'm saying that Google is a pretty well known quantity (whether you like them or not). You have a good base of knowledge upon which to make your decision.

Scroogle is not a known quantity. Scroogle could be backed by anyone (frankly, even Google themselves), or their data could be bought out by anyone at any time. They've only promised you "proxy anonymity". Nothing more. I see no promise not to keep server logs, or resell them later. And there's no verification to ensure that they are even keeping the one promise they've made.

If you know someone on the Scroogle team, great, you have a basis for trusting them. Personally, I don't. To me, they are a bunch of names on a web page, and I have no assurances that the names even refer to real people.

Maybe they're in it for completely altruistic purposes like they claim. They probably are. But how do I know?

Re:Why should i trust scroogle more than google?

[Anonymous Coward]

I trust Google far more than Daniel Brandt, the stalker who runs Scroogle.

Same guy who posts the RL identities of wikipedia admins on his "wikipedia hivemind" site then blasts them for trying to be somewhat anonymous.

Re:Why should i trust scroogle more than google?

[Anonymous Coward]

>so now quis custodiet ipsos custodes?

Nobody thinks you're smart throwing around Latin phrases. Get your head out of your ass, you pompous simpleton windbag.

Seriously, change the header

[valadaar]

No other comment - this is simply factually wrong. Let me know when Scroogle can't even resolve Google servers, then they are truly blocked.

Re:Seriously, change the header

[Anonymous Coward]

That would require the article contributors to actually read and understand things, or else stop being disingenuous when dealing with companies they have a beef with.

I don't get it - why "scrape" at all?

[NotQuiteReal]

I don't get it - why "scrape" at all? Google has a real search API, do they not?

Re:I don't get it - why "scrape" at all?

[Anonymous Coward]

Because they were too lazy/lacked the skill. I shed no tears for an operation called Screw-gle that can no longer suckle from the teat of Goo-gle.

Re:I don't get it - why "scrape" at all?

[MartinSchou]

Doesn't that one require a key that only supports 1,000 searches per day?

That kind of thing would make Scroogle useless. And since Scroogle has no interest in paying Google for the results, they aren't going to purchase the kind of access they'd like to have.

Re:I don't get it - why "scrape" at all?

[virgilp]

Fair enough.
On the other hand.... do you think that Scroogle really gets more than 1000 searches per day? :P

Re:I don't get it - why "scrape" at all?

[Lunix Nutcase]

So basically Scroogle is a leech and they got what they deserved.

Re:I don't get it - why "scrape" at all?

[Anonymous Coward]

The key is optional and there is no limit on the number of searches... rtfm: http://code.google.com/apis/ajaxsearch/documentation/

Re:I don't get it - why "scrape" at all?

[cortesoft]

Umm well their name IS a portmanteau of 'screw' and 'google'..... so are you really surprised they are trying to leech off google?

Re:I don't get it - why "scrape" at all?

[Lunix Nutcase]

No, I just find it funny that they are butthurt because they can't, for the moment, leech off of Google anymore.

Oh man!

[Anonymous Coward]

I'm gonna miss the pink and cyan MSPAINT art!

www.google.com/ie gone -- also used by the blind.

[bpechter]

The wife used the www.google.com/ie interface for accessability reasons. It worked much better for her with her screen reader. She's totally blind. She'll miss the interface and I know there were others using it for the same reason.

Stick it to google....

[jtcampbell]

......by using a different search engine.

Oh wait - you're weren't generating any revenue for them and were actually costing them bandwidth.

That will really show them!

Re:Stick it to google....

[Anonymous Coward]

actually costing them bandwidth.

No, they weren't. It would take the same amount of bandwidth if I queried them directly. It's the same data-- it's only making a hop through Scroogle. In fact, it's probably took less bandwidth, since Scroogle probably didn't fetch stuff like Google's logo bitmap every time.

Re:Stick it to google....

[jtcampbell]

Yes, however Google makes its money by selling advertising alongside search results. As part of the scraping process, Scroogle removed these adverts. Therefore Scroogle cost Google bandwidth without giving anything back.

Personally I use Adblock, but sites are free to block people using that if they like. Which is just what Google has done in this case.

The real question is

[mr_da3m0n]

Why they are not using the actual REST API provided here?

Sounds like laziness to me, and that they are blaming google for their own shortcomings.

Google search API not applicable

[mmkkbb]

The AJAX in the name didn't tip you off? If you put a Google search box on your own webpage then searches done from that box still go directly to Google's machines and there is NO anonymizer.

Re:Google search API not applicable

[clone53421]

If the connection was performed by sockets between Scroogle’s own servers and Google’s (which is what they were doing with their SSL searches to screen-scrape the results from the old /ie interface previously) it would be the same level of anonymity as before. AJAX is just a Javascript interface to open sockets and make HTTP/HTTPS requests.

It’s just a matter of server side vs. client side. The primary reason that an AJAX search is done by your browser rather than your own webpage is because it saves your server the bandwidth and time (and saves the visitor the time too) that would have been required if it was done server-side. It could be done using server-side scripting.

Re:Google search API not applicable

[vbraga]

Why can't a server side script make a JSON (or SOAP, or ..) request and serve the results in the same way the client side script do? It could use the API and still go through a proxy, can't it?

Re:Google search API not applicable

[idontgno]

Well, except for the fact that the user submits the query to the anonymizing proxy, and the proxy (using the API) submits the query and returns the response. Which is all the webscraper approach ever did.

Maybe you're getting confused by the use of AJAX? Think anonymizing server running server-side Javascript, not client browser directly executing Javascript.

other solutions

[Anonymous Coward]

Seeks remains, http://www.seeks-project.info/site/ with a list of publicly available nodes.
Also http://www.googlesharing.net/ should do the trick.

Lazyness

[SmallFurryCreature]

Really, I hate this stuff, because I am sometimes asked to write a scraper and it is bloody stupid. Because the moment something changes, anything changes, you have to check it and check it again. And you know, many sites change their layout all the time, if for no other reason then to fix bugs. That is nothing to say of seasonal changes.

And then there is the legal side, and the ease with which to block you.

So don't fucking scrape, especially with a well developed and documented API around. Really, scraping google? WHY!

This has to be one of the most pathetic whines ever. Oh nozers, they changed their layout now your scraper is the brokerz. CRY ME A RIVER!

NSFW

[Anonymous Coward]

Do not try to go to scroogle.com if you are at work..

Too bad, I really liked Scroogle

[Anonymous Coward]

It was awesome for many reasons. The whole privacy issue was just an added benefit. It basically turned Google back into what it was once was: an easy to use search engine without all the clutter; now that Google added this stupid Everything side bar more than ever.

Google adds more and more crap which Scroogle didn't have. Plus it didn't want to be one of those stupid question answering engines or tried to "enrich" search results with self-compiled categories. Just plain web search with a minimal interface. It's baffling how hard that is to find these days.

Anyway, thanks while it lasted.

Re:www.google.com/ie gone -- also used by the blin

[mea37]

I'm legally blind (but not to the extent that I require a screen-reader) and certainly I advocate for accessability features. But, just like the /ie interface wasn't intended to be a stable screen-scraping interface for Scroogle, it wasn't intended to be an accessability feature. That's the problem with using things in unsupported ways. Sure, they may work now - but you have no assurances going forward.

I'd suggest your wife, and anyone else who finds Google's support for low-vision users lacking, contact them and start lobbying for a proper solution that they will then have proper knowledge of and reason to support.

coverproxy.com

[Anonymous Coward]

Oh well.. coverproxy.com works for me as well.

Learn about the web

[SmallFurryCreature]

Scroogle could use the api on their servers. Really, do you know so little about the web that you think AJAX can't be run server to server? Hell, you don't even need javascript for it, any scripting language can do it if it can do html requests.

No it hasn't.

[geekoid]

Google made some changes to the results that broke Scroogle.

And no, I don't think the intentionally did it.
And yes, I like the new search returns,
and no I don't work fr Google,
and yes there are other ways to do exactly what Scroogle was doing.
and no I don't like these kinds of lists,
and yes I'll stop now.

No, Google doesn't have a real search API.

[Animats]

Google once had a real search API. It was SOAP-based. But they discontinued it years ago.

Google's AJAX search API is, by design, very limited. All you can really do is create a little search widget, and perhaps add some fields of your own. The term prohibits doing much beyond that. "You are allowed to use the API only to display, and to make such uses as are necessary for You to display, Google Search Results on your Property. The API does not provide You with the ability to access, and You are not allowed to access, other underlying Google Services or data. Subject to the limitations and conditions described below, " ... "You agree that You will not, and You will not permit your users or other third parties to: (a) modify or replace the text, images, or other content of the Google Search Results, including by (i) changing the order in which the Google Search Results appear, (ii) intermixing Search Results from sources other than Google, or (iii) intermixing other content such that it appears to be part of the Google Search Results; or (b) modify, replace, obscure, or otherwise hinder the functioning of links to Google or third party websites provided in the Google Search Results. " Given those restrictions, you can't write Scroogle using that API.

We have a SiteTruth search page which uses the Google AJAX API. We're prohibited from re-ordering the entries or removing any of them. Since the whole point of SiteTruth is to re-order search results by business legitimacy, and we don't do that for the Google results, the Google results are inferior to the ones from other search engines. So our primary search page uses Yahoo/Bing.

Re:No, Google doesn't have a real search API.

[clone53421]

So... if somebody performed a search that would return illegal content (child porn, say), the Google search API terms would prohibit you from removing those listings even if you wanted to try to detect and remove such things from the results...

Sounds almost like Google is endorsing the content they provide, to me... like they just guaranteed that none of their results would contain illegal images. Because if they did, and if you knew this, then you would be lawfully required to filter them out (based on your knowledge of it)... which Google forbade.

There is no unscrapable interface

[Arancaytar]

There are only lazy coders.

Google is under no obligation to spend effort making it easy to use their site in a way not intended by them - particularly since Google provides an actual API that does not need any scraping.

It's like reading the newspaper over someone's shoulder in the train, and then complaining that they turn the page too fast to keep up.

Re:www.google.com/ie gone -- also used by the blin

[josquin00]

It appears that Google is working on accessibility features. Check out http://labs.google.com/accessible/ and see if that will work as a replacement for www.google.com/ie in a screen reader.

Site owner, Daniel Brandt, has no concern for you.

[Anonymous Coward]

I'm AC, so I suspect nobody'll ever read this, but I note that scroogle is run by Daniel Brandt, the same "privacy advocate" who thinks it's a great idea to put the names, pictures, and locations of Wikipedia admins up on his "hivemind" page. He regularly gets information from the twits at Wikipedia Review to deliver "dox" on people to him that he and/or they dislike, then posts them on his site. Way to advocate privacy. (Captcha: Salesman)

Personally, I wouldn't trust him (or any site he runs) as far as I could throw him.

IMO, scroogle can go scroo itself. :)

the new sidebar

[slothman32]

I use Google but I don't like the new sidebar.
There doesn't seem to be a way to remove it.
It doesn't appear on a browser from 2001 but does from 2005.
There aren't even any ads.
Can anyone help me remove the sidebar and get it back to the "cleaner" appearance, even with ads?
Without updating to a new browser, or using Google toolbar or something.

There may be privacy issues but as now I am not worrying about them.
I probably should but I don't.
I do like the preference keeping like the 100 items per page though.
I don't know how privacy would work without cookies.

Re:the new sidebar

[clone53421]

google.com#div(leftnav)

A Google Carol

[Anonymous Coward]

Scroogle blocked? Bah Humbugoogle!

out of date info

[Crispy Critters]

What you are saying seems to match the outdated SOAP API, not the current AJAX interface. According to Google, the old interface had a limit but the new one does not.

Sidebar blues

[Anonymous Coward]

I'm sure Google noticed a big uptick in hits from scroogle.org after they foisted^Wintroduced their new sidebar. I know that's why I started using it.

I don't mind the ads, but that sidebar is just an enormous waste of space. (And no, I don't have a wide-screen display, so it wasn't "going to waste anyway".)

Open Google

[supernothing]

For all those who can no longer use Scroogle, but are worried about their privacy while using Google's services, the Firefox plugin Google Sharing allows you to maintain anonymity on all of Google's unauthenticated services. Might be worth looking into.

Google Sharing

[bumby]

I've been using Google Sharing, http://www.googlesharing.net/ , for a while now. Works very well. As with all proxies though, it just moves the point of trust from google to the google sharing-server.

Google said they loved YOU?

[TerrenceCoggins]

OMG google is such a two-timer, they told me the same thing! My mom warned me about people that say they love you just to get into your queries. I feel like such a fool -- and they told me they only kept their relationship status as "single" because they were concerned about over-sharing too much personal info!!

Business plan....

[Anonymous Coward]

1. Build a decent search engine. Don't save your logs
2. Charge $20 per user per year to use it.
3. ...
4. Profit!

Seems solid to me.

Re:Business plan....

[base3]

Except if you're collecting payment information, you'll have identifying information of your customers, and thus will inevitably be pressured by the man to save logs--best case being only with a warrant.

Re:www.google.com/ie gone -- also used by the blin

[Anonymous Coward]

Google now has an accessibility experiment. I'm not sure how good it will be, as I don't use a screen reader, but here it is: http://www.google.com/experimental/#Axs

Re:www.google.com/ie gone -- also used by the blin

[Anonymous Coward]

Have you suggested she try Googly Eyes?

Thank you, and try the veal.

Re:www.google.com/ie gone -- also used by the blin

[Anonymous Coward]

Try http://labs.google.com/accessible/ and provide some feedback.

Does this mean scroogle is...

[croftj]

scroodled?

prettypleasoogle.com?

[Anonymous Coward]

I do hope the scroogle service comes back.
Maybe just changing their domain name will help?

I wonder if the mobile version is stable enough

[clone53421]

It’s probably far more stable (with respect to the underlying HTML that can be scraped) than their standard search results page...

http://www.google.com/m/search?q=hello+world&aq=f&oq=&aqi=g6-k0d0t0&fkt=2581&fsdt=8531&csll=&action=&gl=us&source=mog&ltoken=0d10d1b858de8&safe=active

Use google sharing instead

[Anonymous Coward]

Seems like you could use:

http://www.googlesharing.net/

"The GoogleSharing system consists of a custom proxy and a Firefox Addon. The proxy works by generating a pool of GoogleSharing "identities," each of which contains a cookie issued by Google and an arbitrary User-Agent for one of several popular browsers. The Firefox Addon watches for requests to Google services from your browser, and when enabled will transparently redirect all of them (except for things like Gmail) to a GoogleSharing proxy. There your request is stripped of all identifying information and replaced with the information from a GoogleSharing identity. "

Blocked? or Bricked?

[konohitowa]

When Apple provided software updates for the iPhone that caused jailbreak problems to those owners choosing to accept said updates, I seem to recall that being labeled as "bricked." So, for consistency, shouldn't this at least read:

Google Bricks Scroogle

Re:www.google.com/ie gone -- also used by the blin

[Anonymous Coward]

This is Google's vision-impaired offering: http://labs.google.com/accessible/

If I remember correctly, my blind friend didn't really find it all that much of an improvement; however, he isn't the most computer-literate blind guy out there (mostly just knows enough to get by, but is getting better).

Hopefully, this can become a anonymizing proxy

[Khopesh]

He's "bitching" because maintaining a scraper is intensely time-consuming (trust me, I've written many many scrapers and greasemonkey scripts, including a Scroogle userscript). The IE portal was static for years upon years (see TFA), so the scraper needed no maintenance, which is quite different from any other results display.

However, writing a minimal stripping scraper shouldn't be too hard. Here's an email I sent to Scroogle:

Even if you can no longer offer a completely cleaned search results list, serving a somewhat cleaned version shouldn't be terribly difficult; just remove JS bits and rewrite the forms and images to point to locally hosted copies at scroogle.org. This should remove the cookies and other tracking bits and shouldn't actually be too hard to do (or to maintain).

If you also pass it through an adblock subscription (or equivalent), the ads could be stripped as well, without requiring ongoing maintenance since it comes from another party. If this is too complicated, I think it fair to assume your users will be doing that.

Thanks for running the scraper for all these years!

Google says screw you to Scroogle

[Anonymous Coward]

... would have been a better title.

are liberties essential?

[dAzED1]

Squatters rights, eh? Damn right it's not their home.

Who said anything about "squatters rights?" Most illegal aliens live in apartments for which they pay rent.

A person's ethics can be effectively determined by whether they abide by the law, as it justifiably and accurately applies to them.

... Every single one of the Founding Fathers broke the law, and committed High Treason. If a law is unjust, that doesn't mean you have an obligation to submit to the law regardless; especially when you had no say in the law. And hint - almost no "citizen" has a say in the law these days either, we're deeply run by corporations.

I guarantee that if every illegal worker disappeared tomorrow, after the major upheaval and turmoil subsided the labour market would bring itself to a wage that American workers were willing to work for and employers were willing to pay.

And I guarantee that it wouldn't. The wealth of this nation has been consistently dependent upon indentured or otherwise black-market (pun not entirely intended), labor. There has not been a time in the history of the US where our agricultural and industrial power wasn't completely dependent upon slavery and immigrants. Only, back then we didn't call them "illegals." It was only once we got into the 60s and were no longer able to treat people as second-class citizens legally and openly, that we started losing our industrial and agricultural power. At one point, we were called the "breadbasket of the world," producing upwards of 80% of the grains consumed on the planet. No, indeed, we would not maintain our status if we got rid of ultra-low-wage workers.

Why does a mass flaunting of this so-called liberty by boldly transgressing the laws of the land that you fought to help protect not bother you?

Because I didn't join to protect land and/or the rights of white people; I joined to protect the concept of liberty and freedom. I joined because I believe all men are created equal, and endowed by their creator with certain inalienable rights, and that among these are life, liberty, and the pursuit of happiness. I am disillusioned from the idea that people stop being people, with those inalienable rights, based solely on whether they were born in the US, Iraq, Sudan, or even Mexico. And as a person that visits Mexico frequently, and has seen people being burned alive on the side of the road or hung from bridges, I am intimately familiar with that which a Mexican might want to escape, and why they'd want to come here. Do they fit in perfectly? No. Is it a hard road? Yes. But either we believe in essential liberties and protect them, or we give up believing in essential liberties; it's that simple.

Re:are liberties essential?

[clone53421]

Who said anything about "squatters rights?" Most illegal aliens live in apartments for which they pay rent.

I’m not talking about a building that they’re paying someone to live in. I’m talking about a country they entered illegally. This is not their home. Any residence they might take up here is illegal.

Re:are liberties essential?

[Yuuki Dasu]

I'm not talking about a building that they're paying someone to live in. I'm talking about a country they entered illegally. This is not their home. Any residence they might take up here is illegal.

And here I always thought that home was your environment: your friends, your family, the community you contribute to and are a part of. Apparently none of that has any validity, and it's instead dependent on some mystical property of the land on which you are born, or of the purity of the blood that runs through your veins, as decided by other people hundreds of years ago and upheld by yet others who may be thousands of miles away who neither you nor I will likely ever meet nor be able to hold accountable. Maybe it's just me, but that argument has little resemblance to any idea of "home" I can conjure up.

You would declare America to be the home of the children of expats who have never seen the country and never want to, while denying it to those born a few miles south of the right patch of soil who live their whole lives working to make their place in pursuit of the American dream. Furthermore, your argument ignores (or simply isn't aware of?) the 45% of illegal immigrants who are overstays, and therefore entered the country illegally, many of whom are in the process of naturalizing as citizens but whose status in the meanwhile is either in legal limbo or technically illegal.

Re:are liberties essential?

[clone53421]

And here I always thought that home was your environment: your friends, your family, the community you contribute to and are a part of.

And it is.

Uprooting yourself, invading a country when it is illegal for you to do so, and then planting yourself into a new community does not make that your new home, however, no matter how long you’ve stayed there. Until the law says that it’s legal for you to be living there, you can legally be deported at any time, and that is not your home. Home is a place where you can be confident of the security of your goods and dwelling.

Re:are liberties essential?

[dAzED1]

The welfare laws of the 60s was the first time in our history that either the general population, or the government, cared about calling someone an "illegal alien." At that point, the gov had to limit the number of people eligible for welfare, so they forced you to hate immigrants. For me, that didn't work.

Up until that time the US was the "melting pot," a culture defined by it's aims toward higher ideals, and which embraced immigrants from any culture. Our strength was in the betterment of everyone. We've squandered vast wealth the last few decades, destroying the world's economical stability, by taking our wealth, flipping it in to debt, all for the benefit of mega-corps. Along the way, we stopped being an industrial and agricultural superpower because...we axed our source of low-wage workers.

Just keep in mind that whenever you get angry about immigrants, it's because a Democrat from the 60s wanted you to (they wouldn't have gotten welfare passed otherwise).

Re:are liberties essential?

[clone53421]

So I guess the solution to it all is to repeal welfare.

Re:are liberties essential?

[dAzED1]

or, to come up with a better way of determining who gets it.

Distrowatch & Scroogle ...

[Anonymous Coward]

... I posted the Scroogle story in the Distrowatch forum earlier today but for some mysterious reason it was deleted within a few minutes. When I posted another message, asking why my 1st message had been deleted, this new message, too, was instantly deleted and on top of that my IP was blocked from accessing the DWW web site. Hmm ... Ladislav can be quite a despot at times.

Scroogle is back to normal

[Everyman]

Scroogle is back. Thanks to the help from three Scroogle users, I learned that there is a way to access that same simple interface with an extra parameter in the URL by using www.google.com/search (that param is &output=ie), instead of through the former static page www.google.com/ie without the extra parameter. It appears that both methods amount to the same thing.

I apologize for the title, "Scroogle has been blocked." It was in an old template, afterwhich the program went on to read a current text file. In the future it will read, "Scroogle is having problems with Google." We were IP blocked by Google more than once a couple years ago, but not all of our servers were blocked at the same time and we rerouted traffic, so no one noticed. We got those blocks lifted by Google within a few days.

-- Daniel Brandt, Scroogle programmer and sysadmin; president of nonprofit public charity Public Information Resarch, Inc., owner of Scroogle.org

Re:Scroogle is back to normal

[Anonymous Coward]

Scroogle is back. Thanks to the help from three Scroogle users, I learned that there is a way to access that same simple interface with an extra parameter in the URL by using www.google.com/search (that param is &output=ie), instead of through the former static page www.google.com/ie without the extra parameter. It appears that both methods amount to the same thing.

I apologize for the title, "Scroogle has been blocked." It was in an old template, afterwhich the program went on to read a current text file. In the future it will read, "Scroogle is having problems with Google." We were IP blocked by Google more than once a couple years ago, but not all of our servers were blocked at the same time and we rerouted traffic, so no one noticed. We got those blocks lifted by Google within a few days.

-- Daniel Brandt, Scroogle programmer and sysadmin; president of nonprofit public charity Public Information Resarch, Inc., owner of Scroogle.org

Two thumbs up.

Wicked work Daniel!

Thanks.

Re:Scroogle is back to normal

[Sidn]

Great to hear that!
Perhaps you could set up a status page over at scroogle.org to post service information like this. I just found your post by accident ;)

Re:Scroogle is back to normal

[soppsa]

No offense, but the bad press you gave google because you couldn't find a simple page to screen scrape was distasteful, when your entire organization is parasitic onto google. They owe you nothing, the least you owe them is the benefit of the doubt....

Append to your hosts file

[Anonymous Coward]

0.0.0.0 www.google-analytics.com
0.0.0.0 ssl.google-analytics.com
0.0.0.0 id.google.com
0.0.0.0 tpc.googlesyndication.com
0.0.0.0 partner.googleadservices.com
0.0.0.0 pagead2.googlesyndication.com
0.0.0.0 pagead.googlesyndication.com #text ads

# Google owns doubleclick now...
0.0.0.0 ad-g.doubleclick.net
0.0.0.0 googleads.g.doubleclick.net
0.0.0.0 www.yceml.net
0.0.0.0 www.doubleclick.net
0.0.0.0 ad.doubleclick.net #remove this for atomfilms problems
0.0.0.0 ads.doubleclick.com
0.0.0.0 doubleclick.net
0.0.0.0 ads.doubleclick.net
0.0.0.0 ad2.doubleclick.net
0.0.0.0 ad3.doubleclick.net
0.0.0.0 ad4.doubleclick.net
0.0.0.0 ad5.doubleclick.net
0.0.0.0 ad6.doubleclick.net
0.0.0.0 ad7.doubleclick.net
0.0.0.0 ad8.doubleclick.net
0.0.0.0 ad9.doubleclick.net
0.0.0.0 ad.ch.doubleclick.net
0.0.0.0 ad.doubleclick.com
0.0.0.0 ad.ca.doubleclick.net
0.0.0.0 ad.de.doubleclick.net
0.0.0.0 ad.fr.doubleclick.net
0.0.0.0 ad.jp.doubleclick.net
0.0.0.0 ad.linksynergy.com
0.0.0.0 ad.nl.doubleclick.net
0.0.0.0 ad.no.doubleclick.net
0.0.0.0 ad.sma.punto.net
0.0.0.0 ad.uk.doubleclick.net
0.0.0.0 ln.doubleclick.net
0.0.0.0 m.doubleclick.net
0.0.0.0 m2.doubleclick.net
0.0.0.0 ad.be.doubleclick.net
0.0.0.0 ad.sg.doubleclick.net
0.0.0.0 ad.se.doubleclick.net
0.0.0.0 iv.doubleclick.net
0.0.0.0 ad.au.doubleclick.net
0.0.0.0 ad.es.doubleclick.net
0.0.0.0 ad.ph.doubleclick.net
0.0.0.0 amazon.doubleclick.com
0.0.0.0 m3.doubleclick.net

Re:Append to your hosts file

[YodaYid]

Great idea - I already did :-)

Scroogle is Back up!

[Anonymous Coward]

Scroogle is Back up! And not blocked. Uh well, to bad google you messed up already a couple times in two weeks. I'm using https://us2.startpage.com/ from now on.

Similar situation, Slashdot blocked my login!

[noidentity]

I'm having a similar situation. Slashdot is blocking my login attempts! I used to be able to search for "LOGIN" to find where I enter my info, but now my search fails! They changed it to "Login" or something, which broke my login script. They're blocking me!!! (this is apparently what's happened to Scroogle; not blocked, just a failure on their side)

Re:www.google.com/ie gone -- also used by the blin

[animeshpathak]

I found this at http://labs.google.com/accessible/faq.html

Navigating Search Results
After Google Accessible Search was launched, many of our users sent us feedback about the results page (both Google Accessible and regular search) being difficult to navigate with screenreaders. In response, we have updated the results page in both cases to have section headers that can be used in conjunction with screenreader hotkeys to quickly skim through the page. Thus, once Google has responded to your search query, use your access technology's "move by section" keys to move between the section that displays sponsored ads and the individual results.

Maybe she can switch to using that page?

Thanks,
-A

Re:www.google.com/ie gone -- also used by the blin

[theaceoffire]

Why not use google's page for the blind?

http://labs.google.com/accessible/