Slashdot Mirror
Google Releases Web APIs
skunkeh writes "Google have released the first beta of their Web APIs package. Used in conjunction with a free license key this SOAP based web service allows developers to execute up to 1000 automated queries a day, but is currently available for non-commercial use only. The download comes with Java and .NET code examples and includes a WSDL description for use with other SOAP supporting languages." There's also a write up about uses on Userland.
http://www.soapware.org/directory/4/services/googl eApi/implementations
At the time of posting languages catered for were for AppleScript, Frontier/Radio, Perl, Python and Visual Basic. I've written a basic implementation in PHP which has yet to be added to the list - you can find it here:
http://toys.incutio.com/php/php-google-web-api.htm l
This is a very cool toy.
The old one here - "Google to Offer API"
Don't quote me on this.
http://radio.weblogs.com/0100012/stories/2002/04/1 1/applescriptForGoogleApi.html
has some Applescript for your use
Other than being a really cool idea, this is a great tactical move from Google. On the one hand, by restricting the number of queries made to Google, they ensure that their APIs aren't misused/compromised, it also gives companies an initiative to purchase Google products and deploy this API (probably an unrestricted-query API) on their own network. Furthermore, an API such as this will easily muscle out any sniff of a competition from other search engine wannabes. Google has managed to do all this and yet be as compliant
to an Open Source initiative as possible. Remarkable.
DOH. And I hit submit before the good part:
Your program must include your license key with each query you submit to the Google Web APIs service.
I just had a go with this and some example output is displayed below. Basically you can do a search of their main web pages, request a cached page or use their spellchecker.
n d_Culture/History"}
e mpire.htm" ... "
Dave
$ java -cp googleapi.jar com.google.soap.search.GoogleAPIDemo XXmykeyXX search "british empire"
Parameters:
Client key = XXmykeyXX
Directive = search
Args = british empire
Google Search Results:
======================
{
TM = 0.117071
Q = "british empire"
CT = ""
TT = ""
CATs =
{
{SE="", FVN="Top/Regional/Europe/United_Kingdom/Society_a
}
Start Index = 1
End Index = 10
Estimated Total Results Number = 688000
Document Filtering = true
Estimate Correct = false
Rs =
{
[
URL = "http://www.btinternet.com/~britishempire/empire/
Title = "The British Empire"
Snippet = "| Introduction | Articles | Biographies | Timelines
| Discussio
n | Map Room | Armed Forces | Art
Directory Category = {SE="", FVN=""}
Directory Title = ""
Summary = ""
Cached Size = "5k"
Related information present = true
Host Name = ""
],
...
No, it means posting on Google Groups through the web interface (which already requires an account) will be possible using your single Google account as opposed to a seperate one for Google Groups and the web API.
O'Reilly has a good article here with some code as well in both Java and Perl.
http://www.oreillynet.com/cs/weblog/view/wlg/1283
No Automated Querying
You may not send automated queries of any sort to Google's system without express permission in advance from Google. Note that "sending automated queries" includes, among other things:
- using any software which sends queries to Google to determine how a website or webpage "ranks" on Google for various queries;
- "meta-searching" Google; and
- performing "offline" searches on Google.
Now, how can I use the web API?!Note that this is not in the Google Api TOS wich you must agree to before downloading the api. But in the Google Terms of Service wich you must agree to before creating a Google account needed to use the Google Api.
Still, it's fun and i'll play with it!
You are so right... this is a shocking way of doing buisness. These guys arn't even giving you root access to their servers... Shame on them.
What I find even more amazing is that you seem to expect this unrestricted acces.
Here's a copy of the write up. My machine barely made it to the site :)
Google is just the juice
Thu, Apr 11, 2002; by Dave Winer.
Good afternoon
A very quick piece today, a story, a question, an answer and a pointer.
The story -- 1995. A new release of Netscape. Can't get through to their servers. This thing is exploding. A mind bomb every minute. Wow. I love this. End of story.
The question: Can it happen again?
The answer..
Yes!
This afternoon Google opened a public SOAP 1.1 interface.
Now, from scripts, we can call Google as if it were a script running locally.
What comes back? Data.
What questions should we ask?
That's where the mind bombs will come from.
In the loop
We've been in the loop with Google, privately, for the last few weeks, so we've had a chance to play with ideas and actually have some.
Yesterday, as a tease, I put a Google Box on Weblogs.Com. Every hour it recalcs, showing the top 10 hits on Google for the term weblog. To my surprise, it changes, it's not constant. And it took me to places I didn't know about. The serendipity of queries that run for a long time. That, imho, is where the juice is in the Google API; and probably many or most of the APIs that are sure to follow; because Google is so popular.
Google hits the ball over the net, then we return the volley. Finally, once again, signs of life. Let's hope we learn from the past -- and keep the spark going -- welcoming competition and learning from it instead of snuffing it out. The intoxication of a new idea every day is too good to not want to be there once again.
Maybe the dark ages are over? I hope so.
Google is just the juice
It's happening in real time. As I write this I'm waiting for the embargo to lift. As soon as that happens, we'll start releasing new parts and samples for Radio and Frontier users that connect to Google's SOAP interface, with simple but geekish instructions for getting started.
Later today Google Boxes will start showing up on Radio weblogs, which you can follow through Weblogs.Com. You'll see SOAP developers, on all platforms, getting to work, creating and publishing the glue that turns the Internet, finally, into a fantastic scripting environment. Google is just the juice we need.
Dave Winer
Well this is what it told me:
In the future, your Google account will enable login access to all Google services, including Google Groups posting, Google AdWords, the Google Store, the Google in Your Language program, and more.
(My emphasis)
Notice the difference?
Whilst the potential of a regular Google search is large enough, when you consider the Google search modifiers, the potential becomes staggering. Imagine using the following features:
Does anyone happen to know if you can use the other sections of Google (e.g. news, images etc.)?
Is Google the best company ever or what?!
I think I speak for most when I ask if you can have your results back in the "interesting" language sets:
If they are going to limit you to only 1000 queries, I fail to see the point. It wouldn't be hard at all to write a simple API on your own to, say, a c++ class that spits out the necessary url's [like http://www.google.com/search?hl=en&q=example]
or the like, dispatch them to google port 80, and then parse the results into easily program readable data sets/results? A third party could write this sort of thing easily enough if there was demand for it. I mean, esentially the google search API isn't going to be offering anything not available in the standard forms, is it? Except their spell checker, I believe. [Which you could use via html too, actually, "Did you mean: ______" ]
---
the pen is mightier than the sword, the sword is mightier than the court, the court is mightier than the pen.
How long until slashdot offers this service?
ok then your [sic] infringing on my copyright! Could you as [sic] me next time before STEALING my comments for your own?
It's not something you have to go to google for, but it'd be nice
ummm... yeah... that's partly how they do their whole weighting thing to determine hot websites for search criteria.. without that in place, you'd have to search through tons of crap to find what you want.. I regularly use the "I feel lucky" button on google, because their algorithms manage to pull up what i'm looking for first hit..
The 1000 searches a day is very nice....I know I would never need that many (if results were unlimited anyway).
HOWEVER...you only get 10 results per search??
... sort of.. they have a XML file out there off of the main site that you can query to get the latest headlines...
i've actually used it before with a simple VB app...
email me if you want the code...
I lost my concept of community when my community lost all concept of me.
Anyone else feels that if google ever dissapears, they will become very unproductive?
[alk]
Gee, and now Google will log every search from your automated application.
And now you can create your own peer-to-peer google search which eliminates the logging altogether.
It must have most something when posted. Sonmebody posted the "correct" url in a reply, but it looks exactly the same as my post with the space between the 1s
*shrug*
Ummmmm. Ok, check this out.
/. we have an article about Google releasing their SOAP 1.1 API followed immediately by an article from a guy that set up a spambot trap on his web site, and in the margin a poll about giving spammers what they deserve. Putting 2 and 2 and 2 together, I got 4, popped open a google box and started playing.
This morning on
All I did was ask google to search for "mailto" and "@msn.com" and lo and behold, she spit back 111,000 hits - hits that contain what look like legit email addresses IN THE THREE LINE SUMMARIES.
The point is, now that google can be automated, what's to stop spammers from SOAPing their way into Google to do their harvesting? Would there be any point over what they're doing now? It might be cheaper, because you only have to run over the google results not the whole sites and since Google caches pages, you can even grab addresses from the past, somewhat.
IT ALSO DEFEATS SPAMBOT TRAPS.
Doesn't this give spammers whole new avenues to exploit?
Worse, are webmasters going to have to put a halt to Google crawls?
"Lawyers are for sucks."
- Doug McKenzie
I just submitted a request to have Ruby/Google added to the list of implementations. Until then you can find it at http://www.caliban.org/ruby/
So, I guess
public void Pigeon()
is what makes them crap on your shoulder?
www.lucernesys.comHorizon: Calendar-based personal finance
It seems there is still time to enter the Google Programming Contest and although I have neither the time nor the skill to do it, I do have an interesting idea if someone else wants to take a shot at it.
Years ago, The Hollywood Stock Exchange was a somewhat popular game (maybe it still is, but it doesn't really interest me). The general idea being that you could "Buy shares of your favorite actors, movies, and music artists and watch their values rise or fall based on the success of their careers and personal life."
It would be interesting to see a similar game based on the popularity of queries. It's clear from the Google Zeitgeist that certain search terms do gain and lose popularity on a regular basis, and for someone tapped in to mainstream culture, it may not be too hard to predict.
I suppose you could do the same thing with the other info there (Browsers, OSs, Current Events, etc.) but I don't think it would be as interesting. Although... Anime searches might be neat.
Anyhow, just an idea I'd love to see someone run with.
-Tommy
"I got a half gallon of Jack, and 2 dozen Ant Traps. I'm about to get wild." -me
Maybe now Kuro5hin can redo their Interactive Sucks-Rules-O-Meter.
Same's true if you watch enough Britcoms or other British imports (damn do they make good crime dramas!).
My 2 cents...
GTRacer
- Should be returning to England in a year or so...
Defending IP by destroying access to it? That makes sense, RIAA/MPAA. Go to the corner until you can play nice!
OK, your script parses Google's HTML output today, but what about a year from now when Google changes its output, to say, XHTML or plain text or something. How well will your script work then? Although the Google API could change tommorow like some companies' , in general APIs are more stable. I haven't looked at their API, but I'm guessing it's also easier to develop against their API, and it should be less processor- and network-intensive.
The Google Rights include rights to the following:......(3) the search results and spell checking you obtain when you use Google Web APIs.
I never thought I'd read the words "Google Rights" in a legal document, but anyway, how can Google own the rights to "spell checking".. what exactly do they own? The words that come back? The association of misspelled words to spelled words? How could you abuse that??
I must say this is incredibly cool though.. however I would much rather see a generic "Search Engine API" that isn't owned by Google, and can be implemented by anyone.
True. Plural usage of a company name is correct in British English.
Ian Macdonald, Linux sysadmin & Ruby hacker
Why is this needed? I've been using Google programatically for a while now. What does this offer that I can't use on my own?
Didn't know that. Cool. I stand corrected.
Secession is the right of all sentient beings.
Sorry, I don't remember hating Google.
-- Give him Head? Be a Beacon? :P)
(If you can't figure out how to E-Mail me, Don't.