Slashdot Mirror
Alexa Web Search Platform Released
Philipp Lenssen writes "Amazon's Alexa is releasing their search index (the same that powers the Wayback Machine) to developers via their new Alexa Web Search Platform. The Alexa framework is not for the weak of heart -- expect to learn how to use their C API, and expect to pay micro-amounts for requests and CPU cycles used -- but it also seems to be more powerful than the rival APIs from Yahoo and Google."
One dollar per CPU hour consumed. $1 per gig of storage used. $1 per 50 gigs of data processed. $1 per gig of data uploaded (if you are putting your new service up on their platform).
The diffences are major. Google's API gives access to search results or allows you to execute searches that can already be done through a browser. With G's API you can build apps like Gizoogle and Google Rank Checker. Alexa's API goes beyond allowing users to execute search queries by giving up the content within the index. This is big news for anyone interested in building their own index or accessing content for other sites.
Someone can download billions of pages for several thousand dollars then use that to build their own search engine. Another user could be to mine the web for content such as email addresses(which would be bad). Alexa's announcement is a big shift and was bound to happen. Instead of getting crumbs from Yahoo & Google, they're giving up huge chunks of juicy data.
It seems some people (especially the author of the cited article) missed some very important points:
1. You have access to more than just the index - you have access to the crawled data, which is about 300 Terabyte. So, if you want to do something with the pages, you don't have to download them, you don't have to rely, that they are there - you can use the crawled data to do whatever you want.
2. The processing does not take place on your machine, but on the provided infrastructure. There is a Web-Interface, so you can administer your account, your jobs etc. You do not download any software from Alexa. You get an account on their Linux cluster and there you can compile and run your own arbritrary applications. You are able to provide these results in form of Amazon Web Services.
So, this is much more than Google, MSN or Yahoo offer, it's hard even to compare those services. Alexa is a complete different beast, and it's a huge beast.