This is like saying, to follow what is going on in the entertainment industry you have keep an eye on what Snooki and Kim Kardashian are upto. I am sure Jim Cameron and Peter Jackson agree with you.
In this case Google can do more. Google can open up its index to developers with unlimited API access. Allow local developers to use the latest NLP/datamining algo's to build better local indexes on top of Google data.
"I don't think a human curating process would be able to comb through as much data as quickly"
The only reason this is true is Google doesn't allow it.
Google can open up its index to developers with unlimited API access. As NLP and datamining tools get easier to use, local developers will create better local indexes than what Google can achieve. Google can then select and merge the best local algo's.
In large parts of the world, your only choice when something like this happens, is to go bankrupt.
I agree about crawling. I was talking about building the index.
Google does that better than anyone (by a long short) and still doesn't get it right. Because they are one "small" company in the large scheme of things. Look at WolframAlpha and Freebase. There are so many different ways of dealing with search and Google needs to stop believing it can figure them all out on its own. Developers should be allowed to create local branches of the index and work on them. Google should try letting go of some of the control, to allow innovation, else innovation is constrained by the number of people they can afford to throw at the problem. And most of the people they throw at the problem spend most of their time keeping things from falling apart.
Right now developers aren't allowed to "scrap" the index. This doesn't make sense in terms of allowing innovation in search. My assumption is that, a developer (with experience in data analytics) in a small town in Alaska, if given access to the (realtime) index of data Google has collected on her town, can extract value relevant to that town, better than anything Google can achieve.
There aren't too many organizations public or private left in the world, that can replicate Google's indexing capabilities.
That index has unimaginable untapped value and it needs to be opened up (unlimited api access). It's value can never be tapped by a couple hundred engineers sitting inside Google. Giving a small group within Google complete access to it, is equivalent to only giving a couple scientists access to human DNA data. I hope they find a way of opening it up before the regulators do. If there is anything worth building a platform around. more than a Google+ or an Android or TV or googledocs or chrome or youtube it is that index. I don't know how it would work but they need to start thinking about it as an end goal.
Slashdot Mirror
I'd love to see Zuckerberg pitching new features to grizzled old programmers at a design meeting.
1. People are stupid
2. People do what people around them do
source - http://edge.org/conversation/infinite-stupidity-edge-conversation-with-mark-pagel
This is like saying, to follow what is going on in the entertainment industry you have keep an eye on what Snooki and Kim Kardashian are upto. I am sure Jim Cameron and Peter Jackson agree with you.
In this case Google can do more. Google can open up its index to developers with unlimited API access. Allow local developers to use the latest NLP/datamining algo's to build better local indexes on top of Google data.
"I don't think a human curating process would be able to comb through as much data as quickly" The only reason this is true is Google doesn't allow it. Google can open up its index to developers with unlimited API access. As NLP and datamining tools get easier to use, local developers will create better local indexes than what Google can achieve. Google can then select and merge the best local algo's. In large parts of the world, your only choice when something like this happens, is to go bankrupt.
I agree about crawling. I was talking about building the index. Google does that better than anyone (by a long short) and still doesn't get it right. Because they are one "small" company in the large scheme of things. Look at WolframAlpha and Freebase. There are so many different ways of dealing with search and Google needs to stop believing it can figure them all out on its own. Developers should be allowed to create local branches of the index and work on them. Google should try letting go of some of the control, to allow innovation, else innovation is constrained by the number of people they can afford to throw at the problem. And most of the people they throw at the problem spend most of their time keeping things from falling apart. Right now developers aren't allowed to "scrap" the index. This doesn't make sense in terms of allowing innovation in search. My assumption is that, a developer (with experience in data analytics) in a small town in Alaska, if given access to the (realtime) index of data Google has collected on her town, can extract value relevant to that town, better than anything Google can achieve.
There aren't too many organizations public or private left in the world, that can replicate Google's indexing capabilities. That index has unimaginable untapped value and it needs to be opened up (unlimited api access). It's value can never be tapped by a couple hundred engineers sitting inside Google. Giving a small group within Google complete access to it, is equivalent to only giving a couple scientists access to human DNA data. I hope they find a way of opening it up before the regulators do. If there is anything worth building a platform around. more than a Google+ or an Android or TV or googledocs or chrome or youtube it is that index. I don't know how it would work but they need to start thinking about it as an end goal.