Intelligence is definately the way to go. I know a number of people are working on more intelligent search engines (including me). The problem is that we are a long way from a perfect seach engine which would require that the engine actually understand the text on the pages. I wouldn't hold my breath waiting for this to happen. We can, however, do a much better job than current search engines by looking at phrases and attempting to cluster the results more effectively. By using the results of previous searches, ie. two words frequently searched together become linked, data clustering may become very effective. Primitive techniques like this are already in effect. It will be very interesting to see how well all this stuff works out in the next year or so.
Right now, one of the major problems is sheer scale. Clustering techniques work great for smaller document collections but the sheer volume of the web makes things ugly. Another major problem is preventing "spamming" the engine. Since we are not yet ready to test our engine, we have no idea how bad this will be.
Anyway, I think search techniques will greatly improve over the next few years. This may have the potential to help the web scale more successfully.
Well, I must apologize to all those smalltalk enthusiasts out there. I'm really probably not familiar enough with the language to pass judgement on it. The only experience I've had with it was back in a Concepts of Programming Languages course. I wasn't meaning to dis smalltalk just say that C++ seemed to be up to the task for now ; )
Well, I wasn't intending to offend you. I was merely trying to point out an area which I happen to be trying to work on with intelligent agents. Since the recommendation was for a language to use with intelligent agents I thought it was particularly relevant.
For neural networks, my favorite is Haykin's Neural Networks: A Comprehensive Foundation. Be aware that it is very theoretical and goes through a lot of mathematical background and proofs.
For intelligent agents specifically there just aren't many good ones.
Um.. Speed still does matter quite a bit in many artificial intelligence applications. I guess you guys haven't tried simulating any large neural networks lately, eh?
Err... In my experience, intelligent agents spend a lot of time ripping apart HTML. While agents will not doubt be more advanced in the future, right now they basically cruise the web looking for information. I don't think Perl is appropriate as the language for the neural nets or expert systems in the agent but it is very easy to use to extract URLs, find keyword proximity, etc.
I understand the authors omittance of Perl. I was just trying to point out that it could be useful for agent development today.
Slashdot Mirror
Intelligence is definately the way to go. I know a number of people are working on more intelligent search engines (including me). The problem is that we are a long way from a perfect seach engine which would require that the engine actually understand the text on the pages. I wouldn't hold my breath waiting for this to happen. We can, however, do a much better job than current search engines by looking at phrases and attempting to cluster the results more effectively. By using the results of previous searches, ie. two words frequently searched together become linked, data clustering may become very effective. Primitive techniques like this are already in effect. It will be very interesting to see how well all this stuff works out in the next year or so.
Right now, one of the major problems is sheer scale. Clustering techniques work great for smaller document collections but the sheer volume of the web makes things ugly. Another major problem is preventing "spamming" the engine. Since we are not yet ready to test our engine, we have no idea how bad this will be.
Anyway, I think search techniques will greatly improve over the next few years. This may have the potential to help the web scale more successfully.
Sam Mulder
Well, I must apologize to all those smalltalk enthusiasts out there. I'm really probably not familiar enough with the language to pass judgement on it. The only experience I've had with it was back in a Concepts of Programming Languages course. I wasn't meaning to dis smalltalk just say that C++ seemed to be up to the task for now ; )
Sam
Well, I wasn't intending to offend you. I was merely trying to point out an area which I happen to be trying to work on with intelligent agents. Since the recommendation was for a language to use with intelligent agents I thought it was particularly relevant.
Sam
For neural networks, my favorite is Haykin's Neural Networks: A Comprehensive Foundation. Be aware that it is very theoretical and goes through a lot of mathematical background and proofs.
For intelligent agents specifically there just aren't many good ones.
Sam
Um.. Speed still does matter quite a bit in many artificial intelligence applications. I guess you guys haven't tried simulating any large neural networks lately, eh?
Sam
Err... In my experience, intelligent agents spend a lot of time ripping apart HTML. While agents will not doubt be more advanced in the future, right now they basically cruise the web looking for information. I don't think Perl is appropriate as the language for the neural nets or expert systems in the agent but it is very easy to use to extract URLs, find keyword proximity, etc.
I understand the authors omittance of Perl. I was just trying to point out that it could be useful for agent development today.
Sam