Slashdot Mirror
Reading Guide To AI Design & Neural Networks?
Raistlin84 writes "I'm a PhD student in theoretical physics who's recently gotten quite interested in AI design. During my high school days, I spent most of my spare time coding various stuff, so I have a good working knowledge of some application programming languages (C/C++, Pascal/Delphi, Assembler) and how a computer works internally. Recently, I was given the book On Intelligence, where Jeff Hawkins describes numerous interesting ideas on how one would actually design a brain. As I have no formal background in computer science, I would like to broaden my knowledge in the direction of neural networks, pattern recognition, etc., but don't really know where to start reading. Due to my background, I figure that the 'abstract' theory would be mostly suited for me, so I would like to ask for a few book suggestions or other directions."
Parallel Distributed Processing (both books) by Rumelhart, McClelland, and the PDP research group, 1986. "THE" classic neural network resource--and still somewhat relevant.
Due to the possibility of a robot army rising up, I refuse to help.
Artificial Intelligence: A Modern Approach by Rusell and Norvig is more or less the standard AI textbook and the book I'd suggest to get an overview of AI and its different methodologies. Mind you, it's over 1000 pages, but a very interesting read.
So my feeling is that the first people really to get anywhere with AI will either work for Google or be the neurobiologists who finally crack what is actually going on in there. If I wasn't close to retirement, and wanted to build a career in AI, I'd be looking at how mapreduce works, and the work being done building on that, rather than robotics. I'd also be looking as seriously parallel processing.
So my initial suggestion is nothing to do with conventional AI at all - look at Programming Erlang, and anything you can find about how Google does its stuff.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
http://www.opencog.org/wiki/OpenCogPrime:WikiBook
Some interesting stuff.
How we know is more important than what we know.
In my AI class, last semester, we used Stuart Russell and Peter Norvig's Artificial Intelligence A Modern Approach, 2nd Ed.. It's fairly dry, but good for theory nonetheless. If you're a physics geek, it should be right up your alley; they approach everything from a mathematical angle and then have a bit of commentary on the theory, but never seem to get to the practical uses for the theory.
If you're in the US, send me an email and I'll send you my copy. They charge an arm and a leg for these books and then buy them back for 1/10 the price. I usually don't even bother selling them back.
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
Following Books are must have for machine learning enthusiasts:
Christopher Bishop
http://research.microsoft.com/~cmbishop/prml/
Richard Duda
http://rii.ricoh.com/~stork/DHS.html
There you will get an insight how machine learning methods (like neural networks, SVM, boosting, bayes classificator) work
for general AI (not so much in direction of statistical learning as the books above, but more into higher level learning like inference rules) I can recommend published work done by
Drew McDermott
http://cs-www.cs.yale.edu/homes/dvm/
How about google?
Question: Where can I find a Reading Guide to AI Design & Neural Networks
Answer: Why do you want to AI design & Neural Networks?
Question: Because I want to learn.
Answer: Will learn AI design & neural networks make you happy
Question: Yes
There you go. Now the question is whether Slashdot beats the Turing test on this one.
An Eye for an Eye will make the whole world blind - Gandhi
Those who cannot remember the past are condemned to repeat it.
Adding another point to your feature space, I'll put in a plug for a technique called Stochastic Discrimination. It's not well known but is quite good at pattern recognition and avoids a lot of the weaknesses of neural networks such as over-training. Since it's not so well known, you have to go to the few academic papers to read up on it. Or visit the website http://kappa.math.buffalo.edu/. But it's got a very solid mathematical foundation (developed by a former math professor if mine) and isn't as "hacky" as other techniques.
Devon
On Neural Nets at least.. The only text book that I can think of offhand which is decent is Duda, Hart and Stork
Hawkins, like many others, has ripped off many of his ideas from Steve Grossberg (in this case, the ART model). Although he's not very easy to read, especially if you start much earlier than say, Ellias and Grossberg, 1975. You should also check out the work of people like Jack Cowan, Rajesh Rao, Christof Koch , Tom Poggio, David McLaughlin, Bard Ermentrout, among many, many others. I think the above names are sufficient to start a survey.
start at your school library. Search a few AI books and read a few pages.
I bet your school has access to ACM and IEEE database. You will find good AI papers there too.
If you still want to buy something, try "Machine Learning" by Tom Mitchell. I think it fits for what you are looking for (lot of theoretical stuff, with pseudo code, and tons of references).
machine learning:
http://research.microsoft.com/~cmbishop/index.htm
http://rii.ricoh.com/~stork/DHS.html
ai:
http://cs-www.cs.yale.edu/homes/dvm/
be careful before committing to a large scale neural network project. Aside from the intuition that the brain is a massively interconnected network, no one is really sure what aspect of neural network functionality is necessary for intelligence. My advice to you is to spend time coming to terms with the abstract nature of intelligence rather than coding up elaborate projects. This link is a philosophical discussion on directed behaviour which I found quite interesting (if a bit vague, which is the mark of philosophy). Also, as you become familiar with the literature, you will see many examples of algorithms which claim to model certain aspects of intelligence. These algorithms work because they have a reliable and unambiguous artificial environment from which they draw their sensory information. The problem with practical artificial intelligence is that the real world is extremely ambiguous and noisy (in the signal sense). Therefore the problem is not creating an algorithm which can emulate intelligent behaviour but solving the problem of taking the empirical information of the sensory input and producing from that data a reliable abstract representation which is easily processed by the AI algorithms (whatever they may be, neural networks, genetic programming, decision trees etc) Good luck.
prepare the survey weasels.
Jacek M. Zurada, Introduction to Artificial Neural Systems (see http://www.amazon.com/Introduction-Artificial-Neural-Systems-Zurada/dp/053495460X)
I started with:
- Laurene Fausset. Fundamentals of Neural Networks - Architectures. Algorithms, and Applications, Prentice Hall, 1994
It's pretty old, but still good to consulting about the algorithms and guide to implementation. :)
A complete guide to neural network would be:
- Simon Haykin. Neural Networks - A Comprehensive Foudation, Pretice Hall, 1999.
The best blue book I have :P
Thiago F Pappacena
Christoph Adami's Introduction to Artificial Life. He's a closet physicist and it shows. Do at least read the TOC before you dismiss it.
No folly is more costly than the folly of intolerant idealism. - Winston Churchill
If you just want to experiment with some machine learning/pattern recognition stuff without too much programming, give Weka a try. It is a suite of open source machine learning algorithms packed in a pretty usable interface.
By Tom Mitchel
http://www.ibm.com/developerworks/library/l-neural/
http://www-128.ibm.com/developerworks/library/l-neurnet/?ca=dgr-lnxw961NeuralNet
I would strongly recommend starting with a text book on Cognitive Psychology, or reading it in parallel. AI tends to overlook the fact that intelligence is a human trait, not the most efficient algorithm for solving a logic puzzle. Anderson's book can be recommended: http://bcs.worthpublishers.com/anderson6e/default.asp?s=&n=&i=&v=&o=&ns=0&uid=0&rau=0.
These might seem a little old, but are still a couple of my favorites:
Reinforcement Learning by Sutton & Barto
Machine Learning by Tom Mitchell
and if fire was any good, we would all breath fire by now ...
i would suggest Haykin's Neural Networks: A Comprehensive Foundation although you might look into a more cognitive approach...
I read and liked very much:
Neural Networks: A Comprehensive Foundation (2nd Edition) by Simon Haykin
ISBN-13: 978-0132733502
I'm a PhD neural hypernetwork studying theoretical physics that's recently gotten quite interested in human design...
.. as applied to normal computers. In this case its simply speeded up serial computation - ie the algorithm could be run serially so Programming Erlang is irrelevant. With the brain , parallel computation is *vital* to how it works - it couldn't work serially - some things MUST happen at the same time - eg different inputs to the same neuron, so studying parallel computation in ordinary computers is a complete waste of time if you want to learn how biological brains work. Its comparing apples and oranges.
you said you don't have any formal knowledge on CS. then don't think about neural networks yet, you have to build from the ground up. you need to take algorithms (doesn't matter if you're a programmer) and language theory (languages, regex, ... turing machines) at the very least. after that you can start experimenting with AI.
Ready! Steady! Go!
"Cognitive Science, an introduction to the study of mind" by Friedenberg and Silverman
Firstly it will get you thinking about the relationship between brains and minds, and how the later might be built out of the former. Secondly, Dennet is very interested in the technical aspects of all this and provides lots of suggestions for further reading.
Haven't we had a number of stories recently questioning the validity of CS degrees with lots of (usually sys admins) waffling on about how degrees are a waste of time and how anyone can pick up computer skills? Ok all you "I don't need no degree , I can do it all on my own" , show us how you've all conquered the world of AI where so many others doing BScs, MSCs and PHds degrees have failed?
What? Is that the sound of silence I can hear?
I think 'neural gas' is the area of neural networks research inspired by statistical physics. Don't know if there are any books about it, but you may find a chapter in an ANN textbook, and can certainly find papers vial Google.
Contrary to what others are suggesting, you probably aren't looking for the Russell & Norvig book, which is in fact good and almost qualifies as "the standard AI textbook". I counterrecommend it only because it's about Good Old Fashioned AI, which is interesting stuff, but completely different from what you are asking about.
Read up on neural gas, or pickup a textbook on neural networks. Be forewarned that few ANN reseachers are trying to build brains... like almost everyone else in AI, most ANN researchers are trying to build intelligent solutions to narrow problem sets, rather than trying to build general purpose intelligences.
You can find books on pattern recognition too, though ANN is only one of many approaches in that field.
Sheesh, evil *and* a jerk. -- Jade
http://www.amazon.com/Conscious-Mind-Search-Fundamental-Philosophy/dp/0195117891/ref=sr_1_1?ie=UTF8&s=books&qid=1228220299&sr=8-1
worth reading
We seem to be reading a lot of Skynet related posts these days.
I better get the drapes for the bunker finished!
Without knowing the details about where you stand with things, my advice would be to concentrate on finishing your PhD first. There's no limit to the number of distractions during that final push, but big new areas of study are usually a bad idea.
Assuming that's not an issue (nor or eventually), as a beginner in the field, you don't need to start with articles, there are books that will help for a while. But you may find quickly that you need to place yourself in one of two camps: people who want to develop artificial brains that work just like the real brain, and people who want to develop artificial intelligence that does some/all of the things real intelligence does but isn't constrained to do it the same way people do it. As a quick and dirty litmus test, would you consider your project successful if it had near-perfect memory for names and numbers (like computers do) or flawed memory for names and numbers (like people).
Beyond that, I will recommend the following book some friends of mine wrote:
Computational Explorations in Cognitive Neuroscience
This post sounds like a Turing test to me.
Could be the C2H5OH^H^H^H^H^H^H lateness of the night tho'.
http://www.cs.ualberta.ca/~sutton/book/ebook/the-book.html
Information Theory, Pattern Recognition and Neural Network by David MacKay.
The book is available online: http://www.inference.phy.cam.ac.uk/itprnn/book.html
I did my PhD in neural networks, and have read (and written) widely on the topic. My First recommendation is Chris Bishop's book "Neural Networks for Pattern Recognition". It is somewhat out of date now, but it covers all the widely known methods. Simon Haykin's book, which others have recommended, is also good, but Bishop's is more concise, and better if you don't need to know every detail of every technique. It's also worth investigating the Generative Topographic Mapping, which is not covered by either book.
As a PhD student, you should approach the topic of neural networks with caution! Be prepared to spend a lot of time training networks, re-training, adjusting ad hoc parameters, re-training. Almost all of the time, a neural network can be replaced by a standard statistical method, which will perform better and have a lower computational cost.
Sent from my Tianhe-2 (MilkyWay-2).
Does this annoy anyone else as much as me? Saying I know Assembler is like saying I know Compiler when you mean that you know C++
An Assembler is a program than converts Assembly into machine code. It is not a language.
[/rant]
They are. Ever heard of having genetic algorithms design neural-network controlled players ?
That's one non-interactive AI designing another interactive AI in order to improve a certain function.
And if your criterium is actual reproduction, let's keep in mind that no single humans are capable of even making a C64-level computer from scratch. Even a simple calculator would be pushing it too far for all but a few engineers.
The only way humans are capable of "improving their own design" according to darwin is to have lots of kids, then kill most of them.
You understand, we do expect AI's to do better than that. Because that, they can do today.
First start by Norvig's book for a general overview of Machine Learning. Then the best practical guide to implement backpropagation training for feed forward neural networks is by Le Cun and Bottou: http://leon.bottou.org/papers/lecun-98x (PDF or DjVu versions - 44 pages). However backprop will only reach interesting convergence for 2 to 3 layers NN with labeled data as input which is not the type of architectures presented in On Intelligence. To explore deep architectures such as the Hierarchical Temporal Memory introduced by Jeff Hawkins you should read recent papers on Deep Belief Networks by G. Hinton and Y. Bengio. They share interesting similarities with HTMs among which is the general architecture of the layered cortex as described by the mathematical models of the brain by Karl Friston. DBNs however lacks the temporal / sequencial aspect of HTMs. My personnal take is to use local predictive models such as 2-layers feed forward neural network trained using backprop to predict the future observed data and stack them into a deep structure similar to DBNs and HTMs.
When I studied Neural Networks in my undergrad program, we read Neural Network Design by Hagan, Bemuth, and Deale (ISBN 0971732108). At that time, we had several Physics students in the class as well, with minimal CS backgrounds. The Physics students did a great job of keeping up to speed with the concepts, since they had all of the mathematical background behind the theory.
If you want to go much further in some of the more recent theory behind pattern recognition, I could recommend Kernel Methods for Pattern Analysis by Shawe-Taylor and Christianini (ISBN 0521813972). This book is very challenging, but greatly describes the theory.
that is, its complete bullshit, but as a dream forever out of reach, it drives a lot of important and accidental discoveries, like databases or optical character recognition
so we need lots of bright minds working in AI. none of them will ever actually achieve the goal. but along the way, they will spin off fantastic new technology
so i applaud your focus, but you should be aware that anything you do of any import will be orthogonal to your goals
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
The venerable Duda & Hart book on pattern clasification: its old first edition was focused on probabilistic (bayesian) aproach, but new edition is very different, gives a broad view of pattern clasification and learning techniques, including neural networks.
Sorry, no. Genetic algorithms are optimisation algorithms that use a parallel, quasi-historical method to explore parameter space. They can not an artificial intelligence.
By Prof Penrose.
Your PhD should stand you in good stead for the math required.
Bad analogies are like waxing a monkey with a rainbow.
I had to read this for work. Very good book. You can find the previous version on Amazon for a reasonable price.
Logic is the beginning of reason, not the end of it.
I can only recommend some literature for the classic AI approach that probably isn't your primary interest, since you've mentioned connectionism. Just in case you aren't familiar with it yet, get up to date in discrete mathematics with a focus on logic and model theory first and learn some abstract algebra and topology. That's for the formal stuff that you will encounter in classic AI. Then take a look at Russell & Norvig for an overview. With your background it will be fairly easy reading and you can skip some of the chapters. If for some reason you happen to become interested in knowledge representation (my domain), I'd recommend Friedman & Halpern's "Reasoning About Knowledge" and Halpern's "Reasoning About Uncertainty". As for connectionism and pattern recognition, I suppose you could jump into the primary literature (articles, etc.) immediately, given your theoretical physics background. But are you sure that, say, string theory isn't more interesting and rewarding than neural network programming in the long run?
Recent stuff I ran across that seemed very interesting: http://www.youtube.com/watch?v=AyzOUbkUf3M
Beyond that, Neural Networks are a dead field; they're cool, but can't really do much with them.
"If anything can go wrong, it will." - Murphy
You should read about Igor Aleksander's WISARD project although you might be better off reading one of his papers rather than spending on a book http://www.iis.ee.ic.ac.uk/aleksander/publication.html
I'd recommend "The Age Of Spiritual Machines: When Computers Exceed Human Intelligence" by Ray Kurzweil. The first chapter is a bit dense, but it really picks up from there. It touches on a lot of highly technical issues, such as artificial intelligence and quantum computing, without being overtly technical itself. It would be a good launch-point into some heavier reading, is it contains a very extensive bibliography and recommended reading list.
Penguin has an excerpt from Chapter 6: Building New Brains
UTF-8: There and Back Again
You should read Minsky's Society of Mind and follow on (not necessarily that it has the answers, but that it is a very different outlook).
The recent paper by Leslie Valiant (not his book) has some very intriguing speculations about computation with random networks and sparse, low weight coding. This would make perfect sense to a physics oriented person.
Pentti O Haikonen: The Cognitive Approach to Conscious Machines. Imprint Academic, UK 2003 Easy reading, no mathematics here, lots of ideas. Based on cognitive sciences, engineer's insights and common sense. Pentti O Haikonen: Robot Brains; circuits and systems for conscious machines.Wiley and Sons, UK 2007 Haikonen envisions autonomous robots that perceive and understand the world directly, acting in it in a natural human-like way without the need of programs and numerical representation of information. By developing higher-level cognitive functions through the power of artificial associative neuron architectures, the author approaches the issues of machine consciousness.
However, to make this work you need a very efficient inter-process messaging prototcol that allows multiple virtual neurons to send messages to another virtual neuron. Languages like Erlang are optimised for doing this.
If I wanted to replicate the "brain" of a sea slug, which has (I believe) about 26 neurons, it would be much easier and cheaper to do this on a standard computer running 26 pseudo-parallel processes, than on 26 computers each imitating a single neuron, with a huiige number of potential interconnects.
As to what those pseudo-parallel processes look like, they have to respond every time a message is received (equivalent to a pulse from another neuron) by doing a calculation based on state history and then deciding when next to send an output to the destination process. For small numbers of neurons this is a manageable programming task; for large numbers, like brains with billions of neurons it is not.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
iRobot by Isaac Asimov is definitely the book you need. It has the three laws so it should help get you started.
As has been already mentioned, Artificial Intelligence: A Modern Approach by Rusell and Norvig (or AIMA) is essentially the only choice for serious study of AI. Your relative algorithmic naivite will make it a bit of a struggle, but there is a long history of smart physicists moving into AI.
Unfortunately, there is also a long history of smart outsiders getting trapped in "junk AI". These are the branches of AI that exist more because the metaphor is compelling rather that the results or prospects. These include: Neural Networks, Genetic Algorithms, Ant Colony Optimization, etc. I won't claim there is no good work in these areas, but there is too much fascination with the techniques themselves over the results, such that research constantly "solves" problems that would be done better with other techniques, but yet are somehow "interesting" because a neural net does it. The mainstream of AI is mystified why anyone would be interested in a technique that works 80% as well as the state of the art just because some guy in the 50s attached the word "neural" to it.
If you want to simulate brains, you should study neuroscience. If you want to know what's going on in mainstream AI, you should bone up on probability, statistics, and linear algebra (if you're the right kind of physicist, you already have the math you need).
Before you mod me as flamebait, please note that I do know what I'm talking about. My PhD is in AI and I'm professor in a CS department in an undergraduate engineering school, where I teach AI and Robotics. I was once the maintainer of the comp.ai FAQ, and I have published several papers in neural networks and genetic algorithms.
If it starts with 'Cognitive ...' don't read it.
Ignore all info from psychology, they don't know anything.
Neural networks are a dead end. Don't waste your time.
Philosophy isn't a good starting point, but you can read from Marvin Minsky.
In my experience physicist don't have the right mindset to understand the issues involved, but Douglas Hofstadter is a fine exception (he is a computer scientist also).
In my opinion only computer scientist have the right mindset to understand the issues surrounding intelligence. The issues are (in short) information and organization.
As a rule, don't try to recreate or emulate intelligence, but start to solve the issues that hold you back to implement it. This will also make you understand the problems involved.
J.
Depends on where you want to go.
I was an undergraduate in Electrical Engineering and Physics and then went straight to a PhD in Biophysics where I ended up working in a behavioral neurobiology lab with all manner of invertebrates where I designed tools to record from their nervous systems and to analyze the data.
Essentially, there's nothing we don't know about the brain. Electrical Engineers have been developing the tools to create computers and feedback sensory systems for over a hundred years (if not longer). If you want to get into this world, I'd recommend looking at control theory. Neural networks like the human brain are "simply" massively paralleled adaptive feedback control systems.
Insects are amazing creatures to study. They represent many highly specialized systems that boil down to specialized sensory-behavior feedback loops with dashes of memory and categorical perception thrown in. They are easy to study in most cases and there are fewer confounding issues than when studying larger brains.
Treating AI as some sort of "other" discipline than adaptive feedback controls is indicative of a wrong idea about the field. AI is object abstraction from multiple sensory modalities and heuristic learning algorithms and state machines which control behavioral outputs.
It's not something different than building a cruise control for a car or a web browser for a PC. These are just dumb forms of AI with limited adaptability.
I prefer the use of the term "non-biological intelligence" instead of "artificial intelligence."
Gödel, Escher, Bach: an Eternal Golden Braid by Douglas Hofstadter.
Although not recent (written in 70's) and not an official AI text, it offers some good insights on how minds and computers work.
MIT has been publishing their classes online for some time now on OpenCourseWare. I suggest you check out 6.034 Artificial Intelligence from the Fall 2006 semester.
They were using Patrick Winston's Artificial Intelligence. Might be worth a look.
Is a company that started after Hawkings wrote the book. If you are interested in following more in what he described then that is where to go. Neural Networks tend to be fairly good at dealing directly with real world stimuli. Bayesian networks tend to work better for higher cognition.
If your looking for neural network stuff though look up the Ersatz brain project.
Me degree is in AI, so I've come across quite a few books on the subject. I have to say that I didn't find Rusell and Norvig all that useful. For pattern recognition using statistical methods or multi layered perceptrons (neural networks) Machine Learning by Tom Mitchell is probably better. I would also recommend An Introduction to Genetic Algorithms (Complex Adaptive Systems) by M Mitchell for an interesting approach to neural network training.
"Society of Mind" by Minsky...
It seems you are confusing the broader field of artificial intelligence with the subfield of machine learning. One good book in on this subject is Machine Learning by Tom M. Mitchell. AIMA by Russel and Norvig discussed in other replies is a good overview book but contains a lot about searching and planning which is good to know but not rely necessary if you want to build a âoebrainâ. If you plan on designing neural networks that simulate biologically plausible processes I would recommend reading about Spiking Neural Networks. A good tool for this is the NEural Simulation Tool (NEST) available free of charge from http://www.nest-initiative.org/
And rutabagas. Does anybody actually EAT those gawdawful things? Normally I'm pro-vegetable but rutabegas really piss me off. Oh, I'm SO angry now.
Back in the late 1970s-early 1980s, Byte Magazine had several really good primer articles on AI, expert systems, and neural nets. I spent many an hour reading them back in my university days. They even had an entire issue dedicated to artificial intelligence. They had articles like The Brains of Men and Machines, A Model of the Brain for Robot Control.
In one of the articles they look at the structure of the brain and nervous system in terms of motor control. A lot of processing gets done outside of the brain. They talked about feedback loops: how muscle stretch sensory neurons would send a signal up to the brain, but also across to the neurons triggering muscle contraction. There are chains of feedback loops that are linked together and talking to each other all along the signal path from brain to muscle. Just as an example, in order to bend your elbow the bicep muscle gets triggered to contract, but simultaneously the tricep muscles get commanded to relax. The rate of contraction (and relaxation) of the muscles is adjusted on the fly to compensate for the varying strain resulting from the forearm's weight and momentum. Very good articles. I wish I could find them online.
When our name is on the back of your car, we're behind you all the way!
David MacKay's Book Information Theory, Inference, and Learning Algorithms. It written at a high level and is an extremely difficult read (MacKay puts more information on a page than most do in half a dozen).
Don't enter the PDP club. This book is causing AI research a delay of 10 years. Read instead the book that they accuse of having done so: Perceptrons by Marvin Minsky, and anything else by him. The old Principles of Neurodynamics by Rosenblatt is also interesting, but not good if you just want to learn the thing.
That is for neural networks... There is also a famous NN book by Simon Hayking, but I am not a big fan.
As for "building a brain", that is something else. You should look for Russel & Norvig, James Anderson, H Simon, A Newell, Zenon Pylyshyn, Douglas Hofstadter... Look for the so-called "cognitive architectures", ACT-R, SOAR... Some of them use neural networks and other "numeric" machine learning techniques inside their systems.
Ah, and do study statistics, it's a must for contemporary AI research. Look for Markov Decision Processes, MDPs, there is a famous book about that by M. L. Putterman. Look also for Reinforcement Learning (the Sutton and Barto book) and Dynamic Programming (the Bellman stuff). In multi-agent systems research that is a big thing right now.
Coming back to the connectionist, pattern-recognition domain, I do like Hinton, but only when he is not fighting against the "competition"...
Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8104 Great reference for me when I was doing my senior research in CS with neural networks.
Like the other people here mentioned, Stuart Russell and Peter Norvig's Artificial Intelligence A Modern Approach, is the text book most intro AI classes use. Another great book is Machine Learning, Tom Mitchell, which is used at a few of the top universities. That's really heavy on the theory. and finally there is The Elements of Statistical Learning, Hastie, Tibshirani, Friedman. I've run across these two books multiple times in the class room and outside in the industry. I've also seen some professors recommend the bishop book above, and duda.
but i'd have to agree with some other people here in that the book On Intelligence is really a different form of AI, in that it tries to model the brain very differently. Traditional AI and neural networks are *vastly* different than what the Hawkins presents. Neural Networks are usually said to be _inspired_ by the brain and is nothing like how it really works. As a few of the other people have mentioned, this book is probably closer to cognitive science and there is a whole different field of research in how the brain works and how to possibly model it.
If you're interested more in this book, I believe that the author had at one point created a small company around implementing it's ideas.
GÃdel, Escher, Bach (http://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach) essential reading I would have thought. It won't give you specifics about neural networks (Rumelhart's book suggested above should give you the basics). It will give you an overall understanding about what is interesting to AI and you will have a good time reading it.
Biased by the fact that I work in an Artificial Life and adaptive systems research group. I would send you away from the classical AI approaches. I have no books in mind, but with your background in Physics I would advice you to look into information theoretical approaches to cognition, they can be quite controversial but I think quite promising.
And more from my personal interests, look into developmental systems, as in does a "brain" develop.
I know I am a bit vague, but so is the research in that field.
(oh for a couple of other buzz words : cohonen maps, and associative memory, for memory models)
Totally agree with the article, btw., excellent link.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
The term AI is so nebulous that it doesn't really mean much of anything. It's more of a functional goal (computer-based human-like ability) than anything more concrete, and as anything that may fall under that general umbrella does become better understood or more concrete, then it tends to be no longer regarded as part of AI (e.g. machine learning, expert systems, speech recognition).
It's also worth noting that natural intelligence is also a rather nebulous concept - you'll find many definitions offered (e.e. an ability to learn and generalize), but almost any definition is doing to be met by some people with "well, that's PART of it...".
The larger problem is that the goal of AI really isn't artificial intelligence(!) - it's an artificial brain/mind. The emphasis on intelligence/cognition is perhaps why so many attempts to achieve AI have failed - because there's no theory of mind or overall brain architecture behind them. Artificial cognition itself is anyways arguably an already solved problem via general symbolic problem solvers like Allen Newell's SOAR, but the ability to manipulate and further refine knowledge isn't of much use if you can't aquire it in the first place. Pairing a cognition capability with an artificially aquired body of real-world knowledge, as done by Cyc, is no better because it is brittle and inflexible - an artifical brain needs to be able to distill it's own ever-changing categories and knowledge out of raw analog sensory input, and this means that symbolic approaches (fuzzy or not) are at best only a small part of the solution (more likley no part at all).
What you really need to do before getting into any technologies that may help implement "AI" is to define what it is that you want to implement. If you want to create an artificial brain, then you will need to figure out what a brain is and how to decompose it into functional blocks - an architecture. Biology texts may be of more use than "AI" ones, as even AI luminaries such as Marvin Minsky have produced laughably simple "theories of the mind". Hawkin's "On Intelligence" is a better start than many, but he also is really focusing on the problem of intelligence, but at least addressing the issue of how to mate this to the perceptual system.
It's only once you know what you want to build (i.e. the hard part of figuring out how the brain works) that the issue of how to implement it comes up, and what existing technologies may or may not be of use (Neural nets, HTMs, SVMs, etc). If you start to implement before having a fairly complete theory of the brain/mind, then your results will only be as good as your goals.
I read "On Intelligence", too. While Hawkins has some interesting thoughts, I was less than inspired. Probably because I read John Searle's "Rediscovery of the Mind" first. Actually, most of Searle's work, as well as the work of Roger Penrose has led me to the conclusion that the Strong AI tract is missing the boat. The Strong AI proponents, like Hawkins, believe that if we build a sufficiently complex artificial neural network we will necessarily get intelligence. Searle and Penrose have very convincing arguments to suggest that this is not the right path to artificial intelligence.
Realistically, how could one build an artificial brain without first understanding how the real one works? And I don't mean how neural networks function; I mean how the configuration of neural networks in the brain (and whatever other relevant structures and processes that might be necessary) accomplish the feat of intelligence. We still do not have a scientific theory for what causes intelligence. Without that, anything we build will just be a bigger artificial neural network.
Also, the thing that Strong AI'ers always seem to forget... An artificial neural net only exhibits intelligence by virtue of some human brain that interprets the inputs and outputs of the system to decide whether the results match expectation (i.e. it takes "real" intelligence to determine when artificial intelligence has occured). Contrast this with the way your brain works and how you recognize intelligence from within, then you'll realize just how far from producing artificial brains we really are...
I'm not saying that artificial intelligence is impossible, and neither is Searle (Penrose is still on the fence). I'm just saying, don't think you can slap a bunch of artificial neurons together and expect intelligence to happen.
I've never understood the draw behind "neural networks" ... it's a really cool-sounding term for an otherwise not-so-exciting algorithm.
A neural network lets you determine an approximation to a function for which there may be no closed-form expression. It's basically a piece-wise linear approximation with heuristic edge-waiting, where the edge weights are "trained" by inputting numerous samples to the "neural network".
by Douglas Hofstadter and the Fluid Analogies Research Group
http://www.wcdd.com/dd/brain/reviews/fluid.html
I recommend a book called Blondie24. It is an easy read that gives a pretty good overview of several AI techniques, including neural networks; it particularly focuses on using evolutionary algorithms to "evolve" neural networks.
I remember back in the 90s, Al Franken had some funny bits on SNL. I'm not talking about Stewart Smalley (which was mostly a waste, IMO, though the Michael Jordan episode was funny). I'm thinking of a weekend update sketch on the Mitt Romney vs Ted Kennedy election. There were some fake ads ("Ted Kennedy: he's a big, fat drunk", "Brigham Young believed a man should have many wives, and so does Mitt Romney. Ted Kennedy: One wife at a time.").
Now he's just a really angry, really weird looking guy. Nobody likes Norm Coleman, but nobody likes Al Franken either (that third party guy got 15%+). In a year when any democrat who can read off a teleprompter could win big, he didn't. Honestly, Minnesota should admit that their candidates suck and leave the seat vacant for 2 years.
Well, this is an easy one. You should read books on how the brain is built. I would read "On Intelligence" by Jeff Hawkins to start. The idea is that you want to see how the brain functions so that we can emulate it. That means you need to understand the functions of both brain hemispheres. The left generally handles linear sequential pattern stream processing while the right handles visual simultaneous pattern stream processing. In short, the left handles language, the right images. The brain functions the way it does because of a complex interplay between these two hemispheres, each assisting the other. For example, though the left hemisphere generates language, the right structures it. So a person whose right hemisphere has been removed can still form language but they ramble and do not make any points. The problem with AI so far is that it only attempts to replicate the functions of the left hemisphere--let alone the right or the interactions between them via the corpus callosum. If you want information on this, I suggest you look at a little open source project called the Godwhale
Excellent textbook on Intelligence: Marvin Minsky "The society of Mind"
Check out www.numenta.com
We could actually build a brain with silicon if motivated, but I think it will prove lots cheaper to use biology and re-purpose existing designs.
-- Programming with boost is like building a house with lego. It's a cool but I wouldn't want to live in it
I found "Mind Matters" by James P. Hogan a broad but well disciplined overview.
I'm a PhD student in theoretical physics ...
I can imagine an announcement some time in future, by a person in extreme denial: ..."
"... This is not some agent provocateur or highly trained assassin we are discussing. Raistlin84 is a theoretical physicist who had hardly earned the distinction of his Ph.D. at the time of the incident
Way to go, Raistlin84! And if you see that person, tell him I said [noise error].
... I figure that the 'abstract' theory would be mostly suited for me ...
Studious type. Man of few words, aren't you?
Artificial intelligence (2nd ed.) structures and strategies for complex problem-solving
George F. Luger - William A. Stubblefield - Univ. of New Mexico, Albuquerque
1993 ISBN:0-8053-4780-1
I have this in hardcover and I used it for reference a lot.
Come on I need Karma!
*Satan Laughs As You Eternally Rot*
If you want an easy to read, well written introduction to neural networks try Phill Picton's, "Neural Networks", 2nd edition, ISBN 0-333-80287-X. It covers lots of types of neural nets, their applications and has very clear well written examples. Its also less than 200 pages in total.
For more general interest you might want to read some of the "classic" literature on the subject such as Ashby's "Design for a Brain" and "Introduction to Cybernetics", Hebb's "The Organization of Behaviour" or McCulloch Pitts 1943 paper "A logical calculus of the ideas imminent in nervous activity" or Rosenblatt's 1958 paper "The Perceptron: A probabilistic model of information storage and organisation in the brain"
Also a good (and easy) read is Braitenberg's "Vehicles: Experiments in synthetic pyschology" which is somewhat robotics oriented but demonstrates how very simple concepts can scale up to produce seemingly intelligent behaviours.
If you want to understand the mathematical basis of neural nets read "Perceptrons" by Minsky and Papert. (make sure its the revised edition not the 1960s original).
Lots of people have already mentioned Russel and Norvig which is basically the standard AI text book. I actually found their explanation of neural nets wasn't great and the book mainly focuses on machine learning techniques. They have a big code library associated with the book but its very tightly integrated and quite hard to actually see whats going on in any given example. I found "Artificial Intelligence" 2nd edition by Rich and Knight (ISBN 0-07-100894-2 or 0-07-052263-4) much more helpful.
Forget this and finish your thesis. You don't want to be a grad student any longer than you have to.
There are probably at least three things you need to consider if you are really interested in this:
- Computing Machinery and Intelligence - Alan Turing
This is the seminal paper in AI, and argues for the case of "strong" AI.
- Goedel, Escher, Bach: An Eternal Golden Braid - Douglas Hofstadter
Covers absolutely everything under the sun. This book explores most questions you could have about AI.
- AI: A Modern Approach - Norvig et. al
As someone suggested, this is an excellent book and covers all of the computational approaches in AI. It is my bible.
At Tech, as at most schools, we can take courses from any department. Whenever I'm curious about something I sign up to audit it, download the prof's book/notes, show up to enough courses to satiate my interest, and stop going when I get too bored.
Secondly, the whole point of learning a message based language like Erlang is that you do not need to worry about race conditions and deadlocking. Obviously you have to worry about deadly embrace...but that is a design issue.
Thirdly, I don't see the point of your argument about simulations not being exact representations of physical processes. If you cannot tell, from the I and the O, whether or not it is a simulation, who cares? One of my cars has a physical accelerator control with a cable to the butterfly, one has a variable resistance transducer feeding into the EMS. Without looking, you could not tell which was which.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
In my graduate level Neural Networks class (taught by a professor with MANY papers on the subject) we use "Neural Networks: A Comprehensive Foundation" The author is Simon Haykin. More of an engineering approach to Neural Networks, but it is a good text none the less. I can also be bought as a digital edition to save you some money
So put that in your pipe and grep it
Start with the basics. The entire field of AI wouldn't exist w/o Claude Shannon. His 1948 paper made this whole field of study something rigorous.
google for this:
Shannon 1948 - The Mathematical Theory of Communication
BTW - Jeff Hawkins is way behind the times. His whole concept of an HTM is nothing more than a well parametrized Helmholtz machine. /posting as AC so "they" won't get me.
Get Valentino Braitenberg's book "Vehicles: Experiments in Synthetic Psychology". Vehicles are his term for robots. It starts very simple and builds to great stuff. It's a great book which is amazingly short though it took me months to read it because I'd read a page or two then spend a few days thinking about it. I can't recommend it enough.
Before wasting billions on AI, wilfrid sellers, "Empiricism and the Philosophy of Mind" should be required reading by anyone in AI, brain research, language, philosophy, accounting, CEO's, CIO's, your dog.
Living in Chile
"AI Techniques for Game Programming" by Mat Buckland.
The book has really good demo code, practical applications and covers genetic evolution of neural network controlled "bots" that can be used in games.
I have looked at a lot of other books on the topic and they all are either math books with no practical applications, or they are guide books that cover a lot, but have no details.
"was given the book"
How do you get people to give you books? I keep trying, but they still make me pay for them.
I don't know if this is still the case, but 10+ years ago California State University Stanislaus had a very well respected AI/NN program. Maybe look around their site or email them to get some suggestions?
Bark less. Wag more.
FWIW, my current top 3 books are:
While not course text books, I think they should be recommended reading for software development/IT students.
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
I would recommend AI Techniques for Game Programming by Matt Buckland. Yes it has Game in the title but the examples from the book aren't just for games. It covers simple evolving neural networks as well as pattern recognition, with explanations to understand the concepts as well as implement them in c++. Good luck!
Is to *not* name your A.I " The Turk" and to not have it specialize in chess.
A great textbook for Neural Networks is the one by Mehrotra, Mohan and Ranka.
kdawson,
It's wonderful to see posts like this, your post got me interested in AI as well. I mean to start learning it step-by-step.
I would appreciate if you could post your research results on the AI books & surrounding academic material. If it's not too much to ask- please post your conclusions in a comment on this article, I've added to myself a Google calendar reminder for 1 month to review the comments here.
You are welcome to contact me directly as well, maxim.veksler+slashdot@gmail.com
Maxim.
AI can work from one of two "ends". I think it is clear the brain is built with neurons. So you might think to study neural networks. But that is like saying computers are built with many interconnected transistors and I want to design a web site so I'll study the physics of semiconductors. No, if your goal is web site yu ned to work at a higher level of abstraction, maybe at the level of PHP or Java Script. Likewise the brain almost certainly organizes networks of neurons into higher level stuctures and those structures into even higher levels. By the time you get to human level language ability you don't even need to understand much about the lowest layers.
That said, the only way I think we will ever get to a true AI i to have people work work with and understand each of the layers. It is likely far to complex for one person to understand it all. So pick a field. Do you like neural networks or computational linguistics or maybe Kinematics?
My interest is how you can use the low level primitives like neural networks to build higher level abstractions. So I think I'd call that the "middle ware". I think people have been working AI from both ends for decades but I'd like to see the ends meet.
One clue are brain waves. I think we store infomation is a rings of neural networks that use feedback. Years ago in the 50's someone seriously suggesting using "moon bounce" as a kind of computer memory. You would encode a signal and transmit it via radio to the moon. then after a few seconds it bounced back to earth and is continously retransmitted as a loop. One can store ove 2 second of dat in the free space between the earth and moon. I think the brain does this. You can store a lot of data n a delay loop. All of short term memory is in there and this holds "stuff" that might end up in longer term memory or other loops.
So if you have a theoretical background and are good at math. Work out how these interconnected feedback loops could work.. I'm pretty sure these are the structures that set just above the neural networks.
The next step is to see how you can build classical AI using loops.
Russel/Norvig has already been mentioned many times above and if you can follow that one, you should be fine regarding basic Computer Science. While you wonder where to go next, take a detour into digital signal processing so that your machines will end up having sensors tailor made for the job they are supposed to do as well as being able to easily transform a dataset into some format that actually makes sense.
send + more == money?
I think an interesting area for you (since you like theoretical models) is learning theory. References:
An Introduction to Computational Learning Theory. Kearns & Vazirani. It's a "baby introduction" easy to read and understand.
Neural Network Learning: Theoretical Foundations. Anthony & Bartlett. I highly recommend this book, written by statisticians, is fairly comprehensive on proving learning bounds for NNs.
A Probabilistic Theory of Pattern Recognition. L. Devroye, L. GyÃrfi, G. Lugosi.
There's an obscure old book by Edward DeBono (now a creativity and problem-solving guru) called "Mechanism of Mind" that I found fascinating. It's very much non-academic and non-computer oriented, but it has an interesting take on pattern recognition and decision making in the human mind (as opposed to the human brain). If you liked "On Intelligence", this is a similar kind of thing, but at a much more abstract level, and without, well... any real academic basis. I think it is out of print now, but maybe it can be found online somewhere.
If you are interested in very general epistemological ideas, I would also recommend Ayn Rand's "Introduction to Objectivist Epistemology" as an outside the box choice for exploring a theory of concept formation that rings true, at least to me, and that I suspect has some application to AI in a way that I don't think has been explored very much.
Insightful and funny are really the same thing, except one has a punch line.
before I'd ever read anything about computer science neural networks, I read steve grands book about making a robot chimp, with a very basic explanation of how neurons work. On that basis I wrote Democracy, a computer game based on a neural network.
Obviously you will learn a hell of a lot from good books, but there's something to be said for just jumping in and coding it 'your way', to see what happens. It will likely make the (somewhat dry) text books on the subject seem much more relevant when you have already tried to code a similar system without a pre-planned idea of how to go about it.
Just an idea.
DRM-free indie games for the PC and Mac: Positech Games
One of the arguments made in the "On Intelligence" book concerned the inadequacy of using the Von Nuemann Architecture with the standard fetch, decode, execute, store paradigm, which is at the heart of all modern computing, to construct a true human like intelligence. You either have to build hardware that approximates the human brain (i.e. lots of nueron like devices) OR you have to simulate such a device on the above mentioned von nuemann type computers (which are inefficient at simulating a brain and require uber computing power to simulate even a much less complicated than human brain). In other words, this is not an area where one can do any useful work that hasn't already been done in isolation on consumer computing hardware.
If you want to understand the theory of neuroscience as opposed to AI techniques that think they know how the brain works, the definitve book (which is recommended reading in my Computational neuroscience course) is Dayan and Abbott's Theoretical Neuroscience : Computational and theroetical modelling of neural systems.
I agree that reading about AI won't tell you much about how the brain works. You should decide whether your goal is really to learn how the brain works, or how a neural network or a machine learning algorithm works. And keep in mind that there's a lot of crap out there.
For the former, I'd recommend Principles of Neural Science (Kandel), Theoretical Neuroscience (Dayan and Abbott) and Methods in Neuronal Modeling (Koch and Segev).
For a nice combination of both, and with the physics background in mind, I'd highly recommend Biology and Computation: A Physicist's Choice (Gottfreund and Toulouse).
The only path to a real AI is Lisp, ARM, and a room full of babies and various input devices. The babies stay in the room until they're fifty. Then we recalibrate and run the program again.
I know it may seem surprising, but that book is the definitive source to neural networks, written by the developer of back prorogation network training (the definitive training mechanism for NN).
All other things, don't really matter quite as much, as they are more "case study" and specific. Not really informative in the classical sense, or in the general sense.
The only thing you have to know is proofing your inputs is the entire joke.
NN in general is just a nonlinear black box modeling system (combinations of S-cruves more often then not). If you want really in depth understanding of these concepts, avoid NN classes at first, and go toward control system and decision sciences. I assume since your a PHD candidate you can probably breeze by the bitter beginnings, and get to optimal control quickly, and then to fanciful things such as adaptive systems, and nonlinear systems, and their control theories.
This will give you a damn good basis on the real world of modeling and control. Most importantly, the mathematical and physical aspects of what you are doing.
None of this will really help you with AI; last I heard NN's were no longer being seen as a feasible solution.
I did some extensive undergraduate research at Rutgers University, New Brunswick utilizing NN for pattern recognition and modeling the human visual system. They have a whole laboratory as part of the Biomedical Engineering Dept dedicated to this area of study. Google EVANGELIA MICHELI-TZANAKOU and you should be able to find her email. She is an expert in this field and has held seats in the IEEE society for her knowledge in this area. She was my mentor for my senior design projec. She has a harsh personality but would be more than willing to point you in the right direction. Good luck!
I'm surprised this hasn't been mentioned yet but Kenneth Stanley did some interesting work at the University of Texas on NEAT, Neuro Evolution of Augmenting Topologies. He and others have expanded this in several directions including things like Compositional Pattern Producing Networks (CPPNs) that can be joined together into a larger network.
I actually just signed up for Safari to read the chapter in AI Techniques for Game Programming on NEAT and some other approaches.
I also found that the books by Gerald Edelman (Neural Darwinism, A Universe of Consciousness, Bright Air, Brilliant Fire, et al) provide a refreshing look at the biological foundations.
...to lure the future 15 to 25 year old Male freedom fighters into trusting thus permitting them to infiltrate their ranks. DUH!
Over-the-top Response Guy! Giving "Over-the-Top Responses" since 1970.
"I have deeply regretted that I did not proceed far enough at least to understand something of the great leading principles of mathematics, for men thus endowed seem to have an extra sense." --- Charles Darwin
Here are some classics in the field. I'll let you google them yourselves.
LeCun et al, 1998: Gradient-based learning applied to handwriting recognition. A deep convolutional net that can read handwriting, and was deployed nationally . Yann LeCun tells me that a patent lawyer in California reimplemented in his free time as a hobby, so it can't be that hard.
LeCun et al, 1998: Efficient BackProp. Tricks and implementation details that are not discussed often.
Btw, as I understand it LeCun was offered a position to be head of Google research. He declined, and Corinna Cortes took the job instead. Regardless, if you googled Yann for a while, Google ads would try to entice you to work at Google.
There is a recent trend in neural networks towards something called Deep Learning. This deep neural networks more closely mimic how the brain works, and are supported by arguments from neuroscience, circuit complexity, and machine learning. You can read more about them here:
Bengio, 2007: Learning deep architectures for AI
Hi my friend.
You will find mostly two kind of books. Those that have the classical style (basically almost everything before Brooks, though Valentino Braintenberg (Braintenberg vehicles) was already on the way) that ussually are called GOFAI, Good old fashioned Artificial Intelligence. There you will find most of the formal descriptions of AI, including Neural Networks and many other excellent (formal, well defined, interesting, concrete, etc...) algorithms. The other kind of books belongs to the Embodied AI approach. Which basically attempts to understand intelligence as an emerging property from the body-environment interaction (the fact that body evolved together with the brain, may be pointed out). As a physicist I suggest you to look to this side of AI, because the role of physics is very strong and there are many opportunities to define new concepts and formalize notions that still wait to be formalized: "embodiment", "situatedness", "ecological balance", "morphological computation".
To get in touch with this concepts you can read anything from Brooks and there is a nice book with lots of references: "Understanding Intelligence" by Rolf Pfeifer and Christian Scheier.
There you will find some answers...well...some questions..:D
Most of the references given in the previous replies belong to the GOFAI group, which is very popular among engineers because it gives tools to "controooool" robots...again, many interesting problems await for you in the other perspective of the problem.
www.ailab.ch/carbajal
The general idea for modeling brains using neural networks comes from Marvin Minsky (The Society of the Mind) a while back ago-- many of the papers are available at his web site at MIT. Minsky and Papert invented artificial neural networks (or what would evolve into that).
Neurologists offer a very useful point of view, with studies of the actual human brain. You may want to look at Gazzaniga. They offer the view that the brain is not a random organization of neural networks, but an organized set of organs connected in specific ways. (You may also want to look at The Language Instinct from Steven Pinker, for a human language cognitive perspective).
Finally, many have tried to simulate human brain functions. You may look at Cognitive Theories, and probably CMU's ACT (Andersen and others; most papers are online). In particular ACT has started to integrate neural networks and high-level symbolic processing for simulating human vision.
Most recent research is often in print years down the road, so the best way is often to track advances on the labs web sites.
Have fun.
Well once you have read a bit and want to play, may I suggest you look into Breve for your experimenting. Think of it as your AI simulation Expert Lego set. Lots of tools to visualize your algorithms. Cheers.
I do classification/pattern work in PERL + Java all the time for bioinformatics. A great book that got me started: Neural Networks: A Comprehensive Foundation, S. Hayken, Macmillian College Publishing,Inc., NY, 1994
If I could only keep one book in my (very large) AI library, it would be "Representation and Understanding" edited by Daniel G. Bobrow & Allan Collins. Although it is from 1975, I believe it is still in print.
If you are just generally interested in learning about AI, and not specifically Neural Networks. I highly highly recommend reading A Field Guide to Genetic Programming. Genetic Programming is really cool! and this is a wonderfully written book. Fun to read and easy to understand, great for anyone with a beginning or advanced knowledge of computer science. Best of all, it is freely downloadable under Creative Commons. Or you can order a print on demand copy from Amazon or Lulu.
I also highly recommend reading some books that are not directly related to AI. Checkout How the Mind Works by Steven Pinker. He is a phenomenal author.
singinst.org/upload/artificial-intelligence-risk.pdf
You should not worry too much about what is normally labeled "AI". The book by Russell and Norvig should really be called "Artificial Intelligence: a Good, Old-Fashioned Approach". Instead focus on the following topics:
1) statistical learning / pattern recognition
2) information theory (including algorithmic kind)
3) computer perception, especially vision.
You will discover that these are the best parts of AI, for the simple reason that they have the prettiest mathematics. As a physicist you should be able to appreciate the deep and not yet fully explored connection between statistical mechanics and information theory.
In terms of actual texts, try:
1) "The Nature of Statistical Learning Theory" by Vapnik
2) "Information Theory" Thomas and Cover
3) various papers by Geoff Hinton et al about neural networks
4) The two big books by Judea Pearl about Bayes Nets.
5) Maximum entropy papers by Jaynes, and the application to statistical natural language processing by people like Rosenfeld, Della Pietra, etc.
for book, just watch the movements of an insect. and make a routine that emulates that.
Good luck with your studies! ~ Jochen
... walk over to the CS department and talk to the chair. Explain what you want and (s)he'll point you to the best person in the department to give you the answers you want, if it isn't him/her-self.
Seriously, why the hell would you ask here when you have far more reputable people a few steps away?
"Neural Networks" by Simon Haykin
Disseminate the Power!
I should have written Daniel C Dennett instead of Marvin Minsky.
I would check out Stanford's (http://www.stanford.edu/) free online courses (there are many other universities participating as well). These are available via iTunes (iTunes U podcasts) or your web browser. Examples are:
Stanford's Programming Methodology - Video -Series of lectures by Professor Mehran Shami for the Stanford Computer Science Department (CS106A). Professor Shami lectures on options and opportunities after his class.
Stanford's Machine Learning (CS229) - Video - Professor Andrew Ng lectures on linear regression, gradient descent, and normal equations and discusses how they relate to machine learning.
I like the classroom videos. You get to see the demos of principles, can skip around, take a nap (!) or whatever and still get the complete course. Many of the courses provide a means for asking questions as well.
I especially enjoyed the machine learning course as it is a mix of fundamentals and the scaling up of those fundamentals to actual work. I know you were looking for written material but for me this multimedia format is so much richer. I hope this helps you.
Be as you would have the world become.
Castration would be enough.
Deus est fatalis