Suggested Books for Learning Cryptography?

Kismet asks: "I see a lot of news and reviews about encryption on Slashdot regularly. The subject seems rather interesting. I suppose that cryptography is covered in CS classes, but for hobbyists, what is the best way to learn? What kind of math background does one need? Are there any good "beginning" books on the subject? What is the history of Cryptography? "