Foundations of Python Network Programming

Sarusa writes "This may be the easiest book review I've ever written. If you program in Python and you want to write Internet applications, go buy Foundations of Python Network Programming by John Goerzen. There. What, you wanted more? Well, okay, but then I'm back to playing Katamari Damacy." Read on for the rest of Sarusa's review. Foundations of Python Network Programming author John Goerzen pages 500 publisher Apress rating 9 of 10 reviewer Sarusa ISBN 1590593715 summary If you program in Python, and you want to write Internet applications, you need this book.

First of all, 'Network' means 'Internet.' Everything in the book concerns protocols running over IP, which is almost anything useful these days. That said, this is a lot of ground to cover -- there's FTP, HTTP, POP3, IMAP, DNS, a veritable explosion of acronyms, and this book does a great job of hitting all the ones you're likely to need.

Foundations assumes you already know Python, but nothing about network programming. The first 100 pages covers the basics of IP, TCP, UDP, sockets and ports, server vs. daemon, clients, DNS, and more advanced topics like broadcast and IPv6. And in case you already know all that, how Python deals with them. This is the only part of the book you will probably read in order. After that you pick what you need.

Find a topic you need to know how to deal with, such as using XML-RPC, and locate the appropriate section of the book. There he'll cover the basics of the topic, show you how to use the correct Python module(s) to implement it, explain any gotchas (this is key!), and write a short but functional application or two that uses it. I'm not sure why this book isn't called 'Practical Python Network Programming.' It's eminently Practical. It won't make your heart race, but it tells you exactly what you need to get the job done.

All this information is out there to find for free, but having it all collected and summarized is worth every penny. And the real value is having the edge conditions and not-so-obvious practical details explained by someone who's obviously used this stuff in the field. Python and its excellent libraries make Internet tasks relatively easy, but it's even easier with some expert help, and the libraries assume you already know what you're trying to do. For example, if you're doing a DNS.Request() record query and using a DNS.Type.ANY, it (for good reason) returns information cached by your local servers, which may be incomplete. If you really need all the records you need to skip your local servers and issue a query to the name server for the domain. This is isn't hard; you just have to know what's going on. Or do you know which exceptions can get raised if you're using urllib to fetch web pages? It's here. Exception handling is not neglected.

So you know what you're getting, here's a laundry list of topics: IP, TCP, UDP, sockets, timeouts, network data formats, inetd/xinetd, syslog, DNS, IPv6, broadcast, binding to specific addresses, poll and select, writing a web client, SSL, parsing HTML and XHTML, XML and XML-RPC, email composition and decoding, MIME, SMTP, POP, IMAP, FTP, MySQL/PostgreSQL/zxJDBC (though you won't learn SQL), HTTP and XML-RPC servers, CGI, and mod_python. As a bonus you get some chapters on forking and threading (for writing servers) and handling asynchronous communication in general.

Just to find something to complain about churlishly, I wish Goerzen had managed to do all this and make it scintillatingly brilliant and witty from cover to cover (all 500 pages); perhaps dropping juicy bon mots of gossip from the Debian project. And while I'm at it I'd like a pony. No, seriously. If you program in Python, intend to do anything Internet related, and aren't already a Python networking god, you need Foundations of Python Network Programming. In terms of 'hours I could have saved if only I had this book sooner' it would have paid for itself many times over.

You can purchase Foundations of Python Network Programming from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

Amazon

[RangerRick98]

The book is cheaper through Amazon

Re:Amazon

[justMichael]

Or you could actually shop around or go to bookpool, where it is $28.50 they don't show up on all book stores.

-- Sex Toys...

Re:Amazon

[Chrax]

Why not? It doesn't cost you a thing and so if you're going to be buying the book through Amazon, why not let him shave a little off the top. Are you really worried Amazon's losing money, or is this some dumb shit about him selling out? Who cares?

Re:Amazon

[Chrax]

Because Apple doesn't make damn good computers and amazon doesn't provide a service that allows people from anywhere to get pretty much any book. Good call. We, in general, don't like Sco because Sco is trying to make money by exploiting Linux (and doesn't really do much else), which is one of the greatest triumphs of free software/open source. We don't like the RIAA because they (at least attempt to) infringe upon fair use. It's not because they're sue-happy that they've caught the ire of many slashdotters. If that were the only criterion half of the United States alone would be the subject of slashdot rage. I don't know of Apple or amazon.com trying to keep me from doing something I want or pay for something that should be free, so I'm not all that fussed.

Twisted Framework

[BridgeBum]

Another good Python network resource: Twisted Matrix, a networking framework for Python.

For those interested in starting in network programming in Python, I'd recommend checking it out.

Re:Twisted Framework

[Phoukka]

The book makes extensive use of Twisted in covering IMAP programming, and also in its exploration of how to design a server, specifically the asynchronous, call-back approach. It does not cover the entirety of Twisted, that would be a book all to itself.

The bonus to Mr. Goerzen's use of Twisted in IMAP is that I came away with a much better understanding of how to use Twisted generally -- I grokked Deferreds for the first time. And I'd read all (ALL) the Twisted documentation I could get my hands on prior to that. That probably gave me the proper background, but the book really kicked in to place those final pieces necessary to get what was going on in Twisted.

The book doesn't just cover "raw" network programming, but covers multiple domain-specific areas and points you to the best libraries and modules to use for the area.

Good stuff, I highly recommend the book.

Typos

[2.7182]

I liked this book a lot, but there are an awful lot of annoying typos. Also my binding broke after a week.

Re:Typos

[Phoukka]

There are a fair number of typos, mainly in the text from what I saw. All the code looks good (though I haven't tested it all). It seems to me that John Goerzen made sure the code all worked, and wrote the text around it, maybe before it. Unfortunately, it looks like it was rushed to meet a deadline at some point, and the text did not get reviewed and edited as thoroughly as it might have needed.

On the other hand, the content is excellent, truly a good book. And so far, my binding hasn't broken, FWIW. But it was stiff, and doesn't want to lay flat on its own.

Re:Typos

[legirons]

"Also my binding broke after a week"

As practicalities go, the one thing I really liked about the last APress book I got ('Dive into Python') was that when I wanted to refer to it at work, I didn't have to carry the book in, I just read the section I wanted on their website.

It was one of those "never going to buy another book without this facility" moments... how could we have missed something so useful for so long?

So back on topic, this book only has one chapter available for download, and it's in PDF rather than anything useful, so I guess it's not general policy to make all APress books downloadable.

I did find it amusing how Visual Basic, C#, and .NET each get their own top-level section at the APress website, while PHP, Perl and Python are down in the "Open-Source" section next to books describing products (Apache, Linux, Plone, MySQL, etc.)

As the author says...

[lukewarmfusion]

"All this information is out there to find for free, but having it all collected and summarized is worth every penny."

Is it? If you are, as the author says, someone familiar with Python but you have no clue about network concepts or programming, perhaps this book isn't for you. The first 100 pages or so are all intro to networking; after that, you have specific Python networking programming topics. Perhaps you'd be better suited with a networking book and then this book (sans the first 100 pages).

I've read a few books on programming languages and when they decide that the reader needs an intro to something, they usually provide pretty poor coverage of that topic. You end up being lost after you get done with the intro section. I did this when I was learning some encryption programming... before I could start actually writing code that deals with encryption, I needed a solid base. Instead of trying to teach me all I needed to know, the reference I was using pointed me at the industry's best encryption and security books and authors (like Bruce Schneier).

Disclaimer: Not having read this particular book, maybe this one is different. I don't know.

Re:As the author says...

[Anonymous Coward]

I totally have to disagree with you. I hate when I spend good money on a book about a subject and then there is a bunch of extra crap thrown in there I feel ripped off. For example: just about half of all the security books I own waste a whole chapter on TCP/IP going over the fucking OSI model and crap. True you should understand TCP/IP if you want to understand security but one chapter in the front of the book is not going to teach you all you need to know about TCP/IP. You'd be better off just buying a seperate book on TCP/IP and the publisher would be better off yanking those chapters from the book. But they keep it in there as padding anyways so that the book is bigger and they can jack up the price.

That is why all the older Oreilly books are so good. No padding. No bullshit. The books were about what the cover said they were about, no less and no more. All meat.

Re:As the author says...

[Phoukka]

Okay, I have read the book, and the review author is correct in his statement. No, the book will not teach you the OSI model, nor will it teach you general networking theory to an expert level. However, I came to the book with a general knowledge of Python and a poor-to-middlin' understanding of network programming in particular. I've built web apps using pre-existing frameworks, for example, but I've never written code to work with sockets.

Having read the book, I understand socket programming, general network programming, and could probably design and implement my own application protocol -- badly, of course, but still... Could I have done this prior to reading this book? No. Did this book make it easy to pick up the necessary background, as well as make it easy to pick up the specifics of network programming in Python? Yes.

This is a great book, and is a must-have for Python programmers.

Re:Look, I program in Perl

[kevin_conaway]

I learned perl before I learned python and I found that the OOP constructs in python were better than perl. In perl they just feel bolted on while in python, they are built in from the ground up. Other than that, not much difference in my opinion

Python makes Windows fun

[mslinux]

I use Python to administer computers... lots and lots of computers. It runs on Macs, Windows, Linux and all variants of Unix. It's one of the most portable languages around.

I used to be a huge Linux buff (and still am when it comes to servers), but intelligent tools like Python make using Windows XP Home a much more fruitful and fun experience as I can actually get stuff done programmatically. Go Python developers and keep up the good work!!!

Python Web Programming

[Anonymous Coward]

I found Python Web Programming a few years back covered almost all of what the review mentioned.

It also had a brief Python tutorial in it, but I kind of skipped over that, so I can't vouch for that part. The rest of the book will definitely teach you a bit about network programming, web/database programming, and things of that nature. For most of the /. programmers it might be pretty old hat since they were doing this stuff in the womb, but for unexperience programmers such as myself, I found it helpful.

Civ IV moddable with python

[ultrabot]

Incidentally, Civilization IV is going to be moddable with Python

Re:Look, I program in Perl

[ultrabot]

Also I really like how functions are first class objects (meaning you can pass them to/return them from functions)

And classes too. What's more, a class is just a callable object that is called exactly like a function - "C()" - and it returns an instance of that class. If a function wants a class for instantiation, you can just pass a factory function that returns some instance of any class.

How many Foundations?

[PCM2]

Foundations assumes you already know Python, but nothing about network programming.

Ah, but what of the Mule? Doesn't the presence of an advanced mutant of his scale defeat this whole premise?

Re:Look, I program in Perl

[Quixotic137]

Put your code in a file called MyModule.py. Then you can use it just like modules in the standard library:

import MyModule
MyModule.SomeFunction()

Or:

from MyModule import *
SomeFunction()

Every Perl programmer should switch to Python.

[pclminion]

And vice versa: Every Python programmer should switch to Perl.

At least, for a month or so.

Knowing multiple languages increases your value as a programmer quadratically. I like to think that languages follow a square law. By doubling the number of languages you know, you quadruple your total skill and marketability as a programmer.

I've done significant stuff in both languages and there are definitely tasks where Python is better -- for example, command-and-control, super-high-level types of apps, which coordinate large systems of smaller programs. And Perl is vastly superior in other situations, such as processing enormous wads of data and formatting output. I've even written hybrid programs where Python and Perl code intertwine.

Step outside your box. You don't have to love the language you're learning, but consider it an investment in yourself. Saving money sucks too, but it's still a good idea.

Re:Every Perl programmer should switch to Python.

[snakecoder]

I am a python bigot but one thing Perl taught me was how to quickly take advantage of regular expressions (looking at other peoples code).

Now I use Pythons re() module which takes a little getting used to. While I'm at it, for those of you that write complex or deep regular expressions where re() craps out, there is a much more stable legacy module called pre() that seems to be undocumented. It works exactly like re(). The only difference, it trades speed for stability.

Re:Every Perl programmer should switch to Python.

[the quick brown fox]

I agree with you, but I don't think you go far enough. Going from Python to Perl isn't stepping very far out of your box. Why not try Lisp, Haskell, Erlang, Scala, Smalltalk, or ...?

Different syntax and different libraries will open your mind a little. But when a language encourages (or forces) you to think differently, that's where your "square law" starts to kick in.

Re:Look, I program in Perl

[pnatural]

Also I really like how functions are first class objects (meaning you can pass them to/return them from functions)

And classes too.

Yes, absolutely. First-class functions and types is one of the most important hallmarks of a good, usable language. Not being able to pass around classes or functions would severely limit how most of my solution sets are defined. Personally, I won't consider languages that don't offer this.

Have you tried Ruby?

[Colonel Panic]

I really think that if you're coming from Perl you'll prefer
Ruby to Python. No indentation hassles with Ruby, for example. You'll also like the way Ruby does OO compared to Perl OO. More Rubilicious links...

Also, The Pragmatic Programmers have released a new edition of Programming Ruby that's a great intro and reference to the language - go buy it from their website.

Ruby: Because I can't wait around for Perl 6 to get finished

Re:Have you tried Ruby?

[ciw42]

Although I've used seeming every language under the sun at some point during the past 20+ years, my language of choice for most projects these days is Python.

As a background to my choice, here's what I use it for:

I tend to write primarily for the Win32 platform and most of my applications have GUI front-ends they speak to MySQL databases, and often also control third party applications via COM. Aside from the COM stuff (the apps I'm controlling are only available for Win32 anyway) my software is fully cross-platform which is desireable. I love and use GNU/Linux extensively, and am starting to see an interest from the SME market which is encouraging.

I've used Python a lot and Perl a fair bit, plus I've looked at and thoroughly expected to fall in love with Ruby and Lua. I didn't.

I've realised that all four languages are so similar in many respects, that it's very difficult to convince a person using one to convert to another unless they have a very specific need. So it's just not worth trying.

If the language you are using does the job for you, then stick with it. Once you know the work-arounds for its deficiencies (and they all have them) then there is even less reason to change.

Trying to be objective, here's how I find each of the languages:

Python - Extremely easy to pick up, which is actually good for experienced programmers as well, but at the same time very flexible and powerful. Very readable and easily maintainable code. Good range of libraries (but nowhere near as many as Perl) which all stick closely to a well established "pythonic" way of doing things. You don't have to choose from a dozen different libraries that all claim to do the same job. The interactive shell is also remarkably useful for experimentation and debugging. Most good programmers indent their code anyway, and I don't know anyone that found the forced indentation a problem unless they were deliberately being arguamentative. The concept of packages is very simple and neat - you don't need to do anything special to allow importing of your code. Object orientation is very flexible, straight-forward and powerful. There are a large number of precompiled libraries with installers for Win32 platforms - don't ever underestimate how important this is in when using scripting languages in the current commercial environment. Extensive and uniform use of dot notation. Good range of freely available cross-platform IDEs. Like most Python bindings, those to GUI libraries are generally much easier to work with than the original C libraries.

Perl - Very powerful but extensive use of special characters rather than keywords can tend to result in code which needs reading several times to fully comprehend. Having built-in regular expressions is both useful and powerful, but only adds to the problem of making code less readable. Th eobject orientated aspects of the language are very much bolted on, and far from elegant. Functionaly they're quite capable, but certainly not pretty. It's very easy to code in your own style with several ways of doing the same thing, not necessarily a bad thing, but it does means there is more to learn of the core language if you want to be confident about being able to maintain code written by others. You do feel that you have flexibility in your choice of coding style which is always nice. Immense number of additional libraries, available from one source - the wonderful CPAN - but there is also a good deal of duplication, and you need to spend time evaluating the options to find one that has the features you need and works the way you'd like. Packages have to be written or at least bundled up as such. That said, it's available by default on *nix systems, it's also very closely tied into the operating system and shell which makes OS related stuff in Perl a breeze. Win32 support is available, but Perl is only truly at home in a *nix environment. The bindings to most cross platform GUIs are aften more complicated and difficult to use than the C equivalents.

Ruby -

SWIG rocks for plugging into C/C++/Libraries.

[SimHacker]

SWIG is an excellent open source tool for plugging C++ and C code and libraries into Python and other programming languages.

I can't say enough good things about SWIG. It's an amazing piece of work that has saved me years of menial labor and enabled me to integrate all kinds of compex code into Python, from hairy C++ templates to third party Win32 libraries for which there is no source code. It works extremely well with Python, and many other languages too.

Here is the blurb from the web site www.swig.org:

SWIG is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages. SWIG is primarily used with common scripting languages such as Perl, Python, Tcl/Tk and Ruby, however the list of supported languages also includes non-scripting languages such as C#, Common Lisp (Allegro CL), Java, Modula-3 and OCAML. Also several interpreted and compiled Scheme implementations (Chicken, Guile, MzScheme) are supported. SWIG is most commonly used to create high-level interpreted or compiled programming environments, user interfaces, and as a tool for testing and prototyping C/C++ software. SWIG can also export its parse tree in the form of XML and Lisp s-expressions. SWIG may be freely used, distributed, and modified for commercial and non-commercial use.

-Don

Re:Look, I program in Perl

[randall_burns]

Python isn't a Swiss Army Chain saw-more like a table saw. Both Python and Perl have their place. I've used Perl happily when doing my own projects-but I've found that Python is _much_ easier to adapt newbies to using-and much easier for process oriented managers to deal with who otherwise might consider Java. I have personally seen older Cobol programmer who were utterly intimidated by VB,Java and other newer languages look and Python and relate easily to Python.

Re:Easiest review to skip

[Boronx]

I don't use Python for much, but one of the primary reasons when I do is that a small, hacked together, uncommented python script is way easier to read months or years later than a similar Perl script or C source.

Re:Look, I program in Perl

[killjoe]

Regardless of the merits of the languages I think you'll severely miss CPAN if you switch to python. CPAN is the best thing about perl and python has nothing really like it. There are a couple of so called repositories but they don't hold a candle to CPAN.

Encryption Programming and Canned Libraries

[billstewart]

The first thing you get from a "solid base" of encryption knowledge is the concept that doing your own encryption algorithms except as an exercise is a really bad idea.... But that doesn't mean that you can write application programs that use standard libraries like OpenSSL without a solid understanding of what the encryption technology's doing - you really do want to be more than a script kiddie in this field.

The Agony of SOAP/WSDL

[Anonymous Coward]

For anybody who has had to deal with this wretched monstrosity, python is a blessing thanks to the SOAPpy package.

The usual way involves a pageful of obscure code, and having to use obtuse WSDL descriptor files and code generators to give you classes.

But, hey, python can generate classes and methods on the fly. So getting the temperature at zip code 90210 becomes a one-liner after some standard imports:

SOAPpy.WSDL.Proxy('http://www.xmethods.org/sd/2001 /TemperatureService.wsdl').getTemp('90210')

I'm not kidding; that line currently returns the floating point number 68.0.

Note: you'll need to remove the extra space in the URL that the slashdot antilameness crud inserts. The full example is found in simpleWSDL.py in that package.

If you're a "$LANGUAGE Programmer"...

[don.g]

If you consider yourself to be a $LANGUAGE programmer, then there's something wrong.

As the parent post says, knowing multiple languages is good. One of my pet annoyances is hearing people describe themselves as a programmer for a specific language -- there are many more out there, and to say you only do one speaks volumes about the lack of breadth of experience you posess.

And don't just stick with imperative object-oriented languages. Try a few declarative languages, like Haskell (functional) or Prolog (logic). Yes, getting your head around them is hard. But you'll be glad you did.

Disclaimer: I'm a student doing an MSc in Computer Science, and by lines of code, most of what I wrote in the last twelve months was Perl, and was completely unrelated to my thesis :-)

Re:If you're a "$LANGUAGE Programmer"...

[Anonymous Coward]

Try a few declarative languages, like Haskell (functional) or Prolog (logic).

You're a little bit off there. Haskell is not declarative. It's just functional, as you said. Prolog is declarative though.

If you want to try a nice variety of languages here's what I suggest:

C - 'nuf said.

Java or C# - Good application programming languages. Similar enough for learning purposes that it doesn't matter which you use. If you want to get into the inner workings, both are very interesting systems to learn about.

Perl or Python - They're not too similar as languages, but their purposes are similar. If you want to do scripting, these are good choices.

Haskell - Purely functional language. As expected from a purely functional language, things like I/O are difficult in Haskell. Still, an excellent way of learning a whole new way to think about programming.

Prolog - Nice declarative language. Thinking in first-order logic is another thing you don't do with common languages. Better get used to it though, since between the semantic web, Spotlight, WinFS, and other metadata, programs are going to have a lot of logical reasoning to do.

Assembly - Sure, you can survive without this these days, but knowing what's going on at the very lowest level gives you a new perspective on programs. It can also help when things go really really wrong with your program and you're not sure you can even trust the VM any more. Save your sanity. Don't look at x86. I suggest MIPS.

awk and sed - Yes, the classics. It doesn't hurt to look at all the common unix tools, but these are the big boys of string processing. You'll need to learn the basic of regexes at some point, and if you don't use Perl, this would be the place to do it.

Optional:

C++ - It's not dead yet. Personally I don't know C++, but most apps out there are still using it, so it's inevitable that I, and you, will see it some day. If you know C and Java/C#, you should be able to read C++ at least.

Smalltalk - It never hurts to study the classics. Also, the distinct lack of ancestry from C that Java and C# have can make it easier to figure out how that whole object-oriented thing is really supposed to work.

BASIC or Cobol - You study Smalltalk to understand how good things could be. You study these to understand how bad they could be.

Lisp - Possibly as an alternative to Haskell. Another classic, and a little more practical than Haskell. I think Haskell is still a better teacher of functional programming, but take your pick.

Fortran - Do you do numerical methods? Then be sure to try Fortran. Just as C is close to the machine, Fortran is close to the math. I can't guarantee you'll like it, but it's worth a look.

Ruby - I think of this language as just plain fun. It has so much syntactic sugar you could bake it into a cake. Be warned, you may end up liking it even more than Perl or Python.

XUL, XAML, or X Forms - I can already hear people saying "What the heck?" This is just my own personal opinion, but declarative UIs are the way to go. How many people would seriously still write them procedurally anyways? Interface Builder, Visual Studio, HTML...people already "build" UIs anyways. I think there's benefits in a rich UI language that's both human and machine readable, hence, I suggest one of these...

XSL - awk gone XML with a few imperative features. Unless you think you will never ever touch XML, learn this. Especially learn this if you think XML is a silly idea. I would generally agree, except some great tools have been created because of it.

Python == good

[mcrbids]

I spent some time learning a bit of Python, only to abandon it. Not because of Python, but because I had too much inertia going in various PHP projects that pulled me back.

Python caused me to change my layout for code, almost instantly eliminating a big problem with c-like code: the missing brace.

Most code is structured like this:

Function fubar {
statement a;
do (b);
if (c()) {
performfunction(x);
}
}

In this small segment, notice that there are to sets of braces - and they don't line up at all. You have to mentally follow the code after "fubar" and see that after the condition "if (c())" in order to mentally track the state of the braces.

Compare this to

Function Fubar
{
statement a;
do (b);
if (c())
{
performfunction(x);
}
}

(slashdot's ECODE filter sux0rs)

If you could see it, you'd notice that the braces line up. The opening and closing braces for the condition "if (c()))" are indented one more than the braces for function Fubar() which are indented more than the line "Function fubar()" itself.

Thus, you merely have to follow the indents to match the opening/closing braces. As a result of this change, I spend less than 5 minutes per week matching up braces without the need for an IDE to match them up for me.

Python seems to be a good language (I like that you can compiles sections of a Python program in c to improve performance without rewriting the whole program) but it's concepts of layout certainly carry beyond Python itself!

Re:Look, I program in Perl

[juhaz]

from sys import stdin
print "Who are you?"
name = stdin.readline().strip()
print "I'm glad to meet you, %s." % name

Overly verbose and complicated, you can write this in two lines and save an import as well as the need to know stdin is a file object to boot. And string formatter operator is not instantly clear, especially to a person who hasn't used printf(), not everyone has C background.

How about

name = raw_input("Who are you? ")
print "I'm glad to meet you, " + name