Slashdot Mirror


Linux System Programming

Jon Mitchell writes "As a Perl programmer recently thrown in to the world of C development on Linux, I have been looking for something that would take my K&R level of experience and bring it up to date with modern methods, hopefully letting me write more efficient and reliable programs. Linux System Programming is a volume that targets this need. Robert Love, former "Chief Architect, Linux Desktop" at Novell, kernel hacker of many years, and Gnome developer of well known features such as Beagle and NetworkManager, attempts in this book to document the Linux system call and C API to common systems programming tasks. Given that he developed the pre-emptive kernel and inotify he has the knowledge." Read below for the rest of Jon's review. Linux System Programming author Robert Love pages 388 publisher O'Reilly Media rating 8/10 reviewer Jon Mitchell ISBN 9780596009588 summary The Linux system call and C API explored in depth. Getting this book out of the box, I had wrongly been expecting a cookbook style that I would get instant gratification from. Although structured around common programming tasks, it doesn't lend itself to just dipping in. The section on time lists a handful of ways that "time" is available to the programmer; jump into the middle of the section and you might miss the most suitable one for the job in hand. The book rewards reading it in larger chunks.

This doesn't mean it is necessary to read it from cover to cover. Logically organized into chapters around "things you want to do", such as file access, memory management and process management it will lead you in with a survey of techniques you might be familiar with, before drilling down with advanced methods.

Knowing advanced methods for performance is great, but not at all costs. One of the most useful and practical lessons this book gives is to encourage you to think about error conditions that may occur during a system call. Early on, in the section on reading files, a detailed example is given on reading from a file. Every possible case of return code from the read call is described together with what it means and how you should handle it — it can be surprising that 7 possible outcomes are listed, with good descriptions of what to do with each of them.

This good practice by example continues throughout the book. Every system call described also lists the errors that may occur. This does show up a slight weakness: many system calls share a common set of errors which are repeated many times in the text. If you are not paying attention it may feel like you are just flipping through man pages. However you are soon halted by the easy introduction of an advanced concept to get your teeth into.

These are done in a nicely graded level for each topic. In "file access" to give an example, you are lead from simple read/write calls, through to what the C library can provide in buffering, to improved performance using mmap. The techniques continue with descriptions of I/O schedulers and how the kernel will order hardware disk access, scatter/gather, and ends up with how it is possible to order block reads/writes yourself bypassing any scheduler.

You are hardly aware of the progression, as the pacing is very well done. New concepts clearly fit into what you have seen so far — current sections signpost the practical use of what is being explained and at what cost, allowing clear consideration of the use of advanced features against any consequences.

For process management discussion starts with fork and exec, before moving onto user ids and groups, covers daemonification and goes onto process scheduling, including real time scheduling. Throughout the book each new call is illustrated with a short code snippet showing the call being used in a practical situation.

Not everything is present and correct. The author immediately states that networking is not covered at all. This is a shame as this subject would benefit from the depth of coverage given to the topics in this book — although no doubt would increase the number of pages considerably. Perhaps scope for a second volume. The length of some sections seems odd — Asynchronous file I/O is whizzed through in a page with no code example, whereas I/O schedulers gets a luxurious 12.

On the other hand there are some unexpected and useful extras, such as a discussion in the appendix of gcc C language extensions and how they might be used to fine tune your code.

The books stated target is for modern Linux development, a 2.6.22 kernel, gcc 4.2 and glibc 2.5. Many calls have been standardized by POSIX, and where this is so it are noted in the text, so a large portion of the content is useful on other systems. There is even the occasional mention of non-Linux system calls, the use of which is not encouraged, but shown so you know how they function if you come across them in older code.

I recommend this book to anyone who has a need to developing Linux applications. The book is not a primer in C on Unix, so you are expected to be familiar at least to the level of K&R. From this level though the journey into getting the best from the kernel and C library into your programs is easy going and enjoyable.

You can purchase Linux System Programming from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

21 of 98 comments (clear)

  1. All accessible from Perl! by Anonymous Coward · · Score: 1, Informative

    On most UNIX systems, the POSIX API is fully available to Perl scripts. One of the great things about Perl is that you get all sorts of high level features that aren't available in C, but then you also get all of the low level features that you often need when writing hardcore UNIX software. Best of all, Perl is damn fast, usually on par with C for most tasks. And it's often a lot faster when doing regular expressions work, for instance.

    1. Re:All accessible from Perl! by mr_mischief · · Score: 3, Informative

      Perl is compiled into an AST, goes through code improvements, and then is executed.

      Since it typically goes through this every time you use a program from the command line, the startup time tends to be pretty heavy.

      If you're using something like mod_perl or FastCGI or some other caching dispatch mechanism, your program gets dispatched without recompilation if it hasn't been changed.

      If your program is long-running, then the startup cost can become negligible.

      Perl's common routines are written in optimized C and with good algorithmic design in mind. If someone writes an equivalent from scratch in C instead of using a good library, then the Perl version will have been designed and refined by far more people.

      It's true that in many cases C comes out well faster than Perl, but those cases are not as common as people tend to think.

    2. Re:All accessible from Perl! by skeeto · · Score: 2, Informative

      Any way you could back that up with some numbers?

      Unless your program only crunches a lot of numbers during its entire runtime (for example the ImageMagick tools) your program will spend most of its time waiting on some kind of I/O. This encompasses pretty much all software you will find on a normal desktop computer. Perl and C both spend the same amount of time waiting on I/O operations. It comes down to spinning disks or waiting on the slow, clumsy fingers of users.

      On the other hand, Perl is faster when it comes to development time. The Perl programmer will write the same program as the C programmer, but in a fraction of the time. If we are generating fractals or something, the C programmer's version will me smaller and run much faster, but, in the same amount of development time, the Perl programmer can write his program and be out to dinner and a movie. Or trolling Slashdot or whatever.

  2. Re:What about a C++ coder? by blitzkrieg3 · · Score: 2, Informative

    Though I haven't read the book, I think it is safe to say that you should familiarize yourself with C a little better before reading this. You should pick up K&R, or at the very least familiarize yourself with the way common data structures look in C.

    Having said that, if you have no problems understanding man pages for system calls, you should be good to go.

  3. Re:K&R by stoolpigeon · · Score: 2, Informative
    --
    It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
  4. Re:What about a C++ coder? by Peaker · · Score: 4, Informative

    How hard would it be for a C++ coder to dig into this book?

    Should be pretty easy. All the code examples are valid C++. All you need to do is remember that "class" is called "struct" and that you have to mangle your own names.

    C++ is not a superset of C, and is definitely not supposed to be written like C.

    For example variable-length arrays (added by C99) are not supported by C++ (which has vector objects instead).
  5. Robert didn't develop the preemptive Linux kernel by Daniel+Phillips · · Score: 2, Informative

    Robert has done plenty of useful work, but it was George AnzigerAnzinger who developed the Linux preemption patch. Robert picked it up, maintained it and got it merged. The credits to George seemed to have gotten lost somewhere in that process.

    Credit where credit is due please.

    --
    Have you got your LWN subscription yet?
  6. Thou shalt not ignore warnings by mi · · Score: 5, Informative

    Build your code with -Wall -Werror (or your compiler's equivalent). Once you clean up all the crud, that pops up, crank it up with -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith. Once there — add -Wreturn-type -Wcast-qual -Wswitch -Wshadow -Wcast-align and tighten up by removing the no in -Wno-unused-parameter. The -Wwrite-strings is essential, if you wish your code to be compiled with a C++ compiler some day (hint: the correct type for static strings is " const char *").

    For truly clean code, add -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls.

    The people, who wrote and maintain the compiler, are, most likely, several levels above you in understanding programming in general and C-programming in particular. Ignoring the advice their code generates is foolish on your part...

    As a minimum, solved warnings will make your code more readable by reducing/eliminating the "Why is he doing this?" questions. More often than not, they point out bugs you would otherwise spend hours chasing with a debugger later.

    And they make your code more portable. But if you don't understand, why a warning is generated — ask around. Don't just "shut it up". For example, initializing a variable at declaration is usually a no-no. If the compiler thinks, the variable may be used before being initialized, scrutinize your program's flow. If you can't figure out, it may some times be better to disable this one warning temporarily with -Wno-uninitialized to move on, instead of shutting it up for ever by a bogus "= 0" or some such...

    The book may well say something about respecting warnings, but the review does not, which is a shame.

    --
    In Soviet Washington the swamp drains you.
    1. Re:Thou shalt not ignore warnings by david.emery · · Score: 4, Informative

      Many studies (e.g. the Bell Labs 5-ESS fault analysis) and anecdotal stories indicate that failing to check the error return on a system call (or any other function, for that matter) is all-too-common. Adding to this problem, when a system call fails, often the manifestation/error/seg fault is not at that point of call, but further down, when a pointer/variable you expect to have meaningful data is null/garbage...

      That's why, when we did the Ada Binding to POSIX (IEEE 1003.5/ ISO 9945), we decided to accept the overhead of imposing exceptions for system call error returns (in most cases). You can't ignore the exception!

      This raised two interesting concerns that we discussed when developing the standard:

      1. What about tasking/threads/concurrency? The requirement on the implementation was to set up per-task errno values. From an implementation perspective, this meant that you needed to go outside of the standard interface to correctly implement POSIX/Ada, as you needed to grab the errno value and load it into task-specific storage, or require that your underlying POSIX threads implementation (if that's how you built the Ada runtime) do that for you. In practice, this is not too onerous, and it's proven to be a real boon for ensuring proper behavior (including debugging) in a multithreaded/multitasking environment.

      2. We also needed to think about the situation (usually representing really poor programming) where an unhandled exception (from a system call, an application call, or a language predefined exception) rips up the callstack and terminates the process. We wanted a return value from the process exit that would be 'close to 1 but not collide with commonly used values.' The number we chose: 42 (with the appropriate citation in the bibliography:-)

      So sure, a C++ program can use the C binding, but I think defining and using C++ exceptions in a better C++ interface would be preferred.

      dave (Tech Editor for the original IEEE P1003.5 project...)

    2. Re:Thou shalt not ignore warnings by mi · · Score: 2, Informative

      So, what you are saying is that you'd rather see the program fail with a completely bogus value you have no idea where it is coming from (which is whatever was on the stack at the time the variable was pushed) than a known invalid initialization value (e.g. -1) you pick and you set your variable to ?

      This sort of error is easily caught with something like Purify or valgrind.

      Also, if the warning was generated, you disabled it, and your program failed with a random result, that's a very good indicator, that you need to re-enable the warning and study the compiler's output and your program's flow once more.

      But invalid initialization value is another way, but these are slightly inefficient and thus bad style. If you can't figure it out, or if the compiler is incorrectly warning you in one file, go ahead and use this method so as not to disable the warning for all other files in your project... Just put a comment next to the bogus initializer, explaining, why it is there — someone (possibly you) will be able to remove it eventually, when the code changes or compiler improves.

      --
      In Soviet Washington the swamp drains you.
    3. Re:Thou shalt not ignore warnings by jgrahn · · Score: 2, Informative

      Build your code with ...

      I always use -W -Wall -pedantic -std=c89 plus any glibc #defines to enable POSIX/BSD/whatever functions I need.

      Seeing people respect and use the gcc warning flags makes me happy, but I don't know why you chose to leave out -pedantic and (more importantly!) the option to select which bloody language you are feeding the compiler.

      But if you don't understand, why a warning is generated ask around. Don't just "shut it up". For example, initializing a variable at declaration is usually a no-no. If the compiler thinks, the variable may be used before being initialized, scrutinize your program's flow.

      I think I understand what you mean: that it's wrong to write int foo=0; if you never intend to use the fact that foo starts out as zero. If so, I agree. That's just a way of making your bugs harder to find, and your code harder to read by obscuring its purpose. But in general, initializing variables is a good idea. In C99 and C++, I usually have something suitable to initialize them with, because I am allowed to declare them where I have the need for them, rather than at the top of the block.

  7. When was it not? by moderatorrater · · Score: 2, Informative

    Their about page calls it the "perl interpreter" multiple times. How is it not an interpreted language?

    1. Re:When was it not? by Mornedhel · · Score: 2, Informative

      Why do you think there isn't an interactive Perl interpreter (at least that I know of)?

      Actually, you can start a debugger session with perl -de 1 (that's the number 1 ; any other empty script will do). That acts like an interactive Perl interpreter would (but really is a loop of "user entry/eval(user entry)/start again").

      Still, you're right in that Perl is a compiled-then-interpreted language (like Python and others).

      --
      This /.-related sig is a stub. You can help Mornedhel by expanding it.
  8. Re:What about a C++ coder? by Chandon+Seldon · · Score: 4, Informative

    C++ is not a superset of C, and is definitely not supposed to be written like C.

    C++ is damn close to being a superset of C. Any C code examples given in this book are almost sure to be valid C++. Further, the fact that C code makes for awkward and ugly C++ code doesn't mean that it isn't *valid* C++ code.

    C and C++ are very different languages in programming style, but anyone who knows C++ already knows the C syntax and semantics - at most they'll need to learn the modern C programming style to actually use it.

    --
    -- The act of censorship is always worse than whatever is being censored. Always.
  9. Re:Ada's approach to syscall-failures by david.emery · · Score: 3, Informative

    That's not surprising, since the use of '42' is an artifact of the Ada binding, and those systems do not by default contain an implementation of 1003.5/9945. They should, but that's another story. Ada actually meshes very nicely with Unix, and is a good choice for system-level programming above the kernel level. Strong Typing -is your friend-! (I've been doing library level system programming on Unix systems, starting with Ultrix in 1984...)

    The standard Linux/Solaris Ada compiler is the GNU Ada Compiler, http://www.gnat.com

    But at least it's good to know there isn't a conflict.

          dave

  10. One of my favourite books ever by Anonymous Coward · · Score: 1, Informative

    As soon as I heard that Robert Love had written this book about userspace programming, I rushed to buy it.

    I had really enjoyed both "Linux Kernel Development" (Developer's Library, 2003) and "Linux Kernel Development 2nd ed." (Novell, 2005). I like how clearly and brightly the author describes linux internals, from the major architectural components to the key code chunks.

    This book was a great surprise. It's the best you may desire when you have to quickly design and develop complex solutions with glibc&gcc, and you want to take full advantage of your powerful linux kernel. The most complete guide through system and library calls, with elegant code examples.

    It is going to stay on my office desk for a quite while.

  11. Re:What about a C++ coder? by Evil+Pete · · Score: 3, Informative

    C++ was originally a superset of C. But later changes to C / C++ have drifted considerably from that. However, that means that generally C shouldn't be a problem for C++ programmers. There are large differences in the philosophy though that will affect the quality of your C code.

    --
    Bitter and proud of it.
  12. Robert Love .... by NullProg · · Score: 2, Informative

    Is a great kernel developer/programmer (He also does columns for Linux Journal). He is not a general purpose Linux programming author.

    Getting this book out of the box, I had wrongly been expecting a cookbook style that I would get instant gratification from. Although structured around common programming tasks, it doesn't lend itself to just dipping in.

    For getting your feet wet with Linux programming I recommend GNU/LINUX Application Programming by M. Tim Jones or Linux Application Development by Michael K. Johnson and Erik W. Troan.

    The Linux Unleashed series is also good (1000+ pages with hundreds dedicated to perl, python, and Gtk programming).

    Enjoy,

    --
    It's just the normal noises in here.
  13. Re:What about a C++ coder? by Curien · · Score: 2, Informative

    C++ was never a superset of C, and it was never intended to be such. Trivially,

    int main(void) {
        int class = 0;
        return 0;
        }

    was never a valid C++ or Cfront program, but it has always been (and probably will always be) a valid C program.

    I'm not an expert on Cfront, but I do know that there are quite a few major differences between C and ARM C++ (sizeof character literals, meaning of empty argument list, type conversions), so your characterization of C++ as having diverged from a superset of C recently is off the mark.

    --
    It's always a long day... 86400 doesn't fit into a short.
  14. Re:What about a C++ coder? by Curien · · Score: 2, Informative

    Both of those features were added to C after C++ was standardized. In particular, C99 VLAs were invented after C++ vectors (which were mostly solidified as part of the STL by the time of the C95 library update). As for your comparison, it would be interesting to know the specifics of your measurements (code, etc).

    The current C++ folks are more interested in fixing the mess they made with templates. Designated initializers would be mostly unnecessary if the language supported named argument mapping a la Ada and ColdFusion. I believe the general feeling is that C++ doesn't need more support for PODs.

    --
    It's always a long day... 86400 doesn't fit into a short.
  15. Re:What about a C++ coder? by Evil+Pete · · Score: 3, Informative

    I didn't say 'recently'. I remember it was stated that C++ WAS a superset. Though it was probably moer accurate to say a superset of ANSI C. In fact there were early C++ compilers that actually preprocessed the C++ code into C first. Of course I am talking 15-20 years ago.

    So I stick to my remarks.

    Bloody young whippersnappers.

    A comment that follows has a link to Stroustrop's page about this. Yes it is not a mathematical superset. But it is practically one:

    Thus, C++ is as much a superset of ANSI C as ANSI C is a superset of K&R C and much as ISO C++ is a superset of C++ as it existed in 1985.

    Well written C tends to be legal C++ also. For example, every example in Kernighan & Ritchie: "The C Programming Language (2nd Edition)" is also a C++ program.


    --
    Bitter and proud of it.