Slashdot Mirror
Math Toolkit for Real-Time Programming
Who & What
Jack W. Crenshaw, Ph.D. (Physics) wrote his first computer program in 1956 for an IBM 650. He has been working with real-time software for embedded systems ever since -- contributing several years to NASA during the Mercury, Gemini, and Apollo programs. In addition to other activities, he is currently a contributing editor for Embedded Systems Programming magazine and author of the Programmer's Toolbox column.
In Math Toolkit for Real-Time Programming, his effort is focused on describing the pitfalls of vendor-provided math libraries and providing robust replacements. In section one he gives a thorough overview of constants and the various manners in which to declare them, naming conventions, and error handling. As the work progresses, in section two, he builds a library of proven algorithms ranging from square roots to trigonometrical functions to logarithms. Did you suffer through calculus in college with a barely passing grade? Section three will teach you more about numerical calculus in a half-hour than you may have learned in three semesters.
KudosMath Toolkit is written in an easy to understand anecdotal manner. You might be tempted to think that the author was animatedly relating the history of computing square roots while having lunch with you. This method works very well and keeps what could be a rather heavy subject from becoming too much of a burden. Most chapters have historical tidbits liberally sprinkled throughout.
Even if college algebra left you with post-traumatic stress disorder, you will not have any trouble with section two. Indeed, you may find yourself intently following the author on the trail of the perfect arctangent algorithm -- much as a sleuth on the trail of a villain.
The depth of knowledge shown, and its presentation, is exceptional. The author's years of experience are evident in his self-confident writing style. You will rarely see a clearer overview of numerical calculus. QuibblesThe cover of the book states: "Do big math on small machines." This, combined with the Real-Time Programming phrase in the title, might lead one to believe that the book's primary audience is intended to be the embedded microcontroller crowd. Sadly, not so. There is very little here for the die-hard assembler programmer other than some very handy integer square root and sine routines - and these examples are in C++. Based on the cover, I would have liked to see a greater emphasis on processors lacking a floating point unit. Also, some code examples in pseudo-assembler would have been welcome, as the author chose C++ as the language of choice for all examples.
CrimesAs is so often the case nowadays, there are various typographical errors scattered throughout. This seems to be an epidemic in current technical books. Fortunately, it didn't affect the readability of Math Toolkit.
ConclusionsI believe Math Toolkit for Real-Time Programming would be a great, perhaps mandatory, addition to the bookshelf of anyone that is involved in writing code that has a heavy math component. Other than the somewhat misleading cover, I cannot find anything truly negative to say about this work. Congratulations are in order to Mr. Crenshaw on a job well done.
The book also includes a CD-ROM of all example source code. In reality, to get the best benefit from the book, you should mostly ignore the CD-ROM and work through the examples. To quote the author: "Never trust a person who merely hands you an equation."
Table of Contents
- Getting The Constants Right
- A Few Easy Pieces
- Dealing with Errors
- Fundamental Functions
- Getting the Sines Right
- Arctangents: An Angle-Space Odyssey
- Logging in the Answers
- Numerical Calculus
- Calculus by the Numbers
- Putting Numerical Calculus to Work
- The Runge-Kutta Method
- Dynamic Simulation
- Appendix A: A C++ Tools Library
I received a review copy of this book from the publisher. Thus, my loyalties and opinions may be completely skewed. Caveat Lector.
You can purchase Math Toolkit for Real-Time Programming from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Back in the day, a book like this would have been a real life saver for those of us slugging it out with brain-damaged operating systems (e.g. MS-DOS). From things like MIDI sequencers to guidance systems, the need for real-time speed was a real issue.
However, with the the maturity of operating systems, many of them now include device drivers, APIs, objects and other goodies that insulate the average programmer from the hassle of issues like latency. So my question is, other than good academic study, would it pay for the rest of us to spend the $$ on such a book?
Though I admit, having to write my share of real-time apps back in the day has me curious enough to put the book on my wishlist.
--- have you healed your church website?
I remember when having a solid math background was de reguire for a programmer. Of course, I'm talking the mid 80's, engineering school and Fortran, so I'm kind of krufty.
I wonder how much better could we be if coders knew basic math, if they know how those little bitty chips actually computed the sine of something instead of assuming it works. We would probably have rock solid operating systems without all the glitzy GUI stuf..
How does the book compare to the classics: Numerical Recipes, The Art of Computer Programming, etc?
I'm surprised to see it posted on /., though, because he's pretty harsh towards the gaming community. In fact, he says near the beginning that game-related technology in CPUs (MMX and so forth) is taking away much-needed brainpower from research that should be reaching towards making chips do more math per unit time (not to mention driving up production costs for toy-obsessed, joyless loners). He calls for an immediate end to the pandering that Intel et al do to get into the pocketbooks of the socially-inept, technology pseudo-elite and wants real reform in the area of empowering science.
Powerful stuff.
Funny this topic should come up. I just did a 'Store Locator' for the company I work for (I'm the IT Manager, belive it or not). All I have is your basic HS diploma, and in creating the search, I realized I don't know a damn thing about sine and cosine. I don't know how they're used, or how they're applied. I have a feeling that they're somehow related to geometry (which makes sense, seeing I have to get a distance between two points on a curve - the earth), but I'm not sure.
Sure, it's probably taken me longer to write this post, than it took to find the php code I used as a basis for the search, but how much math is REALLY needed overall?
I slept through school, I did really bad, all because I felt it was worthless. I did feel that my business class, business law, and basic Algebra has been useful. But overall, it wasn't worth my time. Hell I had a physics teacher who'd pick on me because I was flunking (it's amazing what good test grades + 0 homework does to you), but I just found physics interesting - jeez, it was only HS. I was testing the waters, not padding my GPA. I believe that's what's HS is FOR.
And if you KNOW what you want to do (I knew I wanted to fix/program computers when I played on my Apple ][ in 6th grade), what the hell is college for?
The ease of the internet sure hasn't helped my perception.
Am I the only one?
"I can't give you a brain, so I'll give you a diploma" - The Great Oz (blatently stolen sig)
When I went for my Comp Sci bachelor's I was amazed at how many math-phobics there were in my Comp Sci classes. As part of earning the degree you had to take 4 specified math classes (Calc 1 & 2, Linear Systems, Probability). You only had to take one more math class to get a minor in mathematics, Calculus 3.
Now I've always been big on math but I was kind of surprised at how few people were willing to take a single class to earn a full-fledged minor.
The Forth literature contains many examples of high-performance hardware-integer-math-only routines. A core feature of most Forth algos is rescaling to a power of two space at the start of the algo and from it at the end. This allows bit shift operators to do their stuff. It can take non-trivial fiddling to rescale algorithms - hence, it's nice to just look them up.
Unfortunately, it's tricky to find Forth books these days.
That's a shame, because along with Smalltalk, Lisp and APL, I think Forth is one of the "mind expanding" languages all programmers should at least experience, instead of just deciding C/Java/C++/VB is the one true language.
A year (or so) ago I attended a lecture given by Guy Steele (of Lisp/Java/ Crunchly fame) on his proposal to alter how IEEE floating point numbers are mapped to real numbers. It quickly flew over my head, but gave a great insight into the whole field. Steele then had a fair old "discussion" with the one person in the audience whose head hadn't been overflown (sic), as there was plainly still much controversy left in this area. On trying to do some "why didn't I get this stuff at college" reading, I found there wasn't a great deal of literature.
The reviewer's concern that coprocessor-less systems should be covered is valid, but I'm not sure going as far as assembly is necessary. For example, I once had the privilige of reading through Hitachi's libm implementation for their H8 series microprocessor/microcontroller (one would be generous to call H8 a 16-bit system, and ungenerous to call it an 8-bit system). With one small exception (I think the cos table lookup) the whole thing was in (quite readable) C, and (at least for basic libm stuff) performance was perfectly acceptable. For didactic purposes, a C (or sane C++) implementation would be the thing one would want to find in a book - I get very annoyed at embedded books where the examples are written in asm for the author's favourite (obscure) microcontroller.
## W.Finlay McWalter ## http://www.mcwalter.org ##
Embedded != Assembly coding.
To be honest, a lot of embedded coding is done with C or C++ these days. I've been following Crenshaw's articles in Embedded Developer magazine for years now. He explains a lot of what they try to teach in college Calc, etc. in simple, practical terms, and reduces it to usable algorithms.
I'd probably buy the book and add it to my shelf.
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
But to get a job writing computer graphics software, or audio processing, or designing any sort of embedded hardware, knowledge of advanced math is required. The people who want to do this kind of work pursue higher educations, and if they enjoy what they're doing then that's great, too.
Karma: Bored. (Thinking about resurrecting the "Anyone else is an imposter" joke.)
I graduated as a Math/CS double major from Drake University, where almost all CS majors also got a Math degree because the CS prereqs covered all but 3 of the Math prereqs. It has actually helped me enormously as a programmer to know math: in the past month, I've needed transformation matrices, sine/cosine stuff, and a bunch of other things that, granted, could have been lifted verbatim from Google groups, but it's often faster (and the code is better) if I just do it myself.
I've been programming since 1968, and very little had anything to do with math. People give me the same line, wow, I'm no good with math, I couldn't program, and don't believe me when I say computers add and subtract, multiply once in a while (array subscripting usually), and hardly ever divide.
Scientific or engineering programming, they need the math because they are math programming. The rest, forget it, maybe you add some numbers for a shopping cart, multiply for sales tax, but programming has little use for math.
I learned long ago that when an 8 bitter needs trig functions, you use a look up table generated externally.
Infuriate left and right
As for the title, I agree it's a bit misleading. The book has pretty little to do about real-time (in fact nothing, as far as I could see). What it really should be called is "Computer arithmetic and a little of numerical methods for dummies". This book will help you understand how to write your own libm, and give you some ideas for more advanced tasks, but that's about it.
For me, who didn't know much of this stuff, it was very interesting. It will probably not save you that course in numerical algorithms (which I for one haven't taken), but even then, it will probably contain some interesting tidbits you didn't know.
On the other hand, if you have years of experience in writing computer math routines, it will probably quickly become dull, but that's true about anything you already know.
I won't bash you like some of the other replies to your post, nor will I give you hope that you can advance past a limited set of jobs in the IT industry.
College (esp for computer engineering and CS) fundamentally teaches you:
1. How to solve problems
2. A toolset (ie math, algorithms) to go about solving those problems
True, you may not ever use calculus, but as a computer scientist you will use matrix theory because it is the best way to solve some problems.
This is not only for scientific/research either. If you try to write anything performance related, you'll have to use higher math. Computer science ain't easy.
Let me stress again that college teaches you about your subject matter and how to solve problems for it. You can come up with this stuff by yourself, in my experience only a tiny percent working without a college degree will ever accrue enough to offset what they missed in college.
Why, o why must the sky fall when I've learned to fly?
Bn.com has this book listed at $49.95. Amazon has it for $34.97
Save your self some money!
As a biz-app programmer, I am bothered by too much attention given to floating point math and not enough to decimal math. A decimal-centric approach would give better results for monetary calculations, because any truncation and rounding are at decimal (base-10) boundaries instead of base-2 boundaries. It gives results more like one expects doing it by hand on paper, which shapes most peoples' perceptions of what they expect (and the customer is always right, even if they are boneheads).
The only library I know that supports it is the BC-library sometimes used with PHP. (Well, I guess you could say that COBOL has such also.) It actually uses strings to hold the results so that there is no machine-based limitation on precision size. Plus, that improves its cross-language use since almost everything supports dynamic strings these days.
(Not the fasted approach I suppose, but most biz apps are not math intensive anyhow. Most code is devoted to comparing strings, codes, and ID's and moving things around from place to place. IBM used to include decimal-friendly operations in its CPU's. Those days seem gone for some reason, yet biz apps are still a huge domain.)
Table-ized A.I.
FWIW, There's a lot of people out there who don't think too much of numerical recipes: http://math.jpl.nasa.gov/nr/
Your code may do the job, but does it do the job efficiently? And if it didn't, how would you know?
I changed majors from CS to Mathematics halfway through because I realized that programming is easy; you can always learn a new language or a new technique by picking up the appropriate O'Reilly book on the subject. But writing good programs -- programs that are robust, that scale well, that do as much as possible as quickly as possible -- is really applied math. And math is hard.
You simply have no idea how much you don't know, and with the attitude you have, you probably never will.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
A year (or so) ago I attended a lecture given by Guy Steele (of Lisp/Java/ Crunchly fame) on his proposal to alter how IEEE floating point numbers are mapped to real numbers. It quickly flew over my head, but gave a great insight into the whole field.
Steele is God. He also invented Scheme, wrote the original Common Lisp manual, co-wrote with Harbison a classic reference manual for C, and wrote parallel languages for the Connection Machine.
On trying to do some "why didn't I get this stuff at college" reading, I found there wasn't a great deal of literature.
This is widely considered a good introduction.
But by "math" the reference is almost always to calculus.
But math is not just calculus.
Math includes (and this is a MINIMAL list :
- algebra Every program using symbols to represent things that might vary is using algebra. Algebra isnt just manipulating big expressions to find values of x and y - it is really about using names to refer to values. (For example x=y+1 is fundamentally an algebraic expression.
And there's more - check out Glassner's "Digital Image Synthesis", or Knuth's "Art of Computer Programming", - find places where mathematics is not mentioned. Let alone such things as wavelets, the Mandelbrot set, grammars, text (or UI) layout, automata (and on, and on, and on...). I can show you a very hard mathematical problem (which I'm still working on) based on an algorithm you all know, but that is often coded incorrectly.boolean logic Using logical expressions and understanding what they do is just the predicate calculus. Using logic languages (prolog primarily) is, well, logic.
Linear Algebra Try to program more than minimal graphics without linear algebra.
The structure of numbers computing square roots and the like. This kind of computing also typically involves calculus and its relatives
Calculus many parts of computational mathematics, including things like square roots, sin/cos and the like. Also, finding tangents and normals to surfaces which is a big part of reflection models in graphics. The logic involved is also used in the analysis of algorithms.
logical reasoning Every time someone writes a loop or a recursive function, they are essentially using mathematical induction (albeit informally). Propagation of pre/post conditions (not just in procedure calls, but on the statement to statement level is also logical reasoning (and informal proofs).
Fourier analysis Fourier analysis is essential in image manipulation (including compression), graphics in general, Most algorithms involving sound processing also rely on fourier analysis
Graph Theory Where doesn't graph theory show up? Dependency graphs, path algorithms of all sorts. Trees are graphs. Garbage collection involves graph theory. Programs are (on several levels) graphs. The internet as a network is a graph. Websites are graphs (and it can be interesting and revealing to look at them as such).
Number Theory Cryptography!
If you're not doing any of these things, you may be programming, but you're probably not programming well.
Juris Hartmanis said (half jokingly) in his Turing Award lecture that "Computer Science is the engineering of mathematics" I think its about as good a definition as any I have ever heard.
Well, see, I'm a database guy too. I split my day pretty much equally between SQL and PHP. A lot of people may not consider that "real programming." I do, and in fact I've done some pretty heavy-duty scientific application programming in that past, and I'm here to tell you, I use my math skills just as much now as I did then. Because I don't just write queries and interfaces; I write them with absolutely fanatical attention to detail, and I subject everything I put up on my company's server to the kind of rigorous scrutiny I learned in set theory and algorithm analysis classes. And as a result, it works, and it works well, and on the rare occasions that something doesn't work as well as it should, I know how to fix it. The bigger applications get, the less meaningful the 90/10 theory really is -- for very big applications, a whole of bunch of seemingly trivial speed and scalability tweaks add up to big improvements down the road.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
I used this book as one of the references for my Game Developer Conference course on "Faster Math Functions", and the book is good but has holes. Crenshaw's style shows his crusty old engineer roots at times - his coverage of Mininax polynomials is way behind the times and he seriously needs to get into Mathematica or Maple as his basic high-precision tool.
Work by Tang on combatting destructive cancellation in range reduction, the new semi-table based exponant and log methods, Intel's research into using Estrin's Method based SIMD for evaluating polynomials or Muller's book on Elementary Functions are beyond Crenshaw's experience, and it shows. This is a homebrew book rather than an introduction to the state of the art. More information at SCEA R&D Website.
Jack should just about be a household name. When Apollo 13 went berserk, NASA dusted off some calculations of trajectories that he had previously done (he was no longer working for NASA) and used one of them to bring the astronauts back.