Slashdot Mirror
Secure, Efficient and Easy C programming
cras writes "Feeling a bit of masochist today.. First in the morning I wrote Secure, Efficient and Easy C Programming Mini-HOWTO. And since I already spent a few hours with it, I figured I might just as well see what Slashdot people would think about it."
Pick any two.
"First in the morning I wrote Secure, Efficient and Easy C Programming Mini-HOWTO..."
Damn. What are your plans for the rest of the day?
"First in the morning I wrote"
So did you wake up early this morning, or are you still up from the night before, like me?
-- "Government is the great fiction through which everybody endeavors to live at the expense of everybody else."
It does look like a good start, add a few more chapters and you will be halfway there...
1) Use python with C bindings
Why not fork?
It's kind of funny how this guy voluntarily slashdotted himself by submitting an article with a link to his own site, crashing it instantly. :)
I found strlcat and strlcpy easily ported - simply toss them in the same .c file and dump it into the makefile!
;) Eh, I suppose we all have a certain way of doing things that we don't wish to part with. (*points at the unsafe buffer people*)
On a more serious note, why in Bob's name don't these two functions exist, standard, in Linux? IMO, they should be added, and gcc should give deprecation warnings about the use of non-safe buffer handling functions - sprintf, strcat, strcpy, etc. No offense to purists, but screw the standard. I'll sacrifice some portability of software and such for security.
Oh, and on a side note, you may take my malloc() when you pry it from my cold dead fingers.
in that folks who use C can avoid common pitfalls. But so much of this seems like it has been tackled by C++. Only C++ did it cleaner. C++ is complex though. So this only leaves (horrors) a higher level language that removes all of these implementation details that lead to insecure programs.
Do it in a higher-level language first. Make sure your algorithms are clean and efficient. If and only if you see a performance or resource problem do you rework portions(!!!) in C. As a bonus, the higher level language acts as a code template for faster C development.
Once you are at that point, this Mini-HOWTO will definitely be a great resource to use.
- I don't need to go outside, my CRT tan'll do me just fine.
The way it works is simply letting the programmer define the stack frames. All memory allocated within the frame are freed at once when the frame ends. This works best with programs running in some event loop so you don't have to worry about the stack frames too much. Here's an example program:
That sounds a little like the NSAutoReleasePool in Cocoa/OpenStep. Objects use reference counting, when the count reaches 0, they deallocate themselves. When an object is created, it can get added to the most recent pool. When the pool is deleted, it decrements the reference count of all the objects within it, causing deallocation unless it needs to be kept around longer.
Do you even lift?
These aren't the 'roids you're looking for.
I'm going to start putting that at the end of everything I write so that people can't criticize anything I do. As a matter of fact... I think I'll only write on Sunday mornings after not sleeping the night before. It seems like it's always Sunday morning anyways.
Sex - Find It
But I can't follow this HOWTO very well. Does he have a global variable stored in the file with t_push and t_pop so that t_sprintf can use that variable? But if he has a global variable, than all he's really doing is allocating the maximum amount of memory his program will ever need at the beginning, and managing his memory.
Perhaps working until 4 in the morning on C code has drained my ability to understand.
Some guy spent a couple of hours writing a first draft of a Howto. Thanks Slashdot, I'm sure glad you didn't let this one slip through the cracks! Besides, who cares about these kludgy ways of handling memory. If you don't wan't to worry about memory allocation use C# or java or something. Otherwise, stop eating quiche and write solid code.
"There's a madness to my method." -mthed
it starts off with denouncing GC as oldfashioned, and then proceeds to tout stack-based allocation, which has been available for ages as the alloca() function (which also has portability problems.)
imho, you should use the Boehm Garbage collector, unless you have code that must be guaranteed to be free of space leaks.
Han-Wen Nienhuys -- LilyPond
"Secure, Efficient and Easy C programming in 24hrs"
Did you really read the strncpy and strncat manpages?
To both zero-terminate and check for truncation is arcane, that's why the OpenBSD ppl made strlcat and strlcpy in the first place.
There are already other secure programming faqs, though AFAIR, they suck too. If I were you, I'd put a HUGE disclaimer to take this page as work-in-progress.
(before flaming, write down the correct code to check for truncation for both funcs)
Okay, let's preface. This guy has a good idea in the memory allocation department.
Problem 1:
It's not easy, nor fast to write. Errors are severe if present and undetected. Code required to be reliable might not be a good place to test this allocation method.
Problem 2:
I'm not entirely sure these concepts are very portible outside of GCC. May not be a big deal to most, but uh, multiplatform code is required in some enviroments.
Problem 3:
Any speed increase without massive resource wasting is pure dumb luck during heavy usage, unless used in an application that takes little user input or has limits on the ammount of input.
Just my $0.02.
Some of my personal favorites include:
- Exceptions in C. You can get quite natural-looking exception handling in C, with some convoluted macros. I'm sure most hardcore C coders have come up with their own implementations. Many security bugs happen in parts of the code that handle errors, precisely because errors are rare, and those parts of the code don't get tested well. Using a unified, exception-driven approach to error handling can cut down the risks. IF you do it right.
- The alloca() function. This allocates memory directly off the stack, which is freed when the function returns. Very useful for cases where you want a stack buffer but aren't sure how big it needs to be. Like any other stack buffer, you need to take care not to overflow it. There are portability concerns with this function, but it can still be useful.
- Variable-sized block-chained allocators, which pull chunks of memory out of preallocated segments. The segments are chained together in a linked list. Very effective when you need to make a lot of variable-sized allocations, and do it fast, dammit. It also makes freeing the allocated memory blazingly fast, although it's a "free all or none" approach.
- "Hardened" allocators, which allocate blocks in multiples of the page size, and set memory protections in such a way that buffer overruns cause crashes. This is the easiest way to prevent ANY kind of buffer overrun vulnerability, but wastes memory. See Electric Fence.
Look people.. It takes a keen eye and major discipline to write secure C code. It is not impossible. You have to get in the habit of subconsciously checking yourself at EVERY turn. "Am I accessing a stack variable? Am I doing it CORRECTLY?"DISCIPLINE, DISCPLINE, DISCIPLINE. I fully expect to see the usual barrage of comments to the tune of: "C is outdated, insecure, brittle, yadda yadda..." No. Some PROGRAMMERS are "outdated, insecure, and brittle."
The C language doesn't write bugs. Programmers write bugs. If the programmer can't handle C, then take it away from him. But don't try to take it away from ME.
See HeapAlloc and friends in Win32 for proper implementation.
At any rate, there are better ways to make sure one never leaks memory problems:
1) always set a freed pointer to 0. Most architectures have a predictable behavior in dereferencing a 0 (throws an exceptions).
2) Limit all malloc/free pairs to the same function. If a function just has to allocate and return some buffer, give it a meaningful name to that effect and all a corresponding free version. Then, you can follow the above rule.
3) assert()s are your friend. Use them religiously. They can always be shut off.
4) Use memory tracking software (purify) before ship.
Yes, it's easier to shoot yourself in the foot with C, but you'll gain a huge performance increase. It's all about using the right tool for the right job.
int func(int a);
func((b += 3, b));
why not just use visual basic?
say what you want about it, you don't have to use stupid hacks to avoid buffer overflows.
First off, C++ objects can force the use of all data access through assert()-filled methods, then in optimized mode can be inlined and thus reduced to their C equivalents.
Second, destructors in C++ guarantee clean up of objects, regardless of how you leave scope (natural, return, exception, etc).
Finally, you couple destructors and reference counting auto-pointers, and you have yourself a very nice allocation API that's as easy as Java, but without the performance or unnatural destruction logistics.
In my last project, I used glib from the ground up. I wrote several thousand lines of code before testing it. I made some very aggressive use of glib and gobject. After the code compiled and did not give any runtime warnings anymore it did not contain a single memory leak (verified using valgrind).
glib containts a lot of useful things: lists, trees, hash tables, memory pools, string handling functions and a lot more, everything thread safe.
gobject contains tools on top of glib like "classes" and "objects". It's not the same as in C++ or java, but also very useful. Runtime classes oder data types, generic object properties, reference counting, signal callback, runtime type checking, etc...
The code ist now full of g_... and it took longer than usual because I had to read the documentation, but I think these libraries are very great, and provide a solution for nearly everything that has to do with abstract data types and dynamic memory allocation.
And it's very lightweight, fast and efficient.
Perl is for idiots who think regexps can solve all problems.
s/idiots/wise souls/
s/think/know/
Problem solved.
I write in my journal
I still have yet to write a single useful C program that I couldn't have done in Perl.
Can you write a video driver with acceptable performance in Perl? Can you write programs that do things other than text manipulation, such as (say) a 3D engine and make them faster in Perl than in C? Remember that in the real world, time is money because a shorter execution time means lower system requirements and thus a larger market for mass-market desktop applications.
Will I retire or break 10K?
Well, not quite. An NSAutoReleasePool does not allocate a large region of memory and suballocate objects out of that. What an NSAutoReleasePool does is make it possible to avoid explicitly sending the release message for temporary objects.
For example, from Foo() I allocate an NSObject with [[NSObject alloc] init] and pass that as an argument to Bar() which takes ownership of it. However, I must then ensure that I release the object because Bar() is following good coding practices and retains it, so thus with alloc+retain it's reference count is now 2. So instead what I do is Bar([[[NSObject alloc] init] autorelease]) which allocates NSOjbect (with ref count one) initializes it, marks it for autorelease, and passes sends it to Bar() which retains it (ref count 2) and keeps a pointer to it (presumably it is a method of a class). Coming out of bar the ref count is now 2, and perhaps Foo() proceeds to do some other things. Presumably at some point higher up the call stack (or perhaps at the beginning of Foo()) an NSAutoReleasePool was allocated. At the corresponding exit point (either at the end of Foo() or the end of whatever higher up function) [whateverpool release] will be called. When the pool is released, it will call release on any objects it has been asked to take ownership of. At this point one of two things it true. Either the class that Bar() belongs to has already released the object and thus its reference count went back down to one, and now is going to zero (so bye-bye), or the class that Bar() belongs to has not released the object and doing this release merely brings the refcount back to one such that when the other owner releases the object, its refconut will be zero and it will be freed.
Sorry if that was confusing, but in reality it's really not. It also really helps out when you are coding functions that allocate ObjectA, then allocate ObjectB, then ObjectC, and then find out something is wrong and need to "roll back" to the begining. If you allocate an NSAutoReleasePool at the beginning, and autorelease everything you alloc then if you error out you can free the release pool and everything gets released. If you don't you can simply retain what you need and then free the autorelease pool.
Anyway.. what this guy is REALLY talking about is NSZone. NSZone allocates a chunk of memory which other objects will be allocated from. The caveat being that while the memory will be freed, the objects will not be properly destroyed. Now this guy was talking about holding C strings and the like, so this is not a problem. However, had he been holding some C++ or objective-C objects this would be a problem as none of the destructors/deallocators would ever be called.
I think what it all boils down to is that programmers need to read more code than they write and that we should really be getting Masters of Fine Arts in Programming. I completely agreed with what Dr. Gabriel said. Programming is about as much like building a bridge as writing poetry is. That is to say.. not much.
Going along with that thought, I think it should be pointed out that /EVERYONE/ here who programs in any language (but specifically C programmers, and ESPECIALLY C++ coders) needs to learn Cocoa and Objective-C. I imagine some of the C++ whiny bitches are going to continue to whine about how much easier and better C++ is, but for those of us who actually prefer to wrangle pointers, Objective-C is where it's at. It's like C with JUST enough object orientation, but not overdone in some committee like C++. Also, one should note that I do like C++ quite a bit, but sometimes there's too many provided ways to do things. With Objective-C, the provided ways are almost all good. In addition, like C or C++ you are not limited to doing it that way, it's just that Objective-C only makes it easy to do good things.
Think for example of wxWindows vs. Microsoft MFC. wxWindows is suprisingly similar to Cocoa (although wxWindows does not do ref counting so making sure that one and only one class ever owns an object can be problematic at times). MFC, on the other hand, is rather a bear to work with as Microsoft has written it such that an MFC programmer /can/ do things multiple ways, none of which work very well. Obviously this is a generalization, but I think the average MFC programmer will understand where I'm coming from here. That is, again, except for the whiny C++ and MFC bitches who can't figure pointers out. Go home!
I think the HOWTO should have a reference to obstacks, rather than claiming data stacks are a new invention. (Hint: data stacks have been used many, many times in many, many projects. GNU obstacks are the only one for which I can find a URL at the moment.)
Patrick Doyle
I mod down every jackass who puts his moderation policy in his sig. Oh, wait a sec....
The main problems with it versus broader garbage collection schemes are circular references and overhead.
If two (or more) objects have a reference to one another, the count can never reach zero even if nothing in the main logic points to those objects anymore.
Also, every time an object gains or loses a reference, a check for a count of zero is made. In fuller garbage collection setups, periodic checks are made to all of the objects in a low-priority thread. In some cases, memory usage can be higher, but performance is also higher sometimes and it can handle circular references.
Both are better than repeated use of malloc/free and new/delete though.
--
C also muddies this concept because there are no objects in C.
- I don't need to go outside, my CRT tan'll do me just fine.
From the FreeBSD manpage:
ALLOCA(3) FreeBSD Library Functions Manual ALLOCA(3)
NAME
alloca - memory allocator
LIBRARY
Standard C Library (libc, -lc)
SYNOPSIS
#include <stdlib.h>
void *
alloca(size_t size);
DESCRIPTION
The alloca() function allocates size bytes of space in the stack frame of
the caller. This temporary space is automatically freed on return.
RETURN VALUES
The alloca() function returns a pointer to the beginning of the allocated
space. If the allocation failed, a NULL pointer is returned.
SEE ALSO
brk(2), calloc(3), getpagesize(3), malloc(3), realloc(3)
BUGS
The alloca() function is machine dependent; its use is discouraged.
FreeBSD 5.0 June 4, 1993 FreeBSD 5.0
Regarding scanf(3), many people don't realize this is Bad:
scanf("%s %s", cmd, arg);
This is Good:
scanf("%79s %79s", cmd, arg);
This prevents a buffer overrun if a word contains 80 or more consecutive non-white characters.
Ditto for sscanf(3) and fscanf(3). Never forget the N+1 when declaring the arrays (eg. char s[80] vs %79s) to leave room for the NULL.
Here's a good command to run on all your .c files to find such problems:
And in a document like this, *definitely* point out the whole gets(3) problem; the granddaddy of them all. Never use gets(3), period. Use fgets(3) instead.
The gets(3) interface is inherently insecure; a problem waiting to happen by its mere existence. Any code that uses it is broken.
There are probably some others (someone mentioned strcpy) I'll try to post more if I think of them.
Some of the idea's aren't bad (and those have been done before), but mostly it's just another simple dynamic string library in C.
As for efficency...
...this pretty much speaks for itself. Why Is strconcat() so efficient compared to just doing strcat() multiple times? Because you've got a model for representing the data that has ZERO metadata, and a model for storing the data that requires you to reallocate bits of memory all the time.
Assuming you can just disacount all this overhead by using memory pools, is a simplistic outlook (for instance even if you waste gobs of memory so you don't have to call malloc that much you'll still need to do copies all the time)
There are more than a few much better string libraries out there for C. Probably the best for an IMAP server is probably Vstr as that was deigned to work well in an I/O context (For instance it doesn't need strconcat() like calls in the API because doing repeat adds is just as fast).
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
Why is this bollocks?
To summarise the articles: a bunch of small libraries providing object-based memory allocation and string handling.
Kudos to the poster for enabling himself to write code in a way that's good for him. But that doesn't mean it's good for anyone else.
For example, I'm not going to go and learn 20 more function names and have more library dependencies and I wouldn't recommend anyone else does either.
Finally, suppose one wants a better string library or memory library. There are already plenty of good, with-much-work-done-on-them, open source libraries out there. Tried and tested. Not to mention the C++ STL.
Pick one that means many other people will also be able to read your code and be familiar with the libraries you use! There's nothing I'd hate more than working on a project written by someone using these libraries. Not only do I have to analyze the code, I have to analyze these libraries, and also manage to keep them and their quirks in my head while I am reading the program. Yuck.
Of course Bjarne Stroustrup would say this, but he has some nice examples backing his statements up, too. See his FAQ and his paper on "Learning Standard c++ as a New Language".
Stroustrup explains some nice details on especially this issue of memory constructs. He makes a convincing argument for why C++ is easier for C-style programming... Especially for those of you (One I saw below) who "Don't want to get into c++", realize that you can edge into it pretty easily, and accomplish your tasks more easily and quickly -- give it a try!
Guess what? I got a fever! And the only prescription.. is more cowbell!
- O'Caml is a marvel of strongly typed object orientation, but you'd hardly know it from using it -- there are almost no C-style type declarations; as a ML child, O'Caml uses type inferencing to prove powerful assertions about program validity and improve programmer convenience. It's compiled! And if you watch the ICFP's, you might note that it consistently beats C compilers for speed of execution. '92, if I recall.
- I never really bought OO, so S/ML is fine by me. Still compiled, since 1984.
- And they both descend from ML, started in 1973.
- Lisp was compiled in 59 or 62 (mccarthy or 1.5, chose your valid date). But then, I suppose it'd have to be compiled, since the notion of interpreted code hadn't been concieved of yet!
- Erlang is the last, best, word in concurrent programming. If you want to write a high throughput, reliable threaded application, you shouldn't even think of the word 'C'. This broke out of its lab in '87, first compiler in '91.
- Scheme is often thought of as a testbed for interpreted language concepts, but even it can be compiled, and with concepts such as continuations that can actually make a C programmer's head explode! Since 1982, commercial grade compilers have been available.
- Even haskell is compiled, but as monadic programming is less than 10 years old, no one knows how to always write really fast code in it yet. Leave your number, we'll call you in 2034, right before you gear up to deal with your year 2038 rollover crisis.
Welcome to the late 1970's! We look forward to your eventual arrival in the 80's and early 90's. Please enjoy your stay!ps. As modern coding is more about the manipulation of very complex structures, rather than how to say, walk a linked list; a higher level language, with native support for more complex constructs, has the potential for creating much faster applications than something on the level of C. The reason being is that the h/l compiler can reason about, and thus optimize over, larger components than the C compiler.
You can write C code to extend Python. It's a very common technique. The advantage of C, and it is a big advantage, is speed... but most programs don't need that speed advantage everywhere, only in a few intense and heavily-used operations. (Optimize the innermost loop...)
The advantages of Python for almost every other operation are really too numerous to list.
Your point about "right tool for the right job" is well taken. _Good_ Python programmers learn the C extension API, and use it when appropriate. Guido van Rossum, the creator of Python, even states in one of his papers "If you feel the need for speed, go for built-in functions - you can't beat a loop written in C."
It's rare that you're presented with a knob whose only two positions are Make History and Flee Your Glorious Destiny.
You use a hybrid stack. You push two kinds of objects onto this stack: environment frames and cleanup frames. When you use the TRY macro to open a new exception frame, you push an environment frame onto the stack. This frame includes a jmp_buf struct which is filled in by a call to setjmp(). If an exception is thrown, control will transfer back via longjmp(), using this jmp_buf.
Then, as you allocate memory, open files, lock mutexes, or whatever, you register cleanup functions with another macro. This pushes a cleanup frame onto the stack indicating a function to call along with a single argument.
Then, if an exception is thrown, you just pop cleanup frames off the stack and execute them, until you hit an environment frame. At that point you call longjmp() to transfer control to the user-code exception handler, inside the CATCH macro block. That user code can choose to do cleanup or error recovery, or it can THROW again and continue propagating the exception up the stack.
Figuring out how to implement the TRY, CATCH, ENDCATCH, and THROW macros is fun, so I won't give away exactly how to do it. It's simple, and involves a creative use of a do-while statement.
OK, so that subject's a little inflammatory. But my god, I don't see why anybody is still writing new code in C in this day and age. C++ has been a fast, stable, standardized language for what - 10 years now? All the problems with buffer overflows that require hokey, kludgey workarounds in C are cleanly solved with any well-written string library (like, say, the one in the STL). Memory pools can be nicely wrapped with a class that pushes in the ctor and pops in the dtor, so you don't have to remember to call them in the right order everywhere (just declare an object at the top on the block).
The arguments I've seen against C++ seem to fall into the following categories:
* It adds bloat and it's slow
No, not since optimizing compilers were perfected in the 90s. You can add a lot of overhead to your app by abusing the STL, but for non-trivial applications, you'll never notice it. GCC (at least for the pre-3.0 series) has a really unoptimized template implementation, where "Hello, world" using cout would make a multi-megabyte executable (and be forever compiling it), but more modern compilers, like VC++ and Intel's compiler, do a lot better. Either way, for a real-world app, any size increase will be unnoticable. As for speed, with an optimizing compiler and judicious use of inlining, a C++ program will run just as fast as one written in C.
These complaints may have been true in the days of the Cfront preprocessor, but not today. I don't know about you, but I no longer write code for a 386 with 4 meg of memory.
* I don't like/need/want to learn OOP
You don't need OOP to use C++, but it helps. A class is just a struct where everything's private by default. If you know C, it takes about a day to learn the basics of constructors and destructors, references, and exceptions. Templates and STL will take a bit longer. One great about C++ is that you can just use small bits here and there if you don't want a full-blown OO program.
* It's not as good as Perl/Python/Ocaml/Eiffel/Java/whatever
That's not the point. It's not supposed to be. It's supposed to be as good or better than C. If you want a standalone-executable without linking in a complete interpreter and you don't need a lot of string parsing or regexps, you were using C anyway.
* It won't let me write libraries that work with other languages
Just declare all of your external APIs using 'extern C' and make sure they only use C types in their signatures. Done.
The main reason not to use C++ in new development seems to be "I don't want to learn it" or "I don't know anything about it". If you use either one, I don't ever want to work with you.
What if life is just a side effect of some other process and God has no idea we exist?