Slashdot Mirror

Most automated scans will not take the time to scan for open ports. (that I have experienced)

Also consider FTP with SSL / TLS like what can be done with vsftpd. http://vsftpd.beasts.org/

As mentioned elsewhere in this thread consider using Fail2ban which is easily configured for monitoring failed attempts at connecting to your server and can then block the IP after a configurable threshold is reached.

vsftpd has always been excellent for me--you can limit the anonymous user in a number of ways, and by default it's extremely locked down (may not even be enabled at all, can't remember, but if it's disabled, even when you enable it it's still very locked down).

ProFTPd, while great when it came out in the early 2000s and had many features, was always plagued with security issues, and many places I worked at stopped used it even though it was relatively easy to setup and was feature rich.

vsftpd was at one point the most secure ftpd I could find. Not sure how good it is anymore.

If you need a simple Windows FTPd on a small LAN for quick file transfers check out ftpdmin which is really minimal (free and source available!). I use it all the time for quick ISO transfers on my personal LAN. Does the job. Much better than HTTP transfers, that's for sure.

The idea that software that I have no opportunity to audit runs with my privileges when I run it is fundamentally broken. There is no way to clean malware that had a network connection long enough to hide a trojan anywhere and then overwrite itself to appear relatively innocuous. While a complete redesign of the way permissions and permission-delegation would be ideal, it is not practical in the short term. Using systems like http://plash.beasts.org/ can help.

By then, people might be running VM's within VM's to avoid this kind of ... stuff.

Or perhaps, there will be 5 different lightweight sandboxes like Plash, with each sandbox scanning for attempts to exploit known vulnerabilities of the other sandboxes (this wouldn't prevent some people from getting owned, but it would expose malware quicker).

There was a conversation about this last month on the FD.o mailing list

> Mark Seaborn wrote:
> > I have written a hack for Gtk which replaces Gtk's usual file chooser
> > with a file chooser implemented in a separate process.
>
> Nice, this could be useful. I'm not sure yet if there is a concise
> in-process solution. If not, I'll have a closer look on your code.

The current version is an LD_PRELOAD library that replaces some Gtk
API calls. It works with a lot of applications, but not with Firefox,
which dlopen()s libgtk. The successor is a work-in-progress; it's a
patch to Gtk, which you can find here:
http://repo.or.cz/w/gtk-with-powerbox.git (code)
http://plash.beasts.org/wiki/Story20 (notes)

The intention is that the Gtk code will be separate from the RPC
mechanism used, which is implemented in libpowerbox, of which there
can be different implementations.
For example, there is a DBus implementation:
http://svn.gna.org/viewcvs/plash/scratch/powerbox-dbus/
and the intention is that there will also be a version using Plash's
object-capability comms protocol.

Maybe I should rename libpowerbox to something more descriptive such
as libtrustedpathfilechooser (that's a bit long-winded though).

> Is there a chance that you'll give me permission to "reuse" your
> code under the terms of the MIT-License?

I'm open to the idea of relaxing the licence, but which bits of code
do you have in mind? The Gtk hooks are quite Gtk-specific.

Mark

I hope that it happens, too.

I assume you mean things like shared dlls.

And IPC. FYI, There is nothing inherently dangerous about sharing DLLs, that memory is either read-only or Copy-on-write. There is much less likely to be a bug in something as simple as shared memory handling code than a VM.

access to users documents, Direct X, OpenGL, screenshots (GIMP)

access to users documents: VMs don't give access to user documents or systemwide screenshots by default. This would lose much of the security of running a VM. Direct X, OpenGL: VMware includes experimental support for Direct3D video acceleration. This feature is not fully functional. Virtual box only supports OpenGL. As far as I know VirtualPC doesn't support either.

And it would STILL be more compatible than what we currently get passed off as "backward compatibility"

Really? Because even Wine offers much better support for gaming than any VM I have come across (which is all I use Vista for). I haven't had any trouble with backwards compatibility with Vista, except with device drivers, and the VMs aren't going to have much use for those. And I have continually hit bugs and limitations with every VM I have come across.

Which sounds more security to you?

Well, myself I'd pick an effective way of sandboxing applications without the overhead of a VM, and then sandbox everything whether it is 64bit or not.

Either you didn't read the entire post, you chose to ignore the part that explains why we would be better off memory wise,...

Maybe because the closest thing is an argument was

Literally at worst your XP application would require 3 gigs of ram. Now consider that you now have an OS that cleanly could access up to 16 exabytes of RAM, the memory limitation would be far worse on the 32 bit native than it would be on the clean 64-bit system running virtual machines as they could load more programs, and each program could have more memory than if it ran native on a 32-bit machine.

None of which explains how running 5 3gig VMs could be more memory efficient than running 32bit code directly on a 64bit kernel allocating only and exactly the memory it needs. And 5*3gig of memory is not all that cheap. Have you considered that it may be you who fails to understand? In any case, this thread has become more confrontational than interesting, and have little interest in spending more time on it.

There is no question that programming in C and C++ requires skill, and that memory management is an issue, and automatic heap allocation and garbage collection is the popular solution to that issue, but there is no silver bullets, and they are always compromises. I Yes, project need to be written with security in mind. There are program with few security hole written in C/C++ but most of the time they use special API for strings, allocation, ... For example vsftpd : http://vsftpd.beasts.org/#security Again, there are no silver bullets. For instance, the Java sandbox is one solution to a security issue. It is not perfect, and it's imperfections lead to a false sense of security. It is ok for developers to be sloppy because garbage collection and the sandbox will protect the user. Sandbox doesn't imply high level language. Even in a sandbox, if it is not correctly configured, there can be security hole. On the other side you can be sandboxing in C/C++ (using the OS). For example using chroot/PR_SET_SECCOMP ...) I think the problem is writing a safe application is really hard and can be tricky. You need to know well "secure" programming and what offer the OS. For the average programmer using high level language can help : it is the writer of the high level language who have managed security, not you. And you can expect them to do better than you.

http://www.mythic-beasts.com/

A page that got slashdotted

http://roughly.beasts.org/

Our account of being slashdotted

http://www.ex-parrot.com/~chris/wwwitter/20040911-more_introspective_nonsense.html

Another page that got slashdotted.

http://ex-parrot.com/~pete/upside-down-ternet.html

That's all on our shared hosting server cluster. A Mac Mini will hold up fine too.

We're no stranger to being slashdotted :-)

Whitelisting an entire app is too coarse-grained. We need to be able to whitelist the actions that a specific app can take. For example... by default, microsoft word should only have read-only access to its own files and library files needed to run. It should have no net access whatsoever. It should have no access to any of my personal data files. It should have no access to my keyboard when I'm typing in another app. It should have no access to drawing on any portion of my screen outside its own window. It should have no way to spoof another app's name in the window titlebar. It should only have read/write access to the single file that I click "open" on to edit.

Since I have to choose the files I want to work with in a file open dialog anyway, we force microsoft word to use a system-wide, trusted file open dialog which is the only way to grant it access to more files.

See http://plash.beasts.org/ and http://www.eros-os.org/essays/capintro.html

I am glad to hear that someone is working on a real solution to this problem, because many people do not even seem to be aware of it. XSELinux sounds like exactly what is needed. More information here.

It should be possible to download and run third party software without putting your system at risk. That game you downloaded doesn't need administrative privileges to run - why should it need them to install? Just laziness on the part of the developers, cushioned by the general expectation that users on a Windows machine will have access to the administrator account.

More fundamentally, if you install a game into C:\somegame why should it have write access to any file outside that directory? If you install a spreadsheet program why should it be able to read any files at all except the standard Windows libraries, its own installation files, and files you explicitly load into it? At the moment there's only one level of privilege - essentially 'everything' - and all applications run with that.

See Plash for an implementation of what I mean.

BTW, Plash and Systrace do exactly that. In fact Plash lets you allow Firefox to open Documents (e.g. on ~/Desktop) if and only if you have selected that file from the (trusted) standard file open dialog box, in principle giving firefox the very least priviledge with no inconvienience.

From TFA: Schroepfer also predicts that security will continue to be a problem "for anything written in native code," such as C and C++. For example, he notes that security problems caused by memory issues have evolved over the years; from stack-based exploits, to heap-based, to null pointer exploits.

From http://vsftpd.beasts.org/IMPLEMENTATION: The correct solution is to hide the buffer handling code behind an API. All buffer allocating, copying, size calculations, extending, etc. are done by a single piece of generic code. The size security checks need to be written once. You can concentrate on getting this one instance of code correct.

Can somebody please tell me, why are we still having this discussion?

BTW, Have you heard of Plash or Systrace?

Unfortunately I don't think that many Linux systems are set up the way you describe, though I intend to make it my personal quest to make sure they are.

Also, have you come across a way of stopping GUI applications taking over other GUI application via the X protocol?

I know that it is possible to run X applications in untrusted mode, but I understand that is still possible for untrusted applications to snoop on other untrusted applications via X, so we cannot simply run all applications in untrusted mode.

Actually, under the the current Linux (and Windows etc.) security model, seperating the web browser from the file browser does not gain much security. Although firefox doesn't need to read or write large numbers of random files across your home directory, this does not mean it cannot if someone takes control of it using a buffer overflow.

We could constrain firefox using a utility such as Plash. If Konqueror were designed with the Principle of Least Authority in mind we could run as a seperate process, each constrained in a different Plash environment, having only the rights that that particular tab needed (web access or file access, not both). This would allow us to have an integrated UI and enforce the Principle of Least Authority as well.

Ideally all applications would be run through something like the Principle of Least Authority shell which limits the applications so they can only access those files they actually need to function.

PLASH is a hack that allows you to do something similar under Linux. E.g. it passes in rights to file opened with the GTK file open dialog so that you don't have to confirm that you want to open a files that you have already "passed in" by selecting them in a dialog box.

OTOH existing "Trusted code" initiatives are the equivalent of delcaring a root-only system secure, because you only let a few hundred "trusted" users log into that machine.

IMHO, Operating systems should move towards the "Principle of Least Authority", that each software module should have the least authority required to function. The ideal way of doing this is for the system to enforce ObjectOriented/Functional/Procedural constraints that all OO, functional and procedural programs define implicitly. For example, if you run "uncompress(const File A,File B)" then obviously "uncompress" does should not be able to access File C because File C is outside its scope. Likewise, if you click on "bunnies.doc" in Konqueror and Konqueror runs "OpenOffice(UI ui, FileSystem::open("~/bunnies.doc")), then OpenOffice should not be able to open "~/.bashrc" because OpenOffice was not passed FileSystem. OpenOffice would only be able to access "~/.bashrc" if it is passed the file by an object which has rights to "~/.bashrc", for example ui::FileOpenDialog. The EROS Operating system is built around this principle.

PLASH is an attempt to retrofit this principle into Linux, and Looks really promising. A program constrained by Plash can only open a file if it is passed it on the command-line or by a GTK fileopen dialogbox.

The problem is that the Unix security model grants the entire set of the user's authority to processes they run. That's why the user's entire /home directory is vulnerable. The problem is excessive authority. Unix needs support for least authority, which incidentally doesn't require a new security model. Unix's existing abstractions can be leveraged. The Principle of Least Authority Shell (Plash) http://plash.beasts.org/ demonstrates that we can do least authority on Unix for files without even kernel modification.

What you are describing is the Principle Of Least Authority. PLASH (Principle Of Least Authority SHell) is a nifty project to tackle this at the application level for Linux http://plash.beasts.org/. HP Labs has a project called Polaris which does this for windows http://www.hpl.hp.com/research/mmsl/projects/adv/p olaris.html.

Also it would be one less function that has to be written and maintained across the thousands of projects that use GTK.

I just add a bit on that list from top of my head.
Although I think the listed app goes beyond what the so called 'average pc user' wants, but there goes...

1. Konqueror ( http://www.konqueror.org/ )

2. Email - Sylpheed ( http://sylpheed.good-day.net/ )

3. I think Evolution is more like in this place.

4. Lately "Sound Juicer" is taking more attention too

5. VideoLAN aka VLC ( http://www.videolan.org/ ) and Ogle ( http://www.dtek.chalmers.se/groups/dvd/ ) [and Goggles ( http://www.fifthplanet.net/goggles.html ) for Ogle GUI wrapper] for DVD watching.

6. There are plenty way to do this, but the typical ones could be 'Jinzora' ( http://www.jinzora.org/ ) and 'MusicPD' ( http://www.mpd.org/ ), even plain Apache does it fine too, in a way.

8. If you want easier to manage iptables wrapper, Shorewall ( http://www.shorewall.net/ ) and there are other wrappers too.

9. KOffice ( http://www.koffice.org/ ) and by individual components, Abiword ( http://www.abisource.com/ ), Gnumeric ( http://www.gnome.org/projects/gnumeric/ ), Gnucash ( http://www.gnucash.org/ )

10. Inkscape ( http://www.inkscape.org/ ) or Sodipodi ( http://www.sodipodi.com/ ) for vector graphics.

11. Miranda ( http://miranda-im.org/ ). Windows only.

13. Hmm , Samba? ( http://www.samba.org/ ), WedDAV (Look parent post), FTP (plenty ftp daemons, ex : http://www.proftpd.org/, http://vsftpd.beasts.org/ etc)

16. GPhoto ( http://www.gphoto.org/ ), EOG ( http://www.gnome.org/ ? ), GQView ( http://gqview.sourceforge.net/ ). The latters are for just viewing mainly.

20. FreeNX ( http://www.nomachine.com/ , http://freenx.berlios.de/ ) http://www.poptop.org/ ), L2TPd ( http://sourceforge.net/projects/l2tpd ), RP-L2TPd ( http://sourceforge.net/projects/rp-l2tp/ )

24. Postfix ( http://www.postfix.org/ ), Sendmail ( http://www.sendmail.org/ ), Exim ( http://www.exim.org/ ), Cyrus ( http://asg.web.cmu.edu/cyrus/imapd/ ), Xmail ( http://www.xmailserver.org/ ), qmail ( http://www.qmail.org/ )

25. Spamassassin ( http://spamassassin.apache.org/ )

26. Same as above.

27. XSane ( http://www.xsane.org/ ) for sane frontends.

30. Buzzmachines ( http://www.buzzmachines.com/ ) I could be wrong...

31. 'various GUI frontends' - X CD Roast ( http://www.xcdroast.org/ ), K3B ( http://k3b.sourceforge.net/ )

32. Don't know any opensource ones...

The group policy (Worldwide) is to have *ONLY* Windows 2003 Servers.
Just was doing a scan on the German network, and the main online reporting server/portal is running Linux ....
Hehe.., either that or one of the clear german IT got MySQL and VSftpD http://vsftpd.beasts.org/ on Windows 2003.., and hacked the TCP fingerprint to show up as Linux 2.4


The Admin is just studit and does not have a clue of that Firefox really is.
You should, (which the Admin-Troll is off getting a coffee, install Firefox on his PC and delete the IE icon from desktop and startbar, and rename the Firefox one to "Internet Explorer" and change the icon to the stupid "E".
He probally will not even notice.....
P.S. His password is either "null" or print on a post-it on his monitor......

"Clutch my testes, bloody squirrel humpers!!" -Happy Noodle Boy

A better comparison would probably be the libpng flaw (also seen on Slashdot). However it has nothing to do with Linux itself but most (all?) Linux distros use it as well as some Windows and Mac programs (Mozilla is one of them).

Here here! I am irked at my 31% score. Some of my answers were "right" in that I guessed close to the answer (same order of magnitude), but I was highly uncertain. I was penalized for that and got zero points for several answers that I would have judged "good guesses." I think I went astray by putting too much "uncertainty" on some of my answers.

link to score

Had I been a computer, I think I would have put in put in guesses better for judging. I think Edinburoug is in Scotland (somewhere). But I am a yank and hampered by my lack of knowledge of the kingdom. I have no idea where Cardiff is (now that I think about it, I guess it is in Wales). It is probably on the British Isles, so it is no more than 1000 miles away. I've driven from London to Manchester and it was around 300 miles. So my thought was " my guess is it's about 500 miles, but its somewhere beteen 200 and 1000 miles for cardiff to whales. So I put 500 +/- 500 miles. The "right" input for my guess to get a better score probably should have been 600 +/- 400.

He's not so random, he was the man behind the Political Survey.

Tried the above image in Safari on OS/X and it went bye-bye after a great deal of disk thrashing. Offered me the chance to submit a bug report to Apple, but I didn't bother, as I figure somebody else has told them already...

I got the link from the original full disclosure over here. See near the end of section 1. That link is given in the CERT alert.

I know its a joke, but it seems to work in IE as well, or at least an example PNG crashes it, i suppose one could be crafted for IE to exploit it.

As alex_tibbles pointed out to me:
Or even better(?), check out Political Survey, the open source equivalent, where the methodology is open to all to inspect and criticise.

"Check out the political compass."

Or even better(?), check out Political Survey, the open source equivalent, where the methodology is open to all to inspect and criticise.

I'm the same person who replied to you earlier.

After a copious amount of further searching, I came upon this, which is probablty what was referred to. There's one thread in soc.culture.british that mentions it. Anyway, here it is.
The Motor Bus
What is this that roareth thus?
Can it be a Motor Bus?
Yes, the smell and hideous hum
Indicat Motorem Bum!
Implet in the Corn and High
Terror me Motoris Bi:
Bo Motori clamitabo
Ne Motore caeder a Bo --
Dative be or Ablative
So thou only let us live:
Whither shall thy victims flee?
Spare us, spare us, Motor Be!
Thus I sang: and still anigh
Came in hordes Motores Bi,
Et complebat omne forum
Copia Motorum Borum.
How shall wretches live like us
Cincti Bis Motoribus?
Domine, defende nos
Contra hos Motores Bos!

I think it's pretty shitty qua poem.

Actually, further research indicates "It's by A. D. Godley, a Latin professor at Oxford many decades ago" and "teaches 3d-declension masculine and 2d-declension neuter Latin declensions in all six cases and in both singular and plural").

"Corn and High are two streets in Oxford", and that's, I think, a clear reason for it's being called "the Oxford" poem.

HEY. The beginning of this poem is actually a familiar quotation (But from "The Columbia World of Quotations" not Bartlett's).

QUOTATION: What is this that roareth thus?
Can it be a Motor Bus?
Yes, the smell and hideous hum
Indicat Motorem Bum ...
Domine, defende nos Contra hos Motores Bos!

ATTRIBUTION: Alfred Godley (1856-1925), British scholar. letter, Jan. 10, 1914. "The Motor Bus," Reliquae, vol. 1 (1926).

So who's right?

By the way, at least I'm willing to *not* post as an AC, as well as willing to publically admit I may have been wrong.

I guess this article was written before the ftp.gnu.org compromise. However, has there been *any* reason given on why ftp.gnu.org was running wu-ftpd ( which has a restrictive license) when there are at least 2 GPL ftp daemons ( proftpd and vsftpd) available? Especially given wu-ftpd's long, sad history of insecurity.

vsftpd is all you need for secure ftp.

(and to avoid the same pointless conversation I have had too many times, secure in this context means not having exploits in the code. it does not mean having encrypted passwords - we are talking about anonymous ftp)

wget is the god of resuming either ftp or http. I have yet to see either fail, and don't notice too much of a difference in speed.

There are many, many ftpds where there is one prevailing httpd. Apache is very good at shoveling things across a network to be sure, but there seems to be an ftp for any particular purpose. Lots of more commercial OSen use ProFTPD for its recognizable config file and configurability, also, check out kernel.org's bandwidth meter. I don't know how shoveling out 50-100mbits/sec of data to thousands of users at all times washes with people, but it seems to lend credibility to the idea that ftp is a performer. ncftpd also has legendary performance.. Let's not forget VSFTPD, ftp.redhat.com, ftp.openbsd.org and ftp.suse.com are vsftpd.

All in all, wget, apache, a good ftpd [take your pick, PRP, VS or NC] and ncftp are required for sane internet usage.

And I've never seen wget fail a re-get...

and death to IIS ;p

VSFTPD is the best ftp server out there today, ok maybe Tux 2.0 is as good. VSFTPD has never been broken into and RedHat, and a number of other high profile FTP sites use it on their distribution servers. It's also damned fast. Homepage

I agree that it's a good start. I would also add in links to vsftpd's design & implementation documentation; see here, here, and here. For what it's worth, Maradns's string library is worth examining as well.

I agree that it's a good start. I would also add in links to vsftpd's design & implementation documentation; see here, here, and here. For what it's worth, Maradns's string library is worth examining as well.

I agree that it's a good start. I would also add in links to vsftpd's design & implementation documentation; see here, here, and here. For what it's worth, Maradns's string library is worth examining as well.

NOTE: This is a temporary location for this document, selected with slashdot people specifically in mind.

Copyright (C) 2002 Timo Sirainen <tss@iki.fi>

• Introduction
• Memory Allocations
  • The Old Ways
  • Data Stack
  • Memory Pools
  • Memory Pool API
• String Handling
  • The Old Ways
  • String API
  • String API with Memory Pool API Support
• Buffer Handling
• Real World Usage

Everyone knows about buffer overflows nowadays. Everyone knows some ways to prevent them. But most people still do it in a way that requires them to be unnecessarily careful while coding - even a single carelessly written part of code can be a security disaster.

None of these ideas I've written about is new, but most of them are very rarely used with C programs. I think it's mostly because many people don't know about them or just haven't realized how they could be easily used. Sure, there are also people who will never change their way of coding. And there's also the problem that using non-libc functions make the program bigger, which is pretty annoying with otherwise very small programs.

Besides not only making your code more resistant to buffer overflows, I think many of these features will actually make writing C-code a lot easier and more fun.

I'm not a writer and I'm not too good at english, so sorry about all the spelling and grammar errors :) All of this stuff was written at sunday morning, tired after being awake the whole night and not being able to do anything useful..

• Use static buffers - fast, but lack the ability to grow when needed.
• malloc() - slighly slower than static buffers, possible memory fragmentation, and most importantly it requires the memory to be freed, which can be sometimes very annoying. It's easy to forget to free the memory and cause memory leaks. free()ing already freed memory may also be an exploitable a security flaw.
• Garbage collector - this would be the best way to manage memory. For example OCaml's garbage collector is quite smart by treating long-living allocations differently than temporary allocations. However with C it's not really possible - you can't go moving the allocated memory elsewhere unless you write your program in special way. So there's a few simple non-portable garbage collector implementations for C, but they're not much more different from malloc()ing other than that you don't need to free() memory. They're not fool proof either.

What I haven't yet seen used anywhere outside my own software and some programming languages internals (eg. calling Perl code from C), is using data stack for temporary memory allocations. It has the most important advantage of garbage collectors; allocate memory without worrying about freeing it. It also has a few gotchas, but I'd say it's advantages are well worth it.

The way it works is simply letting the programmer define the stack frames. All memory allocated within the frame are freed at once when the frame ends. This works best with programs running in some event loop so you don't have to worry about the stack frames too much. Here's an example program:

Advantages over control stack:

• Functions can return a value allocated from data stack.
• We can portably specify how much data we want to allocate at runtime, and in some cases we can grow it without copying data.

Advantages over malloc():

• FAST, most of the time allocating memory means only updating a couple of pointers and integers. Freeing the memory all at once also is a fast operation.
• No need to free() each allocation. This has a huge impact on code simplicity. For example: fprintf(f, "'%s'\n", escape_string(input)); vs. char *str = escape_string(input); fprintf(f, "'%s'\n", str); free(str);
• No memory leaks.
• No memory fragmentation.

Disadvantages:

• Allocating memory inside loops can accidentally allocate a lot of memory if the loops are long and you forgot to place stack frame there.
• The memory could accidentally be saved somewhere and accessed after the end of stack frame.
• Debugging for buffer overflows can be difficult, since the regular bound checkers don't work. To kludge around this, I've made an compile time option to allocate all memory using malloc(), but this of course hurts the performance and grows memory usage. Of course, by coding good code you should never need this :)

The second disadvantage is the most problematic one. It's actually two problems; first it shouldn't be mixed with permanent data, and second it shouldn't be temporarily stored to location where it could be accessed outside the stack frame.

The first one is easier to handle. The ideal solution would be to have some tags in C language that would give warning if being lost, for example temporary char *str = t_strdup("str"); char *str2 = str; where the str2 assignment would give a compiler warning about the missing temporary tag. But since this isn't possible we can do almost as well using const keyword, which has exactly that behaviour but restricts us from modifying the memory. Luckily that's not usually needed.

The second one is much more difficult, I haven't found any good way to handle it other than by a) don't even try it, b) fill the freed memory when freeing stack frame so the corrupted value is easily noticed at runtime.

And why ever even do that? Because it's simpler in some situations. For example I have this message header parser, which calls a callback function for each header line. Suppose I only want to find the message's Content-Type, so save it with context->content_type = t_strdup(content_type); Then later I just read it without need to worry about freeing it. Now, the only problem with this is that it relies on the parser not to create a stack frame around the callback which would invalidate our saved return value. While writing the callback it's easy to check if it's possible, but what if the parser was later modified without remembering the special needs of the one callback..

Alloc-only pools can be quite useful for storing larger amount of return values from some function, especially when data stack can't be used (the second problem case above). They still provide relatively easy memory management since you only need to free the pool once.

Alloc-free pools could be useful with some applications to prevent memory fragmentation and memory leaks by grouping related data together into single malloc() block. They could also make the program run faster due to (possibly) better CPU cache utilization. Besides performance reasons, they could also be useful for statistics to find out where exactly the allocated memory goes. They'd however need their own internal malloc() implementation so they're not very simple.

When you have a function that could return a large return value, it's not very efficient to first store it into data stack and then copy it somewhere else. The traditional way would be to just give a (void *result, size_t size) as parameters. This again has the problem that we might not know at all what buffer size is large enough.

So, one way to deal with this is to give the function a memory pool object which can be used by the function to allocate the correct amount of data. Besides the dynamically created alloc-only and alloc-free pools there could be global pools as well: data_stack_pool and system_malloc_pool. Here's an example:

This is where most of the buffer overflows have happened. Nowadays people are more aware of the problem, but many still they do it in unnecessarily hard way.

• strncpy(), strncat(), snprintf() - only snprintf() of these is easy to use safely but it's still somewhat unportable (Windows). strncpy() doesn't necessarily NUL-terminate and many people misunderstand how strncat() works (ie. in very stupid and difficult to use way).
• strlcpy(), strlcat() - much better replacements to above by OpenBSD. Very unportable, but you can easily create your own ones. But these can still be used unsafely if the buffer size parameter is wrong or if the programmer goes playing around with the buffer indirectly, by eg. appending single characters and missing size checks (yes, I've seen this in software that contained "secure" in it's name).
• Dynamically allocating the amount of wanted memory and then using strcpy(), strcat(), sprintf() and direct accessing. This requires you to be very careful with the string size calculations. I don't understand why so many people think that's not a problem, they have this "If you can't calculate the sizes correctly, you're stupid and you shouldn't be coding at all" attitude. Why bother wasting time with that at all when you could be doing more important things?
• Dynamically growing buffers, used by for example GLIB, vsftpd, qmail, djbdns and Postfix. This is definitely the right way; string manipulation is done through API which discourages - or even disallows - direct buffer manipulation.

There's several slight variations how to implement the dynamically growing buffers. Most work by allocating a new string object, using it and then freeing it. qmail and djbdns uses statically created buffers which are reused and never freed - that's pretty dangerous unless you can be sure the buffer isn't in use anymore. It's actually pretty much the same as having static char bigbuf[8192] which is used by several functions.

There's two ways how to manage a buffer - first being the explicit way done by eg. djb:

GLIB supports doing this much more easily:

Or if you wanted a modifyable buffer:

Some people don't seem to like the printf-style or believe it's unsafe. GCC however gives very good warnings about parameters with incorrect type so I don't think there's any need to worry about.

Nothing fancy here really. Just by being able to specify the used memory pool we can easily allocate strings from data stack. I've made several wrapper functions so that instead of p_strdup_printf(data_stack_pool,...) I can just use t_strdup_printf(). Here's a list of few useful functions:

t_strdup_noconst is there mostly to avoid casting const away in those few situations where it's needed. Because it returns char * it could be more easily mixed with permanent data so it's usage should be kept minimal.

t_strconcat() is one function that I also copied from GLIB. It's a bit dangerous though, the terminating NULL is too easy to forget. I've been thinking about removing it entirely, but it's much more efficient than t_strdup_printf() so I haven't yet had the heart :)

Many people concentrate only on string related buffer overflows. Yes, they've been the most common ones but they're quite known now and there's more to buffer overflows than just them. The most obvious one is just another type of buffer handling. All the other data that couldn't be called strings. Often people just hack away separate memory allocations and/or size checks for them. The problems are exactly the same as with strings.

If we create a buffer API and write to buffers only through it, we would prevent almost all buffer overflows, even integer related ones, simply because the buffer API implementation is the only part of code that directly writes to memory, and that of course should be well audited not to overflow in any situation.

Buffer API would be somewhat similar to string API, ideally the string API should be created using the buffer API, or they could even be the same. My buffer API currently looks like this:

Then there are several write, append and truncate functions and a few functions to temporarily limit the accessible buffer range. An example:

There is of course also the possibility of buffer overflows while reading data. That's a bit more difficult to prevent except by careful coding, but usually it's not such a big deal anyway. It might crash the program, or at worst could expose some private information to attacker. If the later is a real problem, you should use multiple processes and IPC to hide the private data.

I've written a fully featured IMAP server using pretty much these techniques. The code still needs some cleanups and there's probably a couple of bugs left, but mostly I think it's not too bad :) The library functions in it are MIT licenced, so go ahead and rip them to your own programs. I could try to create some minimal library out of them if there's enough interest.

vsftpd uses it's own string API for pretty much everything. In only few of the files it's even possible to create a buffer overflow with the used coding style.

I think many people could do themselves a great service by simply studying the vsftpd security libraries. Those are generic enough to be used by almost any other application, and provide a solid foundation to write applications that do not fall apart under classic C exploits: buffer overflows, etc.

Check here the vsftpd website.