Domain: beasts.org
Stories and comments across the archive that link to beasts.org.
Comments · 43
-
Change Ports!
Most automated scans will not take the time to scan for open ports. (that I have experienced)
Also consider FTP with SSL / TLS like what can be done with vsftpd. http://vsftpd.beasts.org/
As mentioned elsewhere in this thread consider using Fail2ban which is easily configured for monitoring failed attempts at connecting to your server and can then block the IP after a configurable threshold is reached. -
Re:Only ProFTPd?
vsftpd has always been excellent for me--you can limit the anonymous user in a number of ways, and by default it's extremely locked down (may not even be enabled at all, can't remember, but if it's disabled, even when you enable it it's still very locked down).
-
Re:Only ProFTPd?
ProFTPd, while great when it came out in the early 2000s and had many features, was always plagued with security issues, and many places I worked at stopped used it even though it was relatively easy to setup and was feature rich.
vsftpd was at one point the most secure ftpd I could find. Not sure how good it is anymore.
If you need a simple Windows FTPd on a small LAN for quick file transfers check out ftpdmin which is really minimal (free and source available!). I use it all the time for quick ISO transfers on my personal LAN. Does the job. Much better than HTTP transfers, that's for sure.
-
O/S Permissions Model Broken - try plash
The idea that software that I have no opportunity to audit runs with my privileges when I run it is fundamentally broken. There is no way to clean malware that had a network connection long enough to hide a trojan anywhere and then overwrite itself to appear relatively innocuous. While a complete redesign of the way permissions and permission-delegation would be ideal, it is not practical in the short term. Using systems like http://plash.beasts.org/ can help.
-
Might be a different world by then
By then, people might be running VM's within VM's to avoid this kind of
... stuff.Or perhaps, there will be 5 different lightweight sandboxes like Plash, with each sandbox scanning for attempts to exploit known vulnerabilities of the other sandboxes (this wouldn't prevent some people from getting owned, but it would expose malware quicker).
-
Re:Strategy fail
There was a conversation about this last month on the FD.o mailing list
> Mark Seaborn wrote:
> > I have written a hack for Gtk which replaces Gtk's usual file chooser
> > with a file chooser implemented in a separate process.
>
> Nice, this could be useful. I'm not sure yet if there is a concise
> in-process solution. If not, I'll have a closer look on your code.The current version is an LD_PRELOAD library that replaces some Gtk
API calls. It works with a lot of applications, but not with Firefox,
which dlopen()s libgtk. The successor is a work-in-progress; it's a
patch to Gtk, which you can find here:
http://repo.or.cz/w/gtk-with-powerbox.git (code)
http://plash.beasts.org/wiki/Story20 (notes)The intention is that the Gtk code will be separate from the RPC
mechanism used, which is implemented in libpowerbox, of which there
can be different implementations.
For example, there is a DBus implementation:
http://svn.gna.org/viewcvs/plash/scratch/powerbox-dbus/
and the intention is that there will also be a version using Plash's
object-capability comms protocol.Maybe I should rename libpowerbox to something more descriptive such
as libtrustedpathfilechooser (that's a bit long-winded though).> Is there a chance that you'll give me permission to "reuse" your
> code under the terms of the MIT-License?I'm open to the idea of relaxing the licence, but which bits of code
do you have in mind? The Gtk hooks are quite Gtk-specific.Mark
I hope that it happens, too.
-
Re:Even VMware doesn't have the perfect VM yet.
I assume you mean things like shared dlls.
And IPC. FYI, There is nothing inherently dangerous about sharing DLLs, that memory is either read-only or Copy-on-write. There is much less likely to be a bug in something as simple as shared memory handling code than a VM.
access to users documents, Direct X, OpenGL, screenshots (GIMP)
access to users documents: VMs don't give access to user documents or systemwide screenshots by default. This would lose much of the security of running a VM. Direct X, OpenGL: VMware includes experimental support for Direct3D video acceleration. This feature is not fully functional. Virtual box only supports OpenGL. As far as I know VirtualPC doesn't support either.
And it would STILL be more compatible than what we currently get passed off as "backward compatibility"
Really? Because even Wine offers much better support for gaming than any VM I have come across (which is all I use Vista for). I haven't had any trouble with backwards compatibility with Vista, except with device drivers, and the VMs aren't going to have much use for those. And I have continually hit bugs and limitations with every VM I have come across.
Which sounds more security to you?
Well, myself I'd pick an effective way of sandboxing applications without the overhead of a VM, and then sandbox everything whether it is 64bit or not.
Either you didn't read the entire post, you chose to ignore the part that explains why we would be better off memory wise,...
Maybe because the closest thing is an argument was
Literally at worst your XP application would require 3 gigs of ram. Now consider that you now have an OS that cleanly could access up to 16 exabytes of RAM, the memory limitation would be far worse on the 32 bit native than it would be on the clean 64-bit system running virtual machines as they could load more programs, and each program could have more memory than if it ran native on a 32-bit machine.
None of which explains how running 5 3gig VMs could be more memory efficient than running 32bit code directly on a 64bit kernel allocating only and exactly the memory it needs. And 5*3gig of memory is not all that cheap. Have you considered that it may be you who fails to understand? In any case, this thread has become more confrontational than interesting, and have little interest in spending more time on it.
-
Re:Open browser engineering issues
There is no question that programming in C and C++ requires skill, and that memory management is an issue, and automatic heap allocation and garbage collection is the popular solution to that issue, but there is no silver bullets, and they are always compromises. I Yes, project need to be written with security in mind. There are program with few security hole written in C/C++ but most of the time they use special API for strings, allocation,
... For example vsftpd : http://vsftpd.beasts.org/#security Again, there are no silver bullets. For instance, the Java sandbox is one solution to a security issue. It is not perfect, and it's imperfections lead to a false sense of security. It is ok for developers to be sloppy because garbage collection and the sandbox will protect the user. Sandbox doesn't imply high level language. Even in a sandbox, if it is not correctly configured, there can be security hole. On the other side you can be sandboxing in C/C++ (using the OS). For example using chroot/PR_SET_SECCOMP ...) I think the problem is writing a safe application is really hard and can be tricky. You need to know well "secure" programming and what offer the OS. For the average programmer using high level language can help : it is the writer of the high level language who have managed security, not you. And you can expect them to do better than you. -
Re:Sorry Server Down - Link To Article
http://www.mythic-beasts.com/
A page that got slashdotted
http://roughly.beasts.org/
Our account of being slashdotted
http://www.ex-parrot.com/~chris/wwwitter/20040911-more_introspective_nonsense.html
Another page that got slashdotted.
http://ex-parrot.com/~pete/upside-down-ternet.html
That's all on our shared hosting server cluster. A Mac Mini will hold up fine too.
We're no stranger to being slashdotted :-) -
principle of least authority
Whitelisting an entire app is too coarse-grained. We need to be able to whitelist the actions that a specific app can take. For example... by default, microsoft word should only have read-only access to its own files and library files needed to run. It should have no net access whatsoever. It should have no access to any of my personal data files. It should have no access to my keyboard when I'm typing in another app. It should have no access to drawing on any portion of my screen outside its own window. It should have no way to spoof another app's name in the window titlebar. It should only have read/write access to the single file that I click "open" on to edit.
Since I have to choose the files I want to work with in a file open dialog anyway, we force microsoft word to use a system-wide, trusted file open dialog which is the only way to grant it access to more files.
See http://plash.beasts.org/ and http://www.eros-os.org/essays/capintro.html
-
Informative parent
I am glad to hear that someone is working on a real solution to this problem, because many people do not even seem to be aware of it. XSELinux sounds like exactly what is needed. More information here.
-
Re:dear lord...
It should be possible to download and run third party software without putting your system at risk. That game you downloaded doesn't need administrative privileges to run - why should it need them to install? Just laziness on the part of the developers, cushioned by the general expectation that users on a Windows machine will have access to the administrator account.
More fundamentally, if you install a game into C:\somegame why should it have write access to any file outside that directory? If you install a spreadsheet program why should it be able to read any files at all except the standard Windows libraries, its own installation files, and files you explicitly load into it? At the moment there's only one level of privilege - essentially 'everything' - and all applications run with that.
See Plash for an implementation of what I mean. -
Re:Sysrq.
BTW, Plash and Systrace do exactly that. In fact Plash lets you allow Firefox to open Documents (e.g. on ~/Desktop) if and only if you have selected that file from the (trusted) standard file open dialog box, in principle giving firefox the very least priviledge with no inconvienience.
-
Security will always be a problem?
From TFA: Schroepfer also predicts that security will continue to be a problem "for anything written in native code," such as C and C++. For example, he notes that security problems caused by memory issues have evolved over the years; from stack-based exploits, to heap-based, to null pointer exploits.
From http://vsftpd.beasts.org/IMPLEMENTATION: The correct solution is to hide the buffer handling code behind an API. All buffer allocating, copying, size calculations, extending, etc. are done by a single piece of generic code. The size security checks need to be written once. You can concentrate on getting this one instance of code correct.
Can somebody please tell me, why are we still having this discussion? -
Links?Hi could you give the links to the technology you are using. I am very interested in this field (giving applications less rights than the user). I have not heard of SAKs before.
BTW, Have you heard of Plash or Systrace?
Unfortunately I don't think that many Linux systems are set up the way you describe, though I intend to make it my personal quest to make sure they are.
Also, have you come across a way of stopping GUI applications taking over other GUI application via the X protocol?
I know that it is possible to run X applications in untrusted mode, but I understand that is still possible for untrusted applications to snoop on other untrusted applications via X, so we cannot simply run all applications in untrusted mode.
-
Do not need to split File/Web browser for securityTruth be told, it still is a cool thing. I'd love Mozilla to be my official interface to my hard drive as well as the web. Unfortunately, security in such a situation really is tough.
Actually, under the the current Linux (and Windows etc.) security model, seperating the web browser from the file browser does not gain much security. Although firefox doesn't need to read or write large numbers of random files across your home directory, this does not mean it cannot if someone takes control of it using a buffer overflow.
We could constrain firefox using a utility such as Plash. If Konqueror were designed with the Principle of Least Authority in mind we could run as a seperate process, each constrained in a different Plash environment, having only the rights that that particular tab needed (web access or file access, not both). This would allow us to have an integrated UI and enforce the Principle of Least Authority as well.
-
False sense of securityThe only way to good security is to not give untrustworthy applications write access to all your files. There are far worse things they could do than add a few bookmarks, E.g. adding key loggers or (Sony) rootkits.
Ideally all applications would be run through something like the Principle of Least Authority shell which limits the applications so they can only access those files they actually need to function. -
You can do this with Linux (E.g. Systrace).Systrace allows you do this with Linux. True capabilities (as in OO constraints rather than POSIX) are even more cool, as they actually make systems simpler by combining authority with designation. For example by saying "cat a" you are both giving cat access to the file "a" and telling cat that "a" is the file you want it to open.
PLASH is a hack that allows you to do something similar under Linux. E.g. it passes in rights to file opened with the GTK file open dialog so that you don't have to confirm that you want to open a files that you have already "passed in" by selecting them in a dialog box.
-
Protecting Users from Rogue SoftwareWhat the article comes down to is that user accounts are of little use to single user PCs. This is well known by security theorists (E.g. you don't need root access to put a keylogger in ~/.bashrc and steal your bank login info). As the author states, the Linux (and Windows and Novell and...) security model is designed to protect the system from the user, but assumes that the user trusts the software.
OTOH existing "Trusted code" initiatives are the equivalent of delcaring a root-only system secure, because you only let a few hundred "trusted" users log into that machine.
IMHO, Operating systems should move towards the "Principle of Least Authority", that each software module should have the least authority required to function. The ideal way of doing this is for the system to enforce ObjectOriented/Functional/Procedural constraints that all OO, functional and procedural programs define implicitly. For example, if you run "uncompress(const File A,File B)" then obviously "uncompress" does should not be able to access File C because File C is outside its scope. Likewise, if you click on "bunnies.doc" in Konqueror and Konqueror runs "OpenOffice(UI ui, FileSystem::open("~/bunnies.doc")), then OpenOffice should not be able to open "~/.bashrc" because OpenOffice was not passed FileSystem. OpenOffice would only be able to access "~/.bashrc" if it is passed the file by an object which has rights to "~/.bashrc", for example ui::FileOpenDialog. The EROS Operating system is built around this principle.
PLASH is an attempt to retrofit this principle into Linux, and Looks really promising. A program constrained by Plash can only open a file if it is passed it on the command-line or by a GTK fileopen dialogbox.
-
We Need Least Authority
The problem is that the Unix security model grants the entire set of the user's authority to processes they run. That's why the user's entire
/home directory is vulnerable. The problem is excessive authority. Unix needs support for least authority, which incidentally doesn't require a new security model. Unix's existing abstractions can be leveraged. The Principle of Least Authority Shell (Plash) http://plash.beasts.org/ demonstrates that we can do least authority on Unix for files without even kernel modification. -
POLA
What you are describing is the Principle Of Least Authority. PLASH (Principle Of Least Authority SHell) is a nifty project to tackle this at the application level for Linux http://plash.beasts.org/. HP Labs has a project called Polaris which does this for windows http://www.hpl.hp.com/research/mmsl/projects/adv/
p olaris.html. -
Actually, standard versions of wrapper 'd be good.Recently the PLASH project had to override the standard GTK file open dialog box to achieve certain security related aims. This was somewhat complicated by the fact that there is no standard GTK file open dialog box function
;).Also it would be one less function that has to be written and maintained across the thousands of projects that use GTK.
-
Re:Opensource list
I just add a bit on that list from top of my head.
Although I think the listed app goes beyond what the so called 'average pc user' wants, but there goes...
1. Konqueror ( http://www.konqueror.org/ )
2. Email - Sylpheed ( http://sylpheed.good-day.net/ )
3. I think Evolution is more like in this place.
4. Lately "Sound Juicer" is taking more attention too
5. VideoLAN aka VLC ( http://www.videolan.org/ ) and Ogle ( http://www.dtek.chalmers.se/groups/dvd/ ) [and Goggles ( http://www.fifthplanet.net/goggles.html ) for Ogle GUI wrapper] for DVD watching.
6. There are plenty way to do this, but the typical ones could be 'Jinzora' ( http://www.jinzora.org/ ) and 'MusicPD' ( http://www.mpd.org/ ), even plain Apache does it fine too, in a way.
8. If you want easier to manage iptables wrapper, Shorewall ( http://www.shorewall.net/ ) and there are other wrappers too.
9. KOffice ( http://www.koffice.org/ ) and by individual components, Abiword ( http://www.abisource.com/ ), Gnumeric ( http://www.gnome.org/projects/gnumeric/ ), Gnucash ( http://www.gnucash.org/ )
10. Inkscape ( http://www.inkscape.org/ ) or Sodipodi ( http://www.sodipodi.com/ ) for vector graphics.
11. Miranda ( http://miranda-im.org/ ). Windows only.
13. Hmm , Samba? ( http://www.samba.org/ ), WedDAV (Look parent post), FTP (plenty ftp daemons, ex : http://www.proftpd.org/, http://vsftpd.beasts.org/ etc)
16. GPhoto ( http://www.gphoto.org/ ), EOG ( http://www.gnome.org/ ? ), GQView ( http://gqview.sourceforge.net/ ). The latters are for just viewing mainly.
20. FreeNX ( http://www.nomachine.com/ , http://freenx.berlios.de/ ) http://www.poptop.org/ ), L2TPd ( http://sourceforge.net/projects/l2tpd ), RP-L2TPd ( http://sourceforge.net/projects/rp-l2tp/ )
24. Postfix ( http://www.postfix.org/ ), Sendmail ( http://www.sendmail.org/ ), Exim ( http://www.exim.org/ ), Cyrus ( http://asg.web.cmu.edu/cyrus/imapd/ ), Xmail ( http://www.xmailserver.org/ ), qmail ( http://www.qmail.org/ )
25. Spamassassin ( http://spamassassin.apache.org/ )
26. Same as above.
27. XSane ( http://www.xsane.org/ ) for sane frontends.
30. Buzzmachines ( http://www.buzzmachines.com/ ) I could be wrong...
31. 'various GUI frontends' - X CD Roast ( http://www.xcdroast.org/ ), K3B ( http://k3b.sourceforge.net/ )
32. Don't know any opensource ones... -
All Crap...
The group policy (Worldwide) is to have *ONLY* Windows 2003 Servers.
Just was doing a scan on the German network, and the main online reporting server/portal is running Linux ....
Hehe.., either that or one of the clear german IT got MySQL and VSftpD http://vsftpd.beasts.org/ on Windows 2003.., and hacked the TCP fingerprint to show up as Linux 2.4
The Admin is just studit and does not have a clue of that Firefox really is.
You should, (which the Admin-Troll is off getting a coffee, install Firefox on his PC and delete the IE icon from desktop and startbar, and rename the Firefox one to "Internet Explorer" and change the icon to the stupid "E".
He probally will not even notice.....
P.S. His password is either "null" or print on a post-it on his monitor......
"Clutch my testes, bloody squirrel humpers!!" -Happy Noodle Boy -
Re:Why doesn't someone sue LINUX?
A better comparison would probably be the libpng flaw (also seen on Slashdot). However it has nothing to do with Linux itself but most (all?) Linux distros use it as well as some Windows and Mac programs (Mozilla is one of them).
-
Re:About the article...
Here here! I am irked at my 31% score. Some of my answers were "right" in that I guessed close to the answer (same order of magnitude), but I was highly uncertain. I was penalized for that and got zero points for several answers that I would have judged "good guesses." I think I went astray by putting too much "uncertainty" on some of my answers.
Here are samples of the scoring I got: population of the whole world in 2004
Yours 6±1 billion
True answer 6.38±0.005 billion ...
got 8 points.
number of floors in the Empire State Building in New York City
Yours 100±20
True answer 102 exactly ...
got 8 points for that too.
number of words in Jane Austen's novel Pride and Prejudice
Yours 100000±200000 words
True answer 122093 words exactly ...
0 points
My guess is w/in 20% of the real answer, 0 points.
distance from Edinburgh to Cardiff, as the crow flies
Yours 500±500 miles
True answer 310±5 miles ...
My guess is about 40 % from real answer, within my bounds of uncertainty, but 0 points nevertheless.
Fraction of the adult population of the UK who are functionally illiterate
Yours 20±10 per cent
True answer 20±0.5 per cent ...
5 points
My guess is almost right on the money, closer than the population of the world guess, but I put too much uncertainty in, only 5 points.
Had I been a computer, I think I would have put in put in guesses better for judging. I think Edinburoug is in Scotland (somewhere). But I am a yank and hampered by my lack of knowledge of the kingdom. I have no idea where Cardiff is (now that I think about it, I guess it is in Wales). It is probably on the British Isles, so it is no more than 1000 miles away. I've driven from London to Manchester and it was around 300 miles. So my thought was " my guess is it's about 500 miles, but its somewhere beteen 200 and 1000 miles for cardiff to whales. So I put 500 +/- 500 miles. The "right" input for my guess to get a better score probably should have been 600 +/- 400.
-
Random Blogger?
He's not so random, he was the man behind the Political Survey.
-
Crashes Safari
Tried the above image in Safari on OS/X and it went bye-bye after a great deal of disk thrashing. Offered me the chance to submit a bug report to Apple, but I didn't bother, as I figure somebody else has told them already...
-
Re:Where did you get the example PNG ?
I got the link from the original full disclosure over here. See near the end of section 1. That link is given in the CERT alert.
-
The latest SP2 fixes it.
I know its a joke, but it seems to work in IE as well, or at least an example PNG crashes it, i suppose one could be crafted for IE to exploit it.
Well using XP SP2 RC2 build 2162 it does nothing in IE other show a broken image link. Whatever Microsoft did in SP2, it seems to have mitigated it. They did recompile major parts of the OS for SP2 with the /GS VC++ stack checking compiler flag. That could have caught it. Or it could be that they were informed about it before full disclosure and they fixed it in SP2. Or that they don't use libpng and their library does it correctly or they fixed the issue by themselves. Whatever be it they seem to have taken care of it. BTW the built-in Windows Picture and Fax Viewer also doesn't crash (nor does mspaint). You can test this out yourself if you have SP2 (don't know if builds earlier than 2162 fix it though) using this image link (Warning! Will crash non patched browsers!) from the original disclosure.
Its reassuring that for once MS has already taken care of some security issue (for XP SP2 at least). -
Re:What's the deal with freerepublic.com?
As alex_tibbles pointed out to me:
Or even better(?), check out Political Survey, the open source equivalent, where the methodology is open to all to inspect and criticise. -
Re:Paul Graham's politics
"Check out the political compass."
Or even better(?), check out Political Survey, the open source equivalent, where the methodology is open to all to inspect and criticise. -
Found it!!!
I'm the same person who replied to you earlier.
After a copious amount of further searching, I came upon this, which is probablty what was referred to. There's one thread in soc.culture.british that mentions it. Anyway, here it is.
The Motor Bus
What is this that roareth thus?
Can it be a Motor Bus?
Yes, the smell and hideous hum
Indicat Motorem Bum!
Implet in the Corn and High
Terror me Motoris Bi:
Bo Motori clamitabo
Ne Motore caeder a Bo --
Dative be or Ablative
So thou only let us live:
Whither shall thy victims flee?
Spare us, spare us, Motor Be!
Thus I sang: and still anigh
Came in hordes Motores Bi,
Et complebat omne forum
Copia Motorum Borum.
How shall wretches live like us
Cincti Bis Motoribus?
Domine, defende nos
Contra hos Motores Bos!
I think it's pretty shitty qua poem.
Actually, further research indicates "It's by A. D. Godley, a Latin professor at Oxford many decades ago" and "teaches 3d-declension masculine and 2d-declension neuter Latin declensions in all six cases and in both singular and plural").
"Corn and High are two streets in Oxford", and that's, I think, a clear reason for it's being called "the Oxford" poem.
HEY. The beginning of this poem is actually a familiar quotation (But from "The Columbia World of Quotations" not Bartlett's).
QUOTATION: What is this that roareth thus?
Can it be a Motor Bus?
Yes, the smell and hideous hum
Indicat Motorem Bum ...
Domine, defende nos Contra hos Motores Bos!
ATTRIBUTION: Alfred Godley (1856-1925), British scholar. letter, Jan. 10, 1914. "The Motor Bus," Reliquae, vol. 1 (1926). -
Re:ftp.gnu.orgAnd according to the vsftpd homepage, they are/were running vsftpd (scroll down a bit).
So who's right?
By the way, at least I'm willing to *not* post as an AC, as well as willing to publically admit I may have been wrong.
-
ftp.gnu.org
I guess this article was written before the ftp.gnu.org compromise. However, has there been *any* reason given on why ftp.gnu.org was running wu-ftpd ( which has a restrictive license) when there are at least 2 GPL ftp daemons ( proftpd and vsftpd) available? Especially given wu-ftpd's long, sad history of insecurity.
-
Re:Well that's good and all, but
vsftpd is all you need for secure ftp.
(and to avoid the same pointless conversation I have had too many times, secure in this context means not having exploits in the code. it does not mean having encrypted passwords - we are talking about anonymous ftp) -
Re:hmm I agree, http resumes nicely �esp w/ wget.
wget is the god of resuming either ftp or http. I have yet to see either fail, and don't notice too much of a difference in speed.
There are many, many ftpds where there is one prevailing httpd. Apache is very good at shoveling things across a network to be sure, but there seems to be an ftp for any particular purpose. Lots of more commercial OSen use ProFTPD for its recognizable config file and configurability, also, check out kernel.org's bandwidth meter. I don't know how shoveling out 50-100mbits/sec of data to thousands of users at all times washes with people, but it seems to lend credibility to the idea that ftp is a performer. ncftpd also has legendary performance.. Let's not forget VSFTPD, ftp.redhat.com, ftp.openbsd.org and ftp.suse.com are vsftpd.
All in all, wget, apache, a good ftpd [take your pick, PRP, VS or NC] and ncftp are required for sane internet usage.
And I've never seen wget fail a re-get...
and death to IIS ;p -
Most Secure FTPD: VSFTPD
VSFTPD is the best ftp server out there today, ok maybe Tux 2.0 is as good. VSFTPD has never been broken into and RedHat, and a number of other high profile FTP sites use it on their distribution servers. It's also damned fast. Homepage
-
Re:a little short??
-
Re:a little short??
-
Re:a little short??
-
mirrorSecure, Efficient and Easy C programming Secure, Efficient and Easy C programming
NOTE: This is a temporary location for this document, selected with slashdot people specifically in mind.
Copyright (C) 2002 Timo Sirainen <tss@iki.fi>
Index IntroductionEveryone knows about buffer overflows nowadays. Everyone knows some ways to prevent them. But most people still do it in a way that requires them to be unnecessarily careful while coding - even a single carelessly written part of code can be a security disaster.
None of these ideas I've written about is new, but most of them are very rarely used with C programs. I think it's mostly because many people don't know about them or just haven't realized how they could be easily used. Sure, there are also people who will never change their way of coding. And there's also the problem that using non-libc functions make the program bigger, which is pretty annoying with otherwise very small programs.
Besides not only making your code more resistant to buffer overflows, I think many of these features will actually make writing C-code a lot easier and more fun.
I'm not a writer and I'm not too good at english, so sorry about all the spelling and grammar errors
Memory Allocations The Old Ways: :) All of this stuff was written at sunday morning, tired after being awake the whole night and not being able to do anything useful..- Use static buffers - fast, but lack the ability to grow when needed.
- malloc() - slighly slower than static buffers, possible memory fragmentation, and most importantly it requires the memory to be freed, which can be sometimes very annoying. It's easy to forget to free the memory and cause memory leaks. free()ing already freed memory may also be an exploitable a security flaw.
- Garbage collector - this would be the best way to manage memory. For example OCaml's garbage collector is quite smart by treating long-living allocations differently than temporary allocations. However with C it's not really possible - you can't go moving the allocated memory elsewhere unless you write your program in special way. So there's a few simple non-portable garbage collector implementations for C, but they're not much more different from malloc()ing other than that you don't need to free() memory. They're not fool proof either.
What I haven't yet seen used anywhere outside my own software and some programming languages internals (eg. calling Perl code from C), is using data stack for temporary memory allocations. It has the most important advantage of garbage collectors; allocate memory without worrying about freeing it. It also has a few gotchas, but I'd say it's advantages are well worth it.
The way it works is simply letting the programmer define the stack frames. All memory allocated within the frame are freed at once when the frame ends. This works best with programs running in some event loop so you don't have to worry about the stack frames too much. Here's an example program:
/* asprintf()-like functon, but allocates the memory from data stack */ const char *t_printf(const char *fmt, ...) __attribute__((format (printf, 1, 2))); void client_input(struct client *client) { const char *cmd; int i; cmd = read_command(client); if (cmd == NULL) return; if (strcasecmp(cmd, "status") == 0) { time_t now = time(NULL); client_send(client, t_printf("Time: %s", ctime(&now))); client_send(client, t_printf("Clients: %u", clients_count)); for (i = 0; i < 10000; i++) { /* without an extra stack frame here, we'd allocate the t_printf() 10000 times before freeing them all. That's probably not be very good. */ t_push(); client_send(client, t_printf("%d: %u", stuff[i])); t_pop(); } } else { client_send(client, t_printf("Unknown command: %s", cmd)); } } void main_loop(void) { unsigned int i, id; fd_set rfds; if (select(max_fd, &rfds, NULL, NUL, NULL) <= 0) return; for (i = 0; i < clients_count; i++) { if (FD_ISSET(clients[i].fd, &rds)) { id = t_push(); client_input(&clients[i]); if (t_pop() != id) { /* we could simply call the missing t_pop()s, but this usually indicates a problem which we want to know. for example we might have leaked it inside a for loop which caused unnecessarily large memory usage. */ panic("missing t_pop()"); } } } }Advantages over control stack:
- Functions can return a value allocated from data stack.
- We can portably specify how much data we want to allocate at runtime, and in some cases we can grow it without copying data.
Advantages over malloc():
- FAST, most of the time allocating memory means only updating a couple of pointers and integers. Freeing the memory all at once also is a fast operation.
- No need to free() each allocation. This has a huge impact on code simplicity. For example: fprintf(f, "'%s'\n", escape_string(input)); vs. char *str = escape_string(input); fprintf(f, "'%s'\n", str); free(str);
- No memory leaks.
- No memory fragmentation.
Disadvantages:
- Allocating memory inside loops can accidentally allocate a lot of memory if the loops are long and you forgot to place stack frame there.
- The memory could accidentally be saved somewhere and accessed after the end of stack frame.
- Debugging for buffer overflows can be difficult, since the regular bound
checkers don't work. To kludge around this, I've made an compile time
option to allocate all memory using malloc(), but this of course hurts
the performance and grows memory usage. Of course, by coding good
code you should never need this
:)
The second disadvantage is the most problematic one. It's actually two problems; first it shouldn't be mixed with permanent data, and second it shouldn't be temporarily stored to location where it could be accessed outside the stack frame.
The first one is easier to handle. The ideal solution would be to have some tags in C language that would give warning if being lost, for example temporary char *str = t_strdup("str"); char *str2 = str; where the str2 assignment would give a compiler warning about the missing temporary tag. But since this isn't possible we can do almost as well using const keyword, which has exactly that behaviour but restricts us from modifying the memory. Luckily that's not usually needed.
The second one is much more difficult, I haven't found any good way to handle it other than by a) don't even try it, b) fill the freed memory when freeing stack frame so the corrupted value is easily noticed at runtime.
And why ever even do that? Because it's simpler in some situations. For example I have this message header parser, which calls a callback function for each header line. Suppose I only want to find the message's Content-Type, so save it with context->content_type = t_strdup(content_type); Then later I just read it without need to worry about freeing it. Now, the only problem with this is that it relies on the parser not to create a stack frame around the callback which would invalidate our saved return value. While writing the callback it's easy to check if it's possible, but what if the parser was later modified without remembering the special needs of the one callback..
Memory PoolsAlloc-only pools can be quite useful for storing larger amount of return values from some function, especially when data stack can't be used (the second problem case above). They still provide relatively easy memory management since you only need to free the pool once.
Alloc-free pools could be useful with some applications to prevent memory fragmentation and memory leaks by grouping related data together into single malloc() block. They could also make the program run faster due to (possibly) better CPU cache utilization. Besides performance reasons, they could also be useful for statistics to find out where exactly the allocated memory goes. They'd however need their own internal malloc() implementation so they're not very simple.
Memory Pool APIWhen you have a function that could return a large return value, it's not very efficient to first store it into data stack and then copy it somewhere else. The traditional way would be to just give a (void *result, size_t size) as parameters. This again has the problem that we might not know at all what buffer size is large enough.
So, one way to deal with this is to give the function a memory pool object which can be used by the function to allocate the correct amount of data. Besides the dynamically created alloc-only and alloc-free pools there could be global pools as well: data_stack_pool and system_malloc_pool. Here's an example:
extern Pool data_stack_pool, system_malloc_pool; /* generic dynamically growing array */ struct array *array_create(Pool pool, size_t initial_size) { struct array *arr; arr = p_malloc(pool, sizeof(struct array)); arr->pool = pool; arr->size = initial_size; arr->data = p_malloc(pool, arr->size); return arr; } void array_destroy(struct array *arr) { p_free(arr->pool, arr->data); p_free(arr->pool, arr); } void array_grow(struct array *arr, size_t size) { if (size > arr->size) arr->data = p_realloc(arr->pool, arr->data, size); } void func(void) { struct array *arr; t_push(); array_create(data_stack_pool, 1024); /* ... */ array_destroy(arr); /* not really needed, t_pop() frees it anyway */ t_pop(); } String HandlingThis is where most of the buffer overflows have happened. Nowadays people are more aware of the problem, but many still they do it in unnecessarily hard way.
The Old Ways:- strncpy(), strncat(), snprintf() - only snprintf() of these is easy to use safely but it's still somewhat unportable (Windows). strncpy() doesn't necessarily NUL-terminate and many people misunderstand how strncat() works (ie. in very stupid and difficult to use way).
- strlcpy(), strlcat() - much better replacements to above by OpenBSD. Very unportable, but you can easily create your own ones. But these can still be used unsafely if the buffer size parameter is wrong or if the programmer goes playing around with the buffer indirectly, by eg. appending single characters and missing size checks (yes, I've seen this in software that contained "secure" in it's name).
- Dynamically allocating the amount of wanted memory and then using strcpy(), strcat(), sprintf() and direct accessing. This requires you to be very careful with the string size calculations. I don't understand why so many people think that's not a problem, they have this "If you can't calculate the sizes correctly, you're stupid and you shouldn't be coding at all" attitude. Why bother wasting time with that at all when you could be doing more important things?
- Dynamically growing buffers, used by for example GLIB, vsftpd, qmail, djbdns and Postfix. This is definitely the right way; string manipulation is done through API which discourages - or even disallows - direct buffer manipulation.
There's several slight variations how to implement the dynamically growing buffers. Most work by allocating a new string object, using it and then freeing it. qmail and djbdns uses statically created buffers which are reused and never freed - that's pretty dangerous unless you can be sure the buffer isn't in use anymore. It's actually pretty much the same as having static char bigbuf[8192] which is used by several functions.
There's two ways how to manage a buffer - first being the explicit way done by eg. djb:
struct stralloc str = {0}; stralloc_copys(&str, "string"); stralloc_cats(&str, str_variable); stralloc_catulong0(&str, int_variable);GLIB supports doing this much more easily:
/* portable asprintf() */ char *str = g_strdup_printf("string%s%d", str_variable, int_variable);Or if you wanted a modifyable buffer:
GString *str = g_string_new(NULL); g_string_append(str, "string"); g_string_append(str, str_variable); g_string_sprintfa(str, "%d", int_variable); /* no g_string_append_int() */Some people don't seem to like the printf-style or believe it's unsafe. GCC however gives very good warnings about parameters with incorrect type so I don't think there's any need to worry about.
String API with Memory Pool API SupportNothing fancy here really. Just by being able to specify the used memory pool we can easily allocate strings from data stack. I've made several wrapper functions so that instead of p_strdup_printf(data_stack_pool,...) I can just use t_strdup_printf(). Here's a list of few useful functions:
const char *t_strdup(const char *str); char *t_strdup_noconst(const char *str); const char *t_strdup_empty(const char *str); /* return NULL if str = "" */ const char *t_strdup_until(const char *start, const char *end); /* *end isn't included */ const char *t_strndup(const char *str, size_t max_chars); const char *t_strdup_printf(const char *format, ...); const char *t_strconcat(const char *str1, ...); /* NULL terminated */t_strdup_noconst is there mostly to avoid casting const away in those few situations where it's needed. Because it returns char * it could be more easily mixed with permanent data so it's usage should be kept minimal.
t_strconcat() is one function that I also copied from GLIB. It's a bit dangerous though, the terminating NULL is too easy to forget. I've been thinking about removing it entirely, but it's much more efficient than t_strdup_printf() so I haven't yet had the heart
Buffer Handling :)Many people concentrate only on string related buffer overflows. Yes, they've been the most common ones but they're quite known now and there's more to buffer overflows than just them. The most obvious one is just another type of buffer handling. All the other data that couldn't be called strings. Often people just hack away separate memory allocations and/or size checks for them. The problems are exactly the same as with strings.
If we create a buffer API and write to buffers only through it, we would prevent almost all buffer overflows, even integer related ones, simply because the buffer API implementation is the only part of code that directly writes to memory, and that of course should be well audited not to overflow in any situation.
Buffer API would be somewhat similar to string API, ideally the string API should be created using the buffer API, or they could even be the same. My buffer API currently looks like this:
/* Create a static sized buffer. Writes past this size will simply not succeed. */ Buffer *buffer_create_static(Pool pool, size_t size); /* Create a static sized buffer. Writes past this size will kill the program. */ Buffer *buffer_create_static_hard(Pool pool, size_t size); /* Create a modifyable buffer from given data. For example: { char buf[1024]; buf = buffer_create_data(pool, buf, sizeof(buf)); } */ Buffer *buffer_create_data(Pool pool, void *data, size_t size); /* Create a non-modifyable buffer from given data. */ Buffer *buffer_create_const_data(Pool pool, const void *data, size_t size); /* Creates a dynamically growing buffer. Whenever write would exceed the current size it's grown. */ Buffer *buffer_create_dynamic(Pool pool, size_t init_size, size_t max_size);Then there are several write, append and truncate functions and a few functions to temporarily limit the accessible buffer range. An example:
int fill_buffer(Buffer *buf, struct data *data) { struct bufdata *bufdata; /* these return value checks aren't really needed, since they fail only if the buffer gets full. and since we're using dynamically growing buffer with no limits, malloc() failure will kill the program long before reaching the high limit. */ if (buffer_append(buf, "data\n", 5) != 5) return -1; /* returns a pointer to just appended space */ bufdata = buffer_append_space(buf, sizeof(*bufdata)); if (bufdata == NULL) return -1; bufdata->field1 = htonl(data->field1); bufdata->field2 = htonl(data->field2); return 1; } int write_data(int fd, struct data *data) { Buffer *buf; size_t size; int ret; t_push(); buf = buffer_create_dynamic(data_stack_pool, 256, (size_t)-1); if ((ret = fill_buffer(buf, data)) > 0) { if (write(fd, buffer_get_data(buf, &size), size) != size) ret = -1; } t_pop(); return ret; }There is of course also the possibility of buffer overflows while reading data. That's a bit more difficult to prevent except by careful coding, but usually it's not such a big deal anyway. It might crash the program, or at worst could expose some private information to attacker. If the later is a real problem, you should use multiple processes and IPC to hide the private data.
Real World UsageI've written a fully featured IMAP server using pretty much these techniques. The code still needs some cleanups and there's probably a couple of bugs left, but mostly I think it's not too bad
:) The library functions in it are MIT licenced, so go ahead and rip them to your own programs. I could try to create some minimal library out of them if there's enough interest.vsftpd uses it's own string API for pretty much everything. In only few of the files it's even possible to create a buffer overflow with the used coding style.
-
very good point!
I think many people could do themselves a great service by simply studying the vsftpd security libraries. Those are generic enough to be used by almost any other application, and provide a solid foundation to write applications that do not fall apart under classic C exploits: buffer overflows, etc.
Check here the vsftpd website.