When Will Linux Have Real Threads?

An Anonymous Coward asks: "After reading the article detailing Microsoft's gripes about Linux, one gripe that wasn't specifically mentioned came to mind, "When will Linux have *real* threads?" Many of the developers I work with, including myself, prefer Solaris as an enterpise platform over NT/2K as well as Linux and *BSD because it offers truly lightweight threading and better thread scheduling than the others. Linux does have thread libraries available but they all map threads to Linux kernel processes. Also, all of the thread synchronization and scheduling tweaks that you can do in a program on Solaris don't seem to work as expected/desired on Linux. Are there any projects out there that are trying to put true threading in the Linux kernel and libs?"

Comments

Portable thread library for Linux and others

[goingware]

The ZooLib cross-platform application framework has a platform independentent C++ thread and mutex library for Linux (and other POSIX), Win32, Mac OS (with handrolled threads), and BeOS.

I'm pretty perplexed why Linus should think that thread programming is harder. This makes me wonder whether he's ever really tried in a serious way - a lot of programs are much easier to write when you use threads.

Just ask anyone who's had to write their own task scheduler for private use within one program, as I have when writing my Raindrops screensaver for the Mac, or when I wrote a test tool for MacTCP, that enables up to 64 TCP streams and UDP pseudo-streams between any combinations of a Mac on a network - I wrote that in 1990 at Apple, well before threads were available on the Mac OS and it was a pain.

But personally I don't see the point of having threads not implemented in the kernel. It means all your threads block if any of them do. That's the point of kernel threads. I think solaris threads work the way they do because the solaris schedular isn't so efficient, so it's more efficient to multitask some of your threads in user space.

This also relates to why multithreaded programs are more efficient and responsive than single threaded ones, even on uniprocessor machines. If your process blocks to await a resource in a single threaded program, you get to wait while time is wasted. This is OK for servers where avergage efficiency is the main concern but is not appropriate for interactive or near-real-time use.

With a multithreaded program, if one thread blocks for a resource, another thread can run, and if it's another thread in the same program your program's utilization of the CPU will be greater. This is especially the case if the kernel is preemptible and multithreaded, as it is on the BeOS and can be with experimental patches on the Linux kernel.

For an experiment, using the same machine, compare the interactice responsiveness of Windows, the BeOS and Linux, and I can assure you that even for uniprocessor machines the BeOS will be much more efficient, because all native BeOS programs are multithreaded. Threading is available on Linux and Windows but are not yet that widely used.

Michael D. Crawford
GoingWare Inc

Resource blocking

[localroger]

While the kernel can deal with it, user-space state machines can't deal with the problem of blocking on a resource.

WTF??? You mean you don't have a way of checking whether a resource is available without blocking your thread so that you can go do other processing while the resource is unavailable?

I have done a lot of programming on proprietary pidgin-language controllers where I have had 4 or 5 state machines going at once (describing the behavior of real-world machinery, generally) and not one of these devices supports threading. (Well, one does now, but nobody uses it because it's so hard to debug.)

While it is re-inventing the wheel, the advantage of doing your own threads by implementing multiple state machines is that you can make your own decisions about what is important. In an embedded environment this can make the difference between an app that will and one that won't work on the available hardware.

Linus hates threads

[Ledge Kindred]

His opinion is that there is very little, if anything, *useful* that thread-based programming brings to the table other than unneeded complexity. (I tend to agree though I've not done a whole lot of threads programming.) Any time someone pops into kernel-list and claims umpteen years experience at threads programming and how wonderful it is to have "real" threads, Linus will invariably offer his condolences and then tell them how silly they've been by dealing with all those concurrency, locking and other threading issues when they could have just done it the simple way by not using threads to begin with.

For some great background information read:

here

here

and here

No, not everything linked there is going to be entirely 100% perfectly relevant, but it's good background for understanding. Those articles also have links to other articles discussing various issues relating to good coding practice and thread-programming under Linux etc.

Then, if you want to compare Microsoft threads to Linux threads, at some point someone (probably Linus) pointed out that Linux can fork() many times faster than Microsoft can spawn a new thread, and in fact creating a new thread and fork()ing a new process under Linux both consumed exactly the same amount of overhead, so trying to claim that Microsoft's thread-based model was more resource-friendly than a Linux/UNIX-like multi-process programming model is ridiculous. You might find this discussion in one of the links referenced above, you might not, I'm not gonna go through the effort of looking for it though.

-=-=-=-=-

Linux has threads, but...

[Chris Siebenmann]

As a number of people have already said, Linux has kernel-supported threads (multiple threads of execution in a single memory space, where one thread performing blocking IO does not block other threads). The Linux kernel does not have a special implementation of 'threads' as distinct from normal processes; instead threads are processes that share (among other things) their virtual memory mappings. Linux also has a nearly complete and correct implementation of POSIX threads, in glibc.

However there are a few things Linux does not have in its thread model:

• Full POSIX thread semantics. Various people (including Linus) think that some of the pickier requirements of POSIX threads are either braindamaged or very hard to implement in the kernel without performance hits or both. Some of this is worked around by the POSIX thread library that is part of glibc, but at a performance cost.
• A system for using many user-level threads and fewer kernel-level threads. I don't know if there are any obstacles preventing this from being implemented entirely in user space, though. The N:M model is apparently common in various other Unix systems implementation of kernel supported threads.
• Efficient kernel scheduling for huge numbers of simultaneously runnable processes or threads. The Linux scheduler does not cope well when you have hundreds or thousands of threads all ready to run.

  This is a problem for the obvious implementation of Java runtimes, as Java programs often spawn very large numbers of threads for various reasons. IBM, among others, have proposed scheduler patches to deal with this; however, all of the patches have hurt performance for the usual/common case of only one or two runnable processes on the system. This is getting to be a hot issue for active several-way SMP machines too (which may have a process or two per processor, adding up to eight to sixteen runnable at once), so patches may get accepted at some point or may start appearing in 'enterprise edition' kernels from various distributions.

Also as people have said, Linux does not need any special minimal 'thread' kernel object: Linux process objects are already as fast (or faster) and as lightweight as thread objects in other systems. Only on some old legacy Unix systems do 'processes' necessarily equate heavy-weight, slowly scheduled and heavily resource-consuming objects. Both Linux and Plan 9 show that full-scale regular processes can also be lightweight and fast, and easily be used to represent threads too.

start with glibc (not slashdot)

[The Pim]

First, thank you for basing your complaint on Microsoft marketing material, vague anecdotes, and loaded terminology. That said, ...

The "linuxthreads" add-on to glibc is heavily used and actively developed--a quick look at the mailing lists and changelogs will convince you. Both POSIX pthread compliance and performance are high priorities, and many users evidently find the thread support quite "real".

As for performance, I do know that a Solaris-like model is under consideration for glibc, although plenty of people (especially kernel developers) have well-founded objections. It might help if you could provide details on why you don't find the current model "lightweight" enough, such as performance numbers from a real application. You might even get some ideas on how to improve your application.

As for features, new functionality is always being added to linuxthreads. If you described exactly which functionality you need on the glibc lists (and on the kernel lists, if applicable), or pointed out shortcomings in existing functionality, you'd probably get some helpful replies. It's acknowledged that some things aren't there yet, but this is most definitely considered a bug (by glibc people, at least), and the priority of bugs is influenced by user input.

Hoping for a new project to appear is misguided when the glibc/linuxthreads wizards have done most of the job, and remain active and responsive. Your best bet is to work with them.

P.S. Linus does not hate threads--think of how ridiculous it would be for a SMP kernel hacker to condemn concurrency within a single address space. He added clone() to Linux, which enables just that! What he hates are POSIX threads.