Slashdot Mirror
Apple's Grand Central Dispatch Ported To FreeBSD
bonch writes "Apple's Grand Central Dispatch, which was recently open sourced, has been ported to FreeBSD and is planned to be included by default in FreeBSD 8.1. Also known as libdispatch, the API allows the use of function-based callbacks but will also support blocks if built using FreeBSD's clang compiler package. There's already discussion of modifying BSD's system tools to use the new technology." The port was originally unveiled last month at the 2009 Developer Summit in Cambridge. Slides from that presentation are available via the Dev Summit wiki.
Always taking from the open source community, and never giving back!
#DeleteChrome
My first question was "So...what does this do?" Apparently it is a more efficient way of scheduling threads on multi-core systems http://images.apple.com/macosx/technology/docs/GrandCentral_TB_brief_20090903.pdf apple's site says this: "Grand Central Dispatch (GCD) in Mac OS X Snow Leopard addresses this pressing need. It’s a set of first-of-their-kind technologies that makes it much easier for developers to squeeze every last drop of power from multicore systems. With GCD, threads are handled by the operating system, not by individual applications. GCD-enabled programs can automatically distribute their work across all available cores, resulting in the best possible performance whether they’re running on a dual-core Mac mini, an 8-core Mac Pro, or anything in between. Once developers start using GCD for their applications, you’ll start noticing significant improvements in performance. " So this seems good then.
Apple maintains their own gcc fork which supports blocks/closures.
Do you even lift?
These aren't the 'roids you're looking for.
now it can die in parallel, optimized for the number of cores and other system activity.
Do you even lift?
These aren't the 'roids you're looking for.
Netcraft confirms it: This joke is dying.
GCD is a mechanism to let one central authority dispatch threads across multiple cores, for all running applications (including the OS).
The other two things you mentioned are ways for programs to more easily use multiple threads, but they are still threads under the main process and not centrally managed - so you have to decide blind how much of the system you can take for your threading needs.
You can compare the Blocks syntax to ways those two systems specify work units to be done in parallel, but that is just a part of the story.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Grand Central is not introducing multithreading - it's introducing comprehensive thread management. So, how many threads are you going to spin for that task? Too many, and you waste a lot of time on thread management and preemption. Too few, and you have processors sitting idle. Now how will you handle this with multiple CPU's? Multiple cores? Hyperthreading? Different cache amounts and layout? OpenCL and GPU processing? Do you know what the rest of the operating system is doing to plan appropriately?
In short, your program can at best make a stab at these issues, and possibly even do a reasonable job if you put a lot of time, effort, and profiling into it. Or you could just use GCD, and let the framework handle it all for you, regardless of whether you're on a Core Solo Mac Mini or a Mac Pro with mutliple OpenCL graphics cards.
It's good stuff. And Apple gave it to the community (much like WebKit enhancements, launchd, etc).
I don't know what kind of crack I was on, but I suspect it was decaf.
it seems like the ability to share work across machines, not just cores, would be a critical difference.
Richard Stallman does not cry into his beer. Microsoft cries into their beer. Richard Stallman cries into his freedom.
Yep, apple didn't give back all their code on WebKit, or all of Darwin, or all of launchd, or all their patches on zfs, or their code on MacPorts, or darwin streaming server, or CalDav, or iCal format, or their Calander server, or their code on their X server, or their code on ruby, or a bunch of code on smart card services...
Wait, yes they did.
Thanks goodness for the GPL, or we might never have convinced Apple to release its code so that FreeBSD could use it!
Wait... what is that? Oh, nevermind then...
"Freedom" is what he calls his beard.
it seems like the ability to share work across machines, not just cores, would be a critical difference.
Neither GCD nor OpenMP allow you to "share work across machines".
On their stolen computer next to all the other things they stole from school. They even steal the flies from outside who swarm around their unwashed bodies. linux is the devolvement of humanity - a linux user is like an anti-human - taking the opposite form of what a real human should be. They do not know how to reproduce, they cannot interact with others and they long ago forgot the concept of personal hygiene. They believe communism is something we should strive for and believe klingon is a real language. yes... we know of your type.
That's not quite correct. There are two parts to GCD: libdispatch and the kernel support. The kernel support is MIT license, so is compatible with GPL. Besides, the kernel component wouldn't be ported anyway--it would be rewritten from scratch if one were doing GCD Linux, so the license doesn't matter.
Libdispatch is Apache license. It runs in the applications, not the kernel, so the kernel being GPLv2 is irrelevant. What's relevant are the licenses of the applications that might want to use libdispatch. Many of those will be licensed as GPLv2 or later. Those are OK, because Apache license is compatible with GPLv3.
Finally, if libdispatch were to be included as a standard part of Linux, even GPLv2 code could dynamically link to it, because it would fall under the system library exception.
Yes, WebKit is based on KHTML; Apple forked it, IIRC.
SSC
If you're creating a serial queue anyway, you no longer have parallelism. If you have multiple serial queues, you may as well have had multiple threads with no interlocking between them. This is just yet another API to do what competent parallel system programmers have been doing since the first thread.
Sam ty sig.
Several things didn't need to be open sourced, such as WebObjects and GCD. Quit your crying. FOSS zealots love to whine about Apple only doing what's required, but they fail to realize that it's a symbiotic relationship. Apple can use existing code to fit their needs, and in return, the open source community gets all of the improvements made by professional coders. It's a win/win, but the mini-Stallmans will never see it that way.
why won't gcc take the patches?
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Because Apple is sticking with GPLv2
It's a win/win, but the mini-Stallmans will never see it that way.
To the contrary: I am a huge fan of Stallman's philosophy and see Apple's work as win/win.
The people that are complaining are the looney Apple Haters, who try and find any point possible by which to attack Apple - never realizing until it is to late the latest position they are attacking from makes no sense.
Please do not taint all those of us who respect the GPL with the same brush the Haters paint themselves in corners with.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
It will support C++ soon enough.
The benefits of clang are that it uses llvm as the compiler, which produces better optimized machine code than gcc currently does. Also, it supports Apple's block syntax (kind of like a pointer to a function), which allows things like libdispatch to do its magic. Also, as a C front end, it has much less cryptic error messages, and actually does a pretty good job of finding missed initializations and other hard to find bugs that usually will get caught at the code execution stage.
You should check out Siricusa's more thorough explanation at arstechnica, in his Mac OS X 10.6 Snow Leopard review. He goes into some detail to show how Apple's use of clang in the new XCode is almost "pornographic for developers..." You wouldn't have the same IDE in BSD or Linux, but the same functionality is there. Someone posted a link much higher in the thread.
-- Len
When is the last time that Apple released an entirely new project? For example Sun released ZFS under the CDDL.
Why reinvent the wheel? They adopt it, improve it and release the improvements. At least they add. They deliver. When is the last time Stallman released an entirely new project? Or finished one? "Although nearly all components have been completed long ago and have been in production use for a decade or more, its official kernel, GNU Hurd, is incomplete and not all GNU components work with it." from the reliable-as-ever Wikipedia. Apple is not Stallman. They do not serve the same market. Apple is in hardware/software/platform, Stallman is in philosophy. Fair enough, einen Unterschied muß es geben.
All those moments will be lost in time, like tears in rain. Time to die.
The probability that Apple migrates away from gcc is approaching 1 at great speed.
That writing's been on the wall for quite a long time now. GCC has been a severe limitation on what they could do with Xcode for far too long. With LLVM, I'm expecting shortly to get away from the edit/compile/debug cycle and have a pause/edit/resume cycle instead.
Right now in Quartz Composer, you can write GL SLANG shaders that are compiled on the fly as you type. It's amazing to see the effects of changes immediately.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
This is just yet another API to do what competent parallel system programmers have been doing since the first thread.
RTFM, and learn what you're missing.
The point of GCD is that taking advantage of multiple cores becomes much less work. When I say "much less", thinks minutes versus days. Besides the ease of use, the underlying implementation also deals with scaling the queues across the number of available cores on the fly, and makes the management of serial dependencies trivial.
Another benefit of GCD over managing the threads yourself is performance. Apple changed the implementation of Objective-C's garbage collector from running on a dedicated thread to running in GCD, and picked up a double-digit improvement in the speed of memory recovery.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Well, we have the subject of this article, Grand Central Dispatch. We also have Darwin & XNU, their version of the Mach kernel. There is also Bonjour, their version of zeroconf, and their streaming server (Quicktime Streaming Server). They also purchased the source to gimp-print (now called Gutenprint) so they no longer have any obligation to keep it open source, but they do, and they keep releasing the source. How much more do you need?
Check out DRM-free movies at http://www.bside.com
Apple isn't "sticking with GPLv2" so much as "actively working to replace every last scrap of GPLvn code with BSD/MIT/Apache code." They appear to be under the impression that GPL is unfriendly to business, which impression is defensible. They are investing many programmer years in Clang/LLVM, and soon enough will be liberated from the tyranny of gcc.
If you mod me down, I shall become more powerful than you could possibly imagine.
But apart from CUPS, LLVM/clang, Webkit, Darwin, launchd, patches for zfs, MacPorts, darwin streaming server, CalDav, iCal, the Calendar server, code for Ruby, code for X server and GCD, what has Apple ever done for us?
Webkit is kinda special. For ages, Apple did nothing other than releasing the full source every time they released binaries.
They did _not_ release anything under SCM control, which meant tracking patches etc was _literally_ impossible for the KDE team. No matter what the KDE team did or how often they asked, this did not change. This is a large part of why KHTML and Webkit are as different as they are now. Using both KHTML and the Webkit Kpart in KDE 4.3.2, I can tell you that there are a lot of little differences in rendering and, that is where it hurts, usability. Sucks.
So while Apple followed the letter of the license, they went against its spirit in every possible way for a long time. This may have changed somewhat (iPod, iTunes...?), but one should be aware of the history.
A massive case of NIH from the GCC team with regard to Apple. See also the patches that Apple submitted well over a year ago adding declared property support to GNU gcc. I found it easier to write all of the code required for clang to support the GNU Objective-C runtime than make even small changes to Objective-C support in GCC (in spite of not having used C++ for a few years and really hating the language).
Oh, and I've had blocks working with clang on FreeBSD for quite a while (since several months before Apple publicly released an OS supporting them, in fact). I added the required runtime library support into Etoile's ObjectiveC2 framework a while ago, which also provides the run time support for most of the Objective-C 2 features. This framework is going to go away soon, because I'm now maintaining a fork of the GNU Objective-C runtime in the GNUstep repository, which merges these improvements and also incorporates most of the ideas from my experimental Objective-C runtime library (without breaking backwards compatibility, although you don't get all of the features if you don't use the new ABI).
By the way, porting to *BSD is much easier than porting to Linux. Libdispatch is based around the kqueue mechanism for unifying kernel event sources, and there is currently no in-tree equivalent for Linux. There are four out-of-tree sets of patches providing equivalent functionality (that I know of, there may be more), as is common with Linux development. On Solaris you can do the same thing with completion ports, which are roughly semantically equivalent but have a different interface.
I am TheRaven on Soylent News
From the professional coders at AT&T?
(ducks...)
In Soviet Washington the swamp drains you.
This is between one and two orders of magnitude more expensive than adding a block to a work queue with GCD.
Switching between the threads is also relatively expensive. At the very least, you need to save all of the registers and update the stack pointer (probably in response to a timer interrupt, so also factor in the cost of a transition from kernel space to users pace). With GCD, the kernel will give you one thread per CPU for each priority level (or fewer if there are already lots of CPU-bound threads), so you minimise number of context switches.
This is effectively what an N:M scheduler does, but is slightly better. With an N:M scheduler, you are switching between some threads in userspace, which saves some overhead, but the threads still have largely-disjoint working sets and so you end up with more cache churn (performance killer on modern CPUs). With GCD, you will only be switching between work queues in a given thread between blocks, so there is no cache churn (the old block's cache lines are no longer needed, so you'll kick them out but not then bring them back in immediately). The other nice side effect of this is that, because GCD knows which work queues are assigned to which threads, it can remove locks used for passing messages between work queues if both queues are in the same threads.
In short, it's an N:M threading model with preemptive kernel threads and cooperative userspace threads. If you read an OS textbook, you'll see that this is, at least in theory, about the most efficient scheduling model possible.
I am TheRaven on Soylent News
Uh, yes? Sure, you *made* them give up their source if they used your project, but that doesn't change the fact that they *did* use your project, and then contribute back hundreds of thousands of improvements. Or are you suggesting that khtml without apple's assistance would be in the position WebKit is now?
In the mean time... When is the last time that Apple released an entirely new project?
You do realise this article is *about* apple releasing the source code to libdispatch?
It goes further and deeper(and uglier) than that. Remember when Apple released the G5's? Apple actually submitted patches to GCC, but they were declined, with GCC's official stance being "It would reduce GCC's portability". However, a few weeks later, IBM submitted some patches for GCC, that were summarily accepted. IBM's patch package contained many of Apple's patches for GCC.
when we start using octal cores
High end workstations are already shipping with dual quad (8 cores total) and will probablly be shipping with dual hex (12 cores total) in the not too distant future. Quad core is fairly common in the midrange and is even available on some fairly low end machines (e.g. the dell vostro 420). Laptops and low end desktops are generally shipping with dual cores.
With the exception of nettops and netbooks single cores are pretty much history.
The bottom line is that cores aren't getting that much faster so most of our gains in computing power are likely to come from more cores and software that needs to get the most out of CPUs is going to need to embrace that and soon.
note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register