Slashdot Mirror
Apple Open Sources Grand Central Dispatch
bonch writes "Apple has open sourced libdispatch, also known as Grand Central Dispatch, which is technology in Snow Leopard that makes it easier for developers to take advantage of multi-core parallelism. Kernel support is not required, but performance optimizations Apple made for supporting GCD are visible in xnu. Block support in C is required and is currently available in LLVM (note that Apple has submitted their implementation of C blocks for standardization)." Update: 09/11 15:32 GMT by KD : Drew McCormack has a post up speculating on what Apple's move means to Linux and other communities (but probably not Microsoft): "...this is also very interesting for scientific developers. It may be possible to parallelize code in the not too distant future using Grand Central Dispatch, and run that code not only on Macs, but also on clusters and supercomputers."
Everyone who reads slashdot isn't an OSX ween and has no idea what "Grand Central Dispatch" is. Perhaps a sentence or two describing why it is important/useful would save users from following the link which doesn't provide that info either.
I want those 5 seconds back! :)
Can somebody explain what this "blocks" is? I mean, C being a block-structured language, I thought it already supported them...
I'm not too well versed in Cocoa development. I pushed some code that should have been in a separate thread into GCD, which requires you to use a block. All in all, I had to add an include, 1 line of code and a closing bracket.
Apple has made some seriously cool stuff here.
the question is:
What license? Apache v2.0
What the fuck is GCD?
Grand Central Dispatch (GCD), named for Grand Central Terminal, is used to optimize application support for multicore processors. It is an implementation of task parallelism based on the thread pool pattern .NET Framework developed by Microsoft.
GCD works by allowing specific tasks in a program that can be run in parallel to be demarcated as blocks.[2] To this end, it extends the syntax of C, C++, and Objective-C programming languages.[2] At runtime, the blocks are queued up for execution and depending on availability of processing resources, they are scheduled to execute on any of the available processor cores[2] (referred to as "routing" by Apple).[3]
see also
# Task Parallel Library - comparable technology in the
# Java Concurrency - comparable technology in Java (also known as JSR 166).
IranAir Flight 655 never forget!
It's a library for task parallelism using a thread pool, introduced in Mac OS X 10.6 (Snow Leopard). Wikipedia tells all.
"We recognize that libdispatch is a new technology and you likely have many questions. Here are some documentation resources for getting started:
Introducing Blocks and Grand Central Dispatch
Concurrency Programming Guide
Grand Central Dispatch (GCD) Reference"
Blocks:
In Snow Leopard, Apple has introduced a C language extension called "blocks." Blocks add closures and anonymous functions to C and the C-derived languages C++, Objective-C, and Objective C++.
Perhaps the simplest way to explain blocks is that they make functions another form of data. C-derived languages already have function pointers, which can be passed around like data, but these can only point to functions created at compile time. The only way to influence the behavior of such a function is by passing different arguments to the function or by setting global variables which are then accessed from within the function. Both of these approaches have big disadvantages
Full Read: http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/10
Directly in line with blocks is Grand Central Dispatch (and this is, where blocks become really usefull):
GDC is a a technology to resolve the concurrency conundrum by giving programmers a very easy way to split tasks into multiple sub-tasks which can then be loaded onto different threads/cpu. All this also works with normal threading, but GDC makes the process far easier, with the intention to prepare OSX for future multicore machines:
http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/12
It does so by using blocks as separate tasks:
http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/13
"When I first heard about Grand Central Dispatch, I was extremely skeptical. The greatest minds in computer science have been working for decades on the problem of how best to extract parallelism from computing workloads. Now here was Apple apparently promising to solve this problem. Ridiculous.
But Grand Central Dispatch doesn't actually address this issue at all. It offers no help whatsoever in deciding how to split your work up into independently executable tasksâ"that is, deciding what pieces can or should be executed asynchronously or in parallel. That's still entirely up to the developer (and still a tough problem). What GCD does instead is much more pragmatic. Once a developer has identified something that can be split off into a separate task, GCD makes it as easy and non-invasive as possible to actually do so.
The use of FIFO queues, and especially the existence of serialized queues, seems counter to the spirit of ubiquitous concurrency. But we've seen where the Platonic ideal of multithreading leads, and it's not a pleasant place for developers.
One of Apple's slogans for Grand Central Dispatch is "islands of serialization in a sea of concurrency." That does a great job of capturing the practical reality of adding more concurrency to run-of-the-mill desktop applications. Those islands are what isolate developers from the thorny problems of simultaneous data access, deadlock, and other pitfalls of multithreading. Developers are encouraged to identify functions of their applications that would be better executed off the main thread, even if they're made up of several sequential or otherwise partially interdependent tasks. GCD makes it easy to break off the entire unit of work while maintaining the existing order and dependencies between subtasks." (source = above url)
Apple has a long tradition to make things its own way, not always in the cleanest way under the hood. This does not always help, and sometimes it just adds to the confusion. Why parallel programming has to be tied to a kernel change and to a language spec change, when a good library (OpenMP, anyone?, but I'm sure there are others) will suffice... Good support for OpenMP or any of the existing shared memory parallel programming libraries would have been much cleaner and portable.
Disclaimer: I use/program/work with an Intel Macbook Pro.
Why would Apple suddenly open-source a major feature of their new OS? Did they get busted violating the GPL? It doesn't make sense that the most closed-source draconian control-freak vendor of all would suddenly open-source major parts of their OS without some legal Sword of Damocles hanging over their head. What aren't we being told here? Whose code did they get caught stealing?
Someone somewhere owes the community an explanation!
Just looked at libdispatch source code. It is apparently very Mac OS X specific, and will take a lot of work to be ported cross platform. The queue implementation also looks like it imposes a lot of overhead, so it is not very useful for parallelizing short-running "blocks" of code.
Why not use something like Cilk which has proven its salt? The only caveat of Cilk is that it requires any parallel code to be compiled with the Cilk compiler (which will generate C code), and unless you're willing to kickstart the Cilk runtime manually, any caller of the parallel code needs to be compiled with Cilk too, which includes main(). The technical reason is Cilk does a continuation passing style transformation (requiring a different calling convention) which allows a function caller to be stolen by the work-stealing scheduler while the function waits for the callee to finish, allowing the critical path to focus on work instead of manipulating the task queue. The result is a highly efficient and scalable parallel execution runtime, the best I've seen.
I once had a signature.
It looks like GCD is very similar to OpenMP. I am always biased toward using an open standard, when possible. Since many compiler vendors support OpenMP, why didn't apple just implement that for Objective-C, instead of creating their own threading solution? Judging from the examples, GCD looks much cleaner and simpler. But that often comes with a price.
Not. Invented. Here.
You should just check the apple web site.
/. was how wonderful and easy-to-use the Apple web site was. If you can't find the answer, including full documentation on the site, you don't need to worry your pretty little haad about it. We're Apple, marketing (and really clever naming) rules.
Yesterdays top news in
See below for marketing jargon speak:
Grand Central Dispatch (GCD) in Mac OS X Snow Leopard addresses this pressing need. It's a set of first-of-their-kind technologies that makes it much easier for developers to squeeze every last drop of power from multicore systems. With GCD, threads are handled by the operating system, not by individual applications. GCD-enabled programs can automatically distribute their work across all available cores, resulting in the best possible performance whether they're running on a dual-core Mac mini, an 8-core Mac Pro, or anything in between. Once developers start using GCD for their applications, you'll start noticing significant improvements in performance.
OpenMP was not invented by Apple, and cannot be used as a lever to undermine gcc.
no need for kernel support
open source implementation
yeah. sounds a helluvalot like vendor lock-in to me
More importantly... "why bother?".
It has a very "big sounding" name for what it is.
A Pirate and a Puritan look the same on a balance sheet.
Thread scheduling is already handled by the operating system. This is OS 101 stuff from the 80s. What ELSE does GCD add?
My multi-threaded apps can already monopolize every CPU on my
box whether it is a 4 core desktop or a 64 core server. Been
that way for a LONG time.
A Pirate and a Puritan look the same on a balance sheet.
Locked into open source software? I can think of worse things.
Absolutely true, but I want to point out that Apple's just trying to make simple things easy -- they're not trying to say that they've finally solved truly capital-h Hard concurrency problems.
lorem ipsum, dolor sit amet
There is no tech that can help you with that. Maybe Apple should solve the solvable problems first (making parallelizing easy) before tackling the unsolvable ones?
What you meant to say was, "Everyone who reads Slashdot isn't an internet weeny, and has no idea how to find out what a technology of which they haven't heard before might be, other than imploring (counterproductively with a snide remark) to be spoon fed by other participants in the discussion." The answer is the wonderful invention, Google. You can type a word or phrase, like "Grand Central Dispatch", and Google will present you with links to document which discuss it. Oftentimes, one of those documents might appear at the web site of an online community encyclopedia, known as Wikipedia, which not only describes the thing in question, but also frequently provides convenient links to additional information.
If you mod me down, I shall become more powerful than you could possibly imagine.
You and I must not be reading the same code.
libdispatch constantly references mach throughout, which is unique to OS X and would take ages to #ifdef out completely. It also makes use of features only available in GCC 4.2.x or later, which means that many older platforms are out in the cold. It also makes a ton of platform assumptions (though luckily many of these are #define'd, so they can be redefined for other platforms later, however there's plenty that isn't, and is coded specifically for Intel platform performance). Furthermore, it codes in some really weird Apple-specific things (like label names for the priority queue levels, #ifdef's for Cocoa support, other very strange code, not clean in anyone's definition).
With some significant work by an interested party, it could be made portable, but Apple didn't Open Source it with the idea that it'd immediately build on non-x86, Linux, or Windows (and in fact, it might be ages before it builds on the latter, using pthreads and functions like sysctlbyname(3) throughout). It could be portable... some day. But as it stands it's not. At all.
Shoot, I already noticed the difference on my 2.5 yr old Mac Pro (1.1). First boot on 10.6 and I was like "wow, feels like a new machine again". All of the bundled apps have been recompiled (64 bit) and cleaned up (and apparently take advantage of GCD everywhere possible). I really didn't think I would see that much of a difference with 10.6 and really only upgraded because I could for $29 (I mean at that price, why not right?) I am very happy with my $29 purchase thus far. I've only had to work through a couple app incompatibilities (and as I have been able to work around them just fine, I am happy.) This is of course just my experience thus far with 10.6. I have no hard benchmark numbers for you. But I noticed right away the smoothness it brought to my older Mac Pro. And it was an easier upgrade than going from 10.4 --> 10.5.
You are posting in a thread about the fact that Apple made their implementation open source and you are claiming vendor lock-in?
Are you one of those rabid Apple-haters we see so often around here? Or are you just amazingly stupid?
I'd say adding anonymous functions to C is more than just 'big sounding'.
Yup.
I know the C-like language geniuses won't jump on Erlang immediately, but the multi-core support is awesome. I'm pretty sure there's a port for every platform too.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
What I really want to see next is libxgrid so that I can use my debian/windows boxes as Xgrid nodes.
My university had around 20 computer labs, some were packed, but some were completely empty. I understand that it'd use more energy, but if you could turn those into a cheap 'super computer' and loan/rent out time to groups on campus.
This isn't offtopic, it's perfectly accurate. Modded +1
People now say "multi-core" where they used to say "SMP." Deep down, they always mean the same thing (since people are always talking about software), so why did they change their wording? Because the most easily-available hardware changed. But the problems and solutions didn't change at all. It all sounds so new and sexy.
A non-standard compiler extension that is guaranteed to be supported by the vendor's compilers is pretty much the definition of vendor lock-in, even if the implementation is open-sourced. If other compiler vendors don't pick it up (and they won't with a standards-based alternative) all code that uses it becomes tied to the vendor.
If functions can now be data passed around as variables, doesn't this reduce the security of an application? Granted, you don't need to use blocks but i can envision a future world where internet server software is written heavily in blocks. What if a block makes it's way into the system from the outside world? Its the same thing as a code injection hack but this technology could potentially make it much more simple to do that. Yet another security concern.
Given the new-ness of the technology it seems like some of these security issues need to be worked out... patterns developed and such.
This technology strikes me as a solution for user experience issues and not bullet proofed for server solutions. I may be wrong.
Not the whole problem, but actually, yes, some of the problem is with languages.
And some problems are not hard to multi-thread, and yet reasonably competent programmers end up sometimes screwing it up anyway. They make mistakes not because they're stupid, but because (for example) they're in a hurry. (All professional programmers are in a hurry. Ship it! Next work order!!)
There are ways we can make that situation easier.
Ah, the old "power is dangerous" argument. You're right, and you're wrong, at the same time. Or rather, you are correct and irrelevant. Foot-shooters' problems are their problems. Your problems are your problems, and you don't want to have them. It's ok to make things easier for you. Don't worry about That Other Guy fucking up.
"Believe me!" -- Donald Trump
Erlang is pretty cool, but I doubt the linux kernel or any other massive c codebase is going to be rewritten in it. GCD is an incremental improvement, that will be more likely to have a bigger impact.
Well.. maybe. Or Maybe not. But Definitely not sort of.
What if you're running two applications that both are capable of monopolizing all your cpu time? How will your app know that it's only going to get 50% of the available cpu time form the OS, so it should only start threads for half the cpus?
GCD decides how many threads a collection of tasks should be split across. If an app running on an 8-core machine wants to run 100 tasks, then they could be spread across anywhere from 1 to 8 threads, depending on what else is running. Since it's the OS that knows what else is running, it can make more intelligent decisions about how many threads should be running.
Grand central dispatch has many innovations, but the key feature it provides is that thread pooling is now handled by the OS not the program. This means that in a dynamic environment you don't have each application stepping on each other when they ask for too many threads --all total-- than the multi-core system can optimally handle. So if Mail asks for fifty threads and Firefox asks for fifty threads and CPU you are running on can realistically only handle 10 threads then GCD figures out how to manage things so you don't get a spinning beachball.
It turns out a lot of tricks were required to do this including a lot of things like just in time compiling LVM and this C-Blocks stuff, but that's way over my head.
Some drink at the fountain of knowledge. Others just gargle.
I've come to really like GCD; I haven't played with it much in Cocoa (Obj-C) but I've been moving some of the stuff I wrote a long time ago in C to use it and I think I can say that what it does is *really* *really* awesome. It helps when writing code to be run in parallel; it does is not help you in determining *what* should be done in parallel. By putting your work into queues, by way of closures (yeah, blocks, whatever...I'm sticking with the closure name), it's up to the underlying OS to determine what thread gets what work, and on what processor. Having worked with multithreaded stuff on Windows, and calling GetThreadAffinityMask or whatever it was, and being told that it's just a *hint* to the OS, which is free to ignore you, which it always did, GCD really does spread out the work evenly among my 16-proc MacPro, and then turns around and does it just as well on the dual-core mini.
I've wanted something like this for years; a really decent OS thread scheduler that divides up the work on the other processors in a sensible fashion. I was even looking into how much effort it would take to write something like this from scratch for Linux, and now I don't even have to. Sweet!
Caveats: This is in OS X only, so no iPhone GCD (at least, not yet...not really necessary until we have multi-core iPhones), and while I've lived with additions to C++ through the years (templates mostly), the idea of adding, well, anything to C seems strange, let alone something as run-time dependent as closures.
I agree. Furthermore, these "high level language" things are just dangerous toys that let overconfident children shoot themselves in the feet more quickly. It should be the law that if you want to program a computer you have to use a real language - Assembly.
Verbification weirds the language.
Utilizing the synergization of benchmark e-solutions to pre-workaround action items!
Apple did not release Darwin under the BSDL. They re-release third party code under the licenses said code was distributed under, and most Apple written code used the Apple Public Source Licence.
***
Example BSDL:
Copyright (c) ,
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY ''AS IS'' AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
***
Example APSL:
APPLE PUBLIC SOURCE LICENSE
Version 2.0 - August 6, 2003
Please read this License carefully before downloading this software. By downloading or using this software, you are agreeing to be bound by the terms of this License. If you do not or cannot agree to the terms of this License, please do not download or use the software.
Apple Note: In January 2007, Apple changed its corporate name from "Apple Computer, Inc." to "Apple Inc." This change has been reflected below and copyright years updated, but no other changes have been made to the APSL 2.0.
1. General; Definitions. This License applies to any program or other work which Apple Inc. ("Apple") makes publicly available and which contains a notice placed by Apple identifying such program or work as "Original Code" and stating that it is subject to the terms of this Apple Public Source License version 2.0 ("License"). As used in this License:
1.1 "Applicable Patent Rights" mean: (a) in the case where Apple is the grantor of rights, (i) claims of patents that are now or hereafter acquired, owned by or assigned to Apple and (ii) that cover subject matter contained in the Original Code, but only to the extent necessary to use, reproduce and/or distribute the Original Code without infringement; and (b) in the case where You are the grantor of rights, (i) claims of patents that are now or hereafter acquired, owned by or assigned to You and (ii) that cover subject matter in Your Modifications, taken alone or in combination with Original Code.
1.2 "Contributor" means any person or entity that creates or contributes to the creation of Modifications.
1.3 "Covered Code" means the Original Code, Modifications, the combination of Original Code and any Modifications, and/or any respective portions thereof.
1.4 "Externally Deploy" means: (a) to sublicense, distribute or otherwise make Covered Code available, directly or indirectly, to anyone other than You; and/or (b) to use Covered Code, alone or as part of a Larger Work, in any way to provi
I don't mind progress, and new standards and all that, and the idea of a "user-friendly scheduler" is really nice, but how hard would it be to make this work with just generic callable objects? it's not that hard to implement a closure in C, and it's been done for years for things like boost and libsigc++ (any signal/callback system that doesn't have upvalues is useless to me at this point). And it's not like these "blocks" are actually compiled and linked at run time, it's just a pointer to a static function with a bunch of extra data on it. If the API just took a callable object (which could be implemented as a "block" if the functionality was there), there'd be no need to wait for some committee to get together and approve an addition to a standard.
--
Stay tuned for some shock and awe coming right up after this messages!
Mike Ash has a series of articles on GCD. He talks about what it is, how to use it, and why to use it.
http://www.mikeash.com/?page=pyblog/
Please tell me I was not the only one who was expecting a railway signalling system....
Do you honestly think MS is going to pick up GCD? As soon as you start using it for the mac version of your multiplatform software, it becomes more complex to support both platforms easily. You either add a lot more development time (read: money) to add this to only one of your code bases, making it more difficult to track problems that affect both, or you don't use it at all (making GCD useless to you) or you drop Windows support (most likely cutting off 90% of your potential userbase for a desktop application).
Go back to 1chan PS: F40PH
You are posting in a thread about the fact that Apple made their implementation open source and you are claiming vendor lock-in?
Are you one of those rabid Apple-haters we see so often around here? Or are you just amazingly stupid?
I must be amazingly stupid because I rather like Apple products.
Proprietary extensions are done for (arguably) the same reason by Microsoft; the goal should instead be to work on better iterations of language standards (C/C++) and not on introducing arbitrary language extensions that are not portable across compilers - especially not really extremely awkward ones like 'anonymous function pointers.' There's a similar argument to be made for 'encouraging' developers to use C# and Objective C.
I must confess after reading comments here and the Wikipedia article, I'm not sure exactly what's novel here.
The "blocks" concept is nothing more than a repacked version of futures as far as I can tell. C++0x has anaymous functions and futures libraries for C++ already exist. I can see value in adding this for C but I sure hope Apple isn't thinking of trying to introduce a competing proposal for C++. We're way beyond that stage.
And I really don't get the connection to LLVM and jitting. What's the advantage?
As for the programming model, futures or OpenMP seem a much better way of expressing these kinds of things and they exist today. Better yet would be automatic parallelism extracted by the compiler when possible. It's limited to loops mainly, but anything to relieve the programmer from specifying this stuff is a win.
As for the scheduler, it seems Apple has simply decided that kernel threads are more flexible than user threads. That may be true, but this is an old argument. I just don't see the novelty.
Can someone enlighten me? An efficient implementation of threads for MacOS is certainly important so I'm not denying that the technology is important. But I don't understand the hype about it being somehow revolutionary.
Saying something is open source is rather meaningless, just tell us the license name, bitches.
It also adds anonymous blocks to C, which is pretty radical. Read the Ars review on Snow Leopard, and skip ahead to the GCD parts - simply put, it is quite awesome.
GCD seems to be little more than an implementation of Intel's Thread Building Blocks adapted to Apple's platform:
http://www.threadingbuildingblocks.org/
So, in a sense, you as Linux users already have it.
This technology is very valuable to massive single-threaded application such as legacy or poorly-implemented UNIX applications. This shouldn't be necessary on the Microsoft platform, which has been fully multi-threaded as a standard practice since the 90's.
If Apple has made the technology more developer accessible, then this will be a valuable contribution to the Linux platform.
In short, Apple is giving it away because it has no proprietary value. They probably just want free labor for maintaining their fork of TBB.
Yes, obviously the OS X kernel already manages the threads. The thing that GCD manages is creating the threads based on current system work loads and the tasks that you give it. If you give it 1000 tasks on a 4 core machine and the FLASH plugin is hogging 100% of one core(as it does a lot), GCD may start 3 threads and when you quit the web browser, it will likely start another thread.
In general with most multithreaded programs, you probably start 1 thread per core because you can reasonably assume that this will perform pretty well if there's not much else going on. GCD is running at the OS level so it knows whether there is load on one core and if the other cores are free. Thus it will not start too many threads which will cause more context switching and degrade performance.
What I'm more interested in is the overall productivity. I'm tired of seeing operating systems treat threads like multiple cores. An i7 is not an 8 core cpu! There is a difference between threads and cores.
When you look at how threads can share cache with each other it becomes obvious that threads can potentially become more productive than cores especially with tasks that need such power.
So the in the end the question in my mind with GCD is, "Does it identify how much memory each queue needs and if the queues share memory? Does GCD do this to manage everything organizing what goes to what thread allowing a speedup especially when needed? Or does it treat threads the same as cores like everything before it?"
My current computer only has a core 2 duo in it so I can not properly run tests but the second I get an i7 I'm going to run multiple tests with GCD, openCL, openMP, ... and see if anything can properly take advantage of threads managing the cache properly. Maybe this is an over the top openCL type task but regardless I wouldn't mind looking at GCD in detail to understand how it figures out the best way to manage the queues.
imho memory management can be the best speedup, not how many numbers can be processed in a given time.
I'm honestly surprised how ignorant and lazy the regular slashdotter has become with the years.
Any self-respected geek should be already keeping up to date with Apple advancements which are and will be impacting techology in the years to come.
If you people haven't noticed already, Apple has been consistently releasing libraries and server software as open source projects for the rest to pick up , use and modify, with liberal licenses.
A friend of mine used to say (can't remember exactly... paraphrasing:)
* Microsoft wants all software to be theirs
* GNU wants all software to be free
* BSD wants all software to be better
And releasing GCD, gentlemen, is another master stroke by Apple, just like WebKit, Bonjour, LLVM, the list goes on, to share knowledge and advance technology by merit, not by forcing it down your throat thanks to the monopoly you have been handed.
The term "block" is familiar to Ruby programmers. It's an old concept which Ruby has made easy to use and hence popular and actually useful.
And here's another lesson which OpenBSD, Apple and Ruby have been putting to work without you noticing guys: any technology that is difficult to use, no matter how good it is, will not be used if gets in your way; the technology must be easy to deploy/use and unobstrusive to be actually used and useful.
Just remember SELinux and how many people just disable it, no matter how good it is (which I don't think it is, but that's for another rant). Then compare it with the technology that OpenBSD has been implementing for memory protection which is unobstrusive and ready to use with no extra configuration. Same with Ruby blocks, which more programmers are using and a lot of software is benefitting from it now, even though higher order functions and closures have been around for ages.
Having Ruby-like blocks in C and Objective-C is so COOL, you must appreciate that if you think you're serious at programming. Apple has already submitted it to be a standard. I believe MacRuby will benefit from this too, which is Ruby written in Objective-C, which implements Ruby classes as Objective-C classes, achieving incredible speed, taking advantage of Objective-C and LLVM technologies.
Now, I want my late '90s Slashdot back please, where you could more easily find insightful and informative comments. There's a lot of garbage and Microsoft apologists nowadays.
The best way to predict the future is to invent it
Systems like GCD from apple have been around for quite some time.
IBM Visual Age Smalltalk has had something like GCD for at least a decade. Smalltalk has had blocks since 1972.
"When I first heard about Grand Central Dispatch, I was extremely skeptical. The greatest minds in computer science have been working for decades on the problem of how best to extract parallelism from computing workloads. Now here was Apple apparently promising to solve this problem. Ridiculous."
Yes, ridiculous since Apple didn't invent it nor did they point out to you all the pit falls of using it. Remember they are a marketing organization and all their publicaly available materials paint a happy wonderful veneer on it.
The truth is that concurrency control is difficult and concurrency control via something like GCD is just as difficult and avoids none of the real serious problems.
I spent a year and a half debugging an in house application for a very large fortune 500 company that made use of this sort of capability for dispatching parallel work. Much of that time was tracking down problems in block oriented concurrency in a very large code base. Programmers assumed that what they were doing was safe but in the end those assumptions often turned out wrong. Data being changed by multiple threads is a serious problem. Locks are needed to avoid data corruption in many cases. That's just one example. Very nasty situations developed that required some very smart people to figure out and find solutions for, and we had a team of the very best. Eventually we identified the issues and found solutions. It took eight experienced developers over a year focused on fixing bugs that were derived from naive application of block oriented parallelism.
Look, if all you have are very simple cases you're home free with GCD. That is nothing new of course as it is with any concurrency. The challenge comes in when your code base grows, changes and your simple cases are no longer simple. The assumptions change.
So learn to use GCD wisely, it's not a magic silver bullet like it's being made out to be by Apple's marketing department. Oh and remember that C isn't as powerful a IDE as Smalltalk so you'll have your work cut out for you debugging any serious concurrency issues in your applications that use GCD, especially if they are huge applications.
I do welcome blocks to C, finally some expressive power that Smalltalk has had since 1972!
Apple also hosts and continues to support CUPS, the Open Source printing system, a project they bought from the developer that owned it. http://www.cups.org/
If 'self' happens to go away while that thread is running, the changed program will show undefined behaviour. These simple changes almost all require garbage collection to be robust.
This is in OS X only, so no iPhone GCD (at least, not yet...not really necessary until we have multi-core iPhones)
I disagree. As a former Scheme programmer I find it a bit maddening to be so close to have closures as a tool again, yet so far... after all, blocks/closures are a nice way to tell GCD just what you want scheduled, but there are plenty of other great uses for closures in day to day programming where you're not even thinking in parallel.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
And it's not like these "blocks" are actually compiled and linked at run time, it's just a pointer to a static function with a bunch of extra data on it.
Not true, blocks can use any variable from the calling context and it will be incorporated at runtime. These are not just function pointers to static methods.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
What if you're running two applications that both are capable of monopolizing all your cpu time?
How often does this actually occur? And if it does and the processes are threaded, won't a smart OS scheduler just migrate the threads so that each application essentially has a CPU (or four) to itself?
I think the big win is more likely that this makes threading on Mac easier. That said, I really hate the way they did this; putting high-level language features like lexical closures in C is just ugly. C is a glorified assembler, and should stay that way :-)
If moderation could change anything, it would be illegal.
No kernel-level support is required for GCD (the kernel-level facilities are mere optimizations). The library is open-source. LLVM's compiler-rt implementation of the runtime support for blocks is available. Microsoft's involvement is not necessary in this case.
As someone with experience responding to retards, in particular those with inflated self-evaluations of competency.... the above comment is garbage. It's absurd that the above post could be useful to anyone evaluating libdispatch. In particular, it ignores the forgone conclusion that porting this to linux or windows would entail the standard macro preprocessor magic to replace the mach calls with equivalents for other target platforms. Anybody actually evaluating libdispatch has seen ported C. Many times. While it's good to have a library that has already been ported, looking at the above comment, I think you'd have to be an idiot to think that the Mach primitives could not be conditionally compiled with #ifdef. Not to mention C++0x is a C++ standard, and this is a C library.
The "cue the foo posts in 3, 2, 1..." posts will commence with no subsequent foo posts in 3, 2, 1...
Well, the primary difference being that there isn't any chance that your thread will wind up on another type of CPU. With an adroit and vigilant combination of OpenCL, GCD, and Blocks, your thread might wind up on the GPU, the CPU, or your fancy schmancy uber cool Cell auxiliary card.
I'm really going to enjoy watching the "Fully multi-threaded as a standard practice since the 90's [sic]" environment of Windows (TM) crumble under its own weight when that mountain of OS and apps must be re-threaded to make them perform well on quad-core, then octo-core, then hexa-core...
Basically, libdispatch just creates a thread pool for each separate task, then uses some clever magic involving an inter-process semaphore to keep them blocking so that no more than enough threads (ie: the number of CPUs) are running at any given time. Nifty, because it means little change needed to be made to xnu. libdispatch is also, theoretically, portable to other platforms, as long as one can provide a blocks runtime and a compiler capable of handling blocks. I noticed a patch on LLVM's mailing list today providing a Linux port of the blocks runtime, and llvm-gcc and clang both are capable of handling blocks and running on Linux.
haha! weiner you mean Osama
Semi-automatic amateur armchair Australian philosopher; conjecture ready at any moment...
Why the heck did they release it under an Apache 2 license, and not under a regular BSD license?
{{.sig}}
May be I'm missing something here, but what this is saying is that I can take a lambda function and schedule it to run in a thread pool, and that the thread pool will dispatch work such that it makes good use of available CPUs and won't push further such that it would cause excess of context switching. How is that new? The Windows "I/O completion ports" mechanism has always been able to do the nice scheduling part, although with an ugly API. Then the newer Windows thread pool API layered nice thread management and work item dispatch on top. On top of all that, .NET has simply QueueUserWorkItem(delegate, context) that does exactly this, with lambdas and all.
Am I missing something? Or there is really nothing new here other than the fact that Apple released an open source library with an API for this?
Erlang is very cool, but it is not designed to replace C. In fact, it is designed to handle some bits of the higher level concurrency stuff and call out to C "drivers" for level work. Apparently Ericsson's switch code has almost as much C/C++ code as Erlang code. GCD addresses concurrency in problem spaces Erlang is completely inappropriate for, just like Erlang plays in spaces that GCD is not appropriate for. They are different tools for different jobs.
Linux should be release under GPL v3 so it can be compatible with Apache license. This would help Linux move forward, invent, and advance the field of computer science instead of just trying to reinvent GCD, ZFS, DTRACE, ect..
I would add "powerful" to your points (which I guess are limited to the desktop):
* Mac OS X: usable and powerful (great UI + great foundation)
* Windows: just plain convenient, thanks to the size of the install base and people familiar to it
* GNU/Linux: powerful, but not usable
Being said that, I'm actually using the three of them at work:
* GNU/Linux for the people that is responsible for a few and very specific tasks for which Ubuntu has been customized. :)
* Windows for the yet-to-be-converted PC because of in-house systems or 3rd party software that require Windows and is pending of getting an alternative
* Mac OS X for people that know better. Which means the IT department
The best way to predict the future is to invent it
Garbage collection is robust enoughâ" the block gets copied to the (garbage collected) heap when it's passed into dispatch_async() or similar calls automatically, and it uses scanned memory to do so, meaning the collector tracks a refcount on 'self'.
In non-collected situations, it's up to the developer to ensure that this doesn't happen somehowâ" for instance by retaining 'self' before the first dispatch_async() and releasing it inside the last block.
Not to mention that extra 10+ GB of free space :)
SWM seeks new sig for a brief fling