Slashdot Mirror
The Father of Multi-Core Chips Talks Shop
pacopico writes "Stanford professor Kunle Olukotun designed the first mainstream multi-core chip, crafting what would become Sun Microsystems's Niagra product. Now, he's heading up Stanford's Pervasive Parallelism Lab where researchers are looking at 100s of core systems that might power robots, 3-D virtual worlds and insanely big server applications. The Register just interviewed Olukotun about this work and the future of multi-core chips. Weird and interesting stuff."
That's a lot of core systems.
Imagine a beowulf cluster of ... oh wait
This is slashdot, you _CAN'T_ post an article that can't be read! timothy, what are you thinking?
Support NYCountryLawyer RIAA vs People
It is time for professor Olukotun and the rest of the multicore architecture design community to realize that multithreading is not part of the future of parallel computing and that the industry must adopt a non-algorithmic model. I am not one to say I told you so but, one day soon (when the parallel programming crisis heats up to unbearable levels), you will get the message loud and clear.
...oh, you know how this one is supposed to go.
let's just hope that more programs in the future are written so that they can scale well to the 100s of cores that new CPUs will have. People were pretty slow to get onto the dual core bandwagon...so here's hoping.
Multi-core chips will be constrained by, among other things, the memory bandwidth going off-chip. Maybe they need larger caches. Maybe they just need to put all the RAM on the chip itself instead of so many other cores. How about 4GB of RAM at 1st level cache speed.
Ultimately, we'll end up with PCs made from SoCs, and direct SATA, USB, Firewire, and DVI interfaces coming out instead of a RAM access bus. By the time they are ready to make 256 core CPUs, software still won't be ready to work well on that. So in the interim, they might as well just do tighter integration (that can also run faster there, too). No more north bridge or south bridge. Just a few capacitors, resistors, and maybe a buffer amp or two, around the big CPU.
About the only thing that won't be practical to put in the CPU for a long time is the power supply. They could even put the disk drive in there (flash SSD).
now we need to go OSS in diesel cars
IAASE and if I recall correctly, Donald Knuth said that the all the advent of multi-core systems showed was that chip developers had run out of ideas and from what I can see happening in the industry today, he was right.
multi-core = multi-kruft
Pardon my ignorance, but we had many supercomputers (more recent one Roadrunner) which use multiple CPUs (and accelerators). Can't we use the programming 'tricks' or 'models' or 'techniques' used there for efficiently using multicores ? I understand multicore has significantly less communication overhead, but overall philosophy of synchronizing, message passing, shared memory etc wouldn't be completely irrelevent ?
For servers, POWER4 was released in 2001. For desktops, the Pentium D came out in mid 2005 and the PowerPC 970MP a few months later. All of these came out before Niagara.
Silly. I cannot believe Donald Knuth would be that dense, there must be more to the conversation.
Every major system in existence today is already a "multiprocessor" system, we just don't think of them that way. The average PC is a parallel system running at least 14 CPUs in parallel. (two or three for every spindle, one or two for keyboard, a few for your broadband modem, a few in your firewall, etc etc etc).
Multicore systems are simply an extension of the existing computational model. Plus, every supercomputer built in the last 20 years has been massively parallel.
Out of ideas? I Don't think so.
I just finished taking a course at MIT on multiprocessor programming. It was taught by the authors of The Art of Multiprocessor Programming, Maurice Herlihy and Nir Shavit. I highly recommend their book, their classes, their expertise. They are now focused on transactional memory, which may make things a bit easier to program in the multiprocessor universe. Of course we can stick with course-grained locking, but as they pointed out early on, Amdahl's Law shows that throwing hardware at a problem may not be successful in upping performance by the amount you expect if the system's scheduler has no hopes of keeping the cores busy due to how you've written your code.
Linked Lists? in order to get to an item, you have to traverse the list until that point. Maybe you could have one thread traverse the list, and dispatch each item to a new thread for processing... but what about counting how many items on the linked list satisfy a condition? maybe round robin assign them to the worker threads, the add up the subtotals...
Seems to me that simple linked list structures may be something to avoid in favor of trees, where you could just send references to branches to threads. Byte streams might have the same issue, for example in decoding variable length characters, you wouldn't want to start in the middle of a character (unicode encoding issues), fixed-width data in an array of a specific length (instead of depending on a terminator) could be easier to break up tasks (such as spell check, send each page to a different thread)
How about pre-emptive multitasking?, if you have hundreds of cores, how often will you need to put a thread on hold, run another thread, then switch back... if you never have to save/restore thread state except when doing unusual tasks like debugging, you could optimize in favor of other operations. You could design threads to stream data like an assembly line, thread A watches for the end/other exceptions, thread B validates the range of values, thread C breaks the data into groups, and passes them along to a collection of other threads. If the chips were optimized to be able to stream data between cores without reaching out to main ram the stream buffers could be rather small, if it's known that the threads involved will never be preempted and each of the tasks has a reasonable upper bound on time. Each core would be looping over a tiny number of instructions, so a large instruction cache wouldn't be needed (per core, but yes for the whole chip package), and since the data is mostly rolling in from one end, and rolling out the other, only the initial input and final output leave the chip.
We already have servers for INSANELY HUGE internet apps, its called a main-frame.
It amazes me to no end, how many people still think its about the CPU. It about throughput, ok? Can we just get that fucking settled already? I don't give a rats ass how many damn cores you have running or if the are running 100 gigahertz, if you are still reading data across a bus, over an ethernet connection, ANYTHING that does not work at CPU speed then it makes little difference, that damn CPU will be sitting there spinning waiting for the data to come popping through so it can do something!
Mainframes use 386 chips for I/O controllers and even those sit there and loaf, talk about a waste of electricity! About .01% of the worlds computers need the kind of power that a CPU with more then say 4 cores provide. Those that do are rather busy doing insanely complex mathematics, but even then I doubt that the CPU(s), even when running at "100%" utilization are actually doing the work that they were programmed to do, rather they are waiting for I/O to a database or RAM and fetching data.
Until someone figures out how to move data in a far far more efficient manner then we currently understand, these mega-core CPU's, while nice to think about, are simply a waste of time and silicon with the possible exception of research.
Hey KID! Yeah you, get the fuck off my lawn!
I took a class from him at Stanford University. He was the worst lecturer I ever had. He clearly cared a lot more about his research than us silly undergraduates. Is this a problem with all well-known professors? And if our teachers don't put any effort into teaching, how are we, the students of the next generation, supposed to contribute to the field?
totally and undeniably allrighty then ... we don't need something new, we just need MORE of the same old stuff ... all hail von neumann !
Free speech was meant to be free for all... how can anyone grow up in a nanny state ?