multithreading vs singlethreading
on
Whither OpenAL?
·
· Score: 1
Yes, the same amount of work has to be done, but (essentially) the OS doesn't have the same insight into the work as you do, so you can do a better job. You can do all the vector math for a 3D scene, and *then* dirty the cache doing some sound processing, but you'll dirty the cache only once, not multiple times when the OS will go back and forth between the two.
The last sentence of your first paragraph is correct, the cache would be dirtied anyway (unless you can write a program which fits all (including data and appropriate kernel parts!) in cache. Then you have no problem.:-)
You are also right in your second paragraph. But we were talking about pre-emptive multitasking here, and the pre-emptive part means just the opposite of what you are saying.
GNU Pth is a non-preemptive multithreading package that gives you this kind of control with a familiar POSIX Threads API.
Personally, I prefer event/message-based architectures, which can scale to multiple threads or processes if needed (like if you want to use multiple CPUs or if you have totally blocking operations like libresolv-based gethostbyname). But GNU Pth is nice too, a shame it's not used more...
Why is rather simple: most game players have a single processor.
When you have vector math routines that are optimized to take as much advantage of the cache as possible and you have a program that spends 70% of its cycles in those math routines, what do you think happen when you have these vector operations going in a tight loop and the scheduler preempts it to run the sound thread a bit? Dirties the cache like hell, performance goes down the drain, everything sucks.
For those who have two processors, inter-processor communication usually kill multi-threaded performance if it isn't VERY carefully done (i.e., it's practically never done correctly). Cache coherency is a killer for most multi-threaded applications on SMP systems.
I don't know if this still works, but a while ago (a year or so), I was playing with the CVS version of OpenAL, and back then it was possible to avoid multi-threading, so I would credit this to the OpenAL developers (unless they broke that feature at some point).
Avoiding preemptive multitasking in high performance single-processor programming is a good idea, period.
Both Quake III and Quadra use single-threaded, non-preemptive sound output with great success, so why are so many Linux/others game developers so stubborn on the idea of putting the sound in its own thread?
Personally, I blame the original Doom port, which everyone duplicated, even though at the time it had big problems of latency compared to the single-threaded DOS version because of latencies in the scheduling of the sound process.
No, threads don't avoid this problem. Thank you for trying, but sorry.
Okay, I'm a NEC/HNSX Supercomputers employee, on the verge of becoming a Cray employee (because of the agreement they signed), but I'm not speaking for anyone else but me here, of course.:-)
I don't know why people bother with such a news. Sun's gonna provide the I/O processor for a not-so-high-end supercomputer. And?
A few weeks ago, there was a real bombshell: Cray would drop the anti-dumping legal action, re-opening the US market to japanese supercomputers. Cray will even become the sole reseller of the NEC SX Series in North America!
If you go take a look at www.cray.com, you'll see that this agreement with Sun occupies a single line in their news listing, while the NEC agreement is a big framed box that occupies about half of my screen here.
For some time now, american supercomputer customers were petitioning to get japanese machines, because it been a long time the american machines had been up to any good. Instead, we hear about the SV2, which will barely surpass the few years old SX-5 processing power, with less memory throughput than the SX-5.
I won't deal with the "no need for big clunky vector supercomputers, we have clusters". I believe a whole lot into clusters, but they're freakin' hard to program, and some things just won't be as fast (hey, the SX-5 CPU has a 256 bytes wide memory path! that's not bits, that's bytes! what can you do with your puny gigabit ethernet cluster interconnections?).
Look at these bandwidth benchmark scores. The closest thing to a cluster, the Origin machines, are literally crushed to bits by the SX-5. And they're doing twice as good than the SV1.
As for using old big iron machines for stuff like fridges and so on, there was a cool thing at one of our customer site, at the University of Stuttgart: a Cray coffee table.:-)
Nothing beats talking about supercomputer technology while drinking some orange juice on top of a Cray machine. NOTHING.
There is a piece on trade secrets on opensource.org, explaining that for some products, closed-source beginnings (with subsequent open sourcing) can make sense.
Whether this is applicable in this case, I won't comment.
As you said, we're implementing C++ only at first, but we have an IDL compiler on the roadmap that will enable someone to come and hack this.
Ideally, the IDL compiler will generate header files that would embed the type information of the interface, but it could also temporarily be used to generate stubs à la SWIG.
But the goal is to have enough reflection to make it possible to handle new interfaces at runtime in scripted languages!
Yeah, ok, maybe asking for ideas on Slashdot isn't so hot, but hey, there's enough people reading it that there ought to be something good to get (statistically speaking)!;-)
--
Re:X11 and top Re:No overhead savings
on
GTK+ without X!
·
· Score: 1
Let's also not forget about shared memory mappings.
Note that he didn't mention "binary" in his post, he just said that text files all over the place aren't that cool. Me, I don't see anything all over the place, everything is in/etc like it should.:-)
A registry-like system build with libPropList could be cool, but it could get big enough to become an annoyance if it was in a single file. It's true some packages could use some more consistency.
I personally think that IMAP access is highly desireable (especially over SSL). The ability to read my mail with all of my folders and stuff identical and syncronized on all my machines (home, work, laptop) is extremely cool.
A web interface alternative is nice too, but be sure it's over SSL.
SGI Origins and Compaq AlphaServer GSes aren't even true shared memory boxes (they are distributed shared memory, NUMA).
Heck, if you want shared memory supercomputing power, get the real stuff, either a NEC, a Fujitsu or a Hitachi (Crays are now way behind in performance for their shared memory offerings, they only have NUMA T3s).
For many (real) applications, a NEC SX-5 with 16 CPUs will kick an AlphaServer GS 32 CPU box right out of the water. Don't look at the Top 500 (they're near the top anyway), their benchmark is so parallel that a distributed.net-style effort could get the top spot...
Forgot the disclaimer (I work for NEC), but I guess that was pretty obvious from my signature...:-)
About some talks about memory bandwidth: the data bus is 256 BYTES wide. I repeat: this is not "bits", but "BYTES"!
There is also a crossbar interconnect device that can connect a number of SX-5s together at 8 or 16 gigabytes per second (I think it's 16, but I'm not sure).
Have you considered a NEC SX-5 or one of their variant? Even smaller models will shove an Origin right into the ground, and many commercial applications like ABAQUS are available for these.
NEC is involved in some Linux strategy, even at the level of its supercomputers. For example, the SX-5 can use an Intel-based Linux machine as a support system.
Yeah, but the IBM LVM can resize filesystems on-the-fly, WHILE THEY'RE MOUNTED!
This feature alone has saved my butt on more than one occasion. There is something quite thrilling to transferring a huge amount of data from another machine, noticing that you won't have enough space, to run to the machine room to slip in an additionnal drive, add it to the logical volume, then grow your filesystem, all of this while the transfer takes place!
This, and the very nicely done SMIT configuration program (like Linuxconf, but done much better (or more complicated, depending on who your talk to)), are the only two things I liked about IBM's AIX. I loved the RS/6000 machines, but AIX is awful. Yet, those two features are almost enough to redeem the whole thing! (adding bash, XEmacs and other GNU utilities don't hurt either!)
1) Performance due to missing and wasted functionality. The "scroll" problem is because unlike Windows & Mac, X does not support a nice scrolling functions like "copy block up and only redraw the bottom line" like Windows does. And thus your hardware accelerated equivalent goes unused. X doesn't have the complex "update/validation regions" This shows it is missing key functions. It's just dated.
What? What's XCopyArea there for?
"update/validation regions"? What the heck is that? Personally, I only redraw what the Expose events tell me I need to redraw, if that's what you mean.
Slashdot Mirror
Yes, the same amount of work has to be done, but (essentially) the OS doesn't have the same insight into the work as you do, so you can do a better job. You can do all the vector math for a 3D scene, and *then* dirty the cache doing some sound processing, but you'll dirty the cache only once, not multiple times when the OS will go back and forth between the two.
The last sentence of your first paragraph is correct, the cache would be dirtied anyway (unless you can write a program which fits all (including data and appropriate kernel parts!) in cache. Then you have no problem. :-)
You are also right in your second paragraph. But we were talking about pre-emptive multitasking here, and the pre-emptive part means just the opposite of what you are saying.
GNU Pth is a non-preemptive multithreading package that gives you this kind of control with a familiar POSIX Threads API.
Personally, I prefer event/message-based architectures, which can scale to multiple threads or processes if needed (like if you want to use multiple CPUs or if you have totally blocking operations like libresolv-based gethostbyname). But GNU Pth is nice too, a shame it's not used more...
Why is rather simple: most game players have a single processor.
When you have vector math routines that are optimized to take as much advantage of the cache as possible and you have a program that spends 70% of its cycles in those math routines, what do you think happen when you have these vector operations going in a tight loop and the scheduler preempts it to run the sound thread a bit? Dirties the cache like hell, performance goes down the drain, everything sucks.
For those who have two processors, inter-processor communication usually kill multi-threaded performance if it isn't VERY carefully done (i.e., it's practically never done correctly). Cache coherency is a killer for most multi-threaded applications on SMP systems.
I don't know if this still works, but a while ago (a year or so), I was playing with the CVS version of OpenAL, and back then it was possible to avoid multi-threading, so I would credit this to the OpenAL developers (unless they broke that feature at some point).
Avoiding preemptive multitasking in high performance single-processor programming is a good idea, period.
Both Quake III and Quadra use single-threaded, non-preemptive sound output with great success, so why are so many Linux/others game developers so stubborn on the idea of putting the sound in its own thread?
Personally, I blame the original Doom port, which everyone duplicated, even though at the time it had big problems of latency compared to the single-threaded DOS version because of latencies in the scheduling of the sound process.
No, threads don't avoid this problem. Thank you for trying, but sorry.
For XPLC, I do this "internals should look good" thing.
There are some functions and code fragments that I re-wrote a number of times because they didn't have the right aesthetics to my eye.
Isn't that the alternate ending of that Antitrust movie?
--
Okay, I'm a NEC/HNSX Supercomputers employee, on the verge of becoming a Cray employee (because of the agreement they signed), but I'm not speaking for anyone else but me here, of course. :-)
I don't know why people bother with such a news. Sun's gonna provide the I/O processor for a not-so-high-end supercomputer. And?
A few weeks ago, there was a real bombshell: Cray would drop the anti-dumping legal action, re-opening the US market to japanese supercomputers. Cray will even become the sole reseller of the NEC SX Series in North America!
If you go take a look at www.cray.com, you'll see that this agreement with Sun occupies a single line in their news listing, while the NEC agreement is a big framed box that occupies about half of my screen here.
For some time now, american supercomputer customers were petitioning to get japanese machines, because it been a long time the american machines had been up to any good. Instead, we hear about the SV2, which will barely surpass the few years old SX-5 processing power, with less memory throughput than the SX-5.
I won't deal with the "no need for big clunky vector supercomputers, we have clusters". I believe a whole lot into clusters, but they're freakin' hard to program, and some things just won't be as fast (hey, the SX-5 CPU has a 256 bytes wide memory path! that's not bits, that's bytes! what can you do with your puny gigabit ethernet cluster interconnections?).
Look at these bandwidth benchmark scores. The closest thing to a cluster, the Origin machines, are literally crushed to bits by the SX-5. And they're doing twice as good than the SV1.
As for using old big iron machines for stuff like fridges and so on, there was a cool thing at one of our customer site, at the University of Stuttgart: a Cray coffee table. :-)
Nothing beats talking about supercomputer technology while drinking some orange juice on top of a Cray machine. NOTHING.
--
I pay through the nose (almost 4$ CDN!) for mochaccino around here. But I am a snob.
--
There is a big advantage to Oracle, even in the face of others big commercial databases: writers do not block readers. Or is it the reverse? :-)
Anyway, nobody blocks nobody, which allows for a large number of long-running queries to run in parallel.
Interestingly enough, PostgreSQL has that feature, while no other commercial database has it.
--
Whether this is applicable in this case, I won't comment.
--
As you said, we're implementing C++ only at first, but we have an IDL compiler on the roadmap that will enable someone to come and hack this.
Ideally, the IDL compiler will generate header files that would embed the type information of the interface, but it could also temporarily be used to generate stubs à la SWIG.
But the goal is to have enough reflection to make it possible to handle new interfaces at runtime in scripted languages!
Yeah, ok, maybe asking for ideas on Slashdot isn't so hot, but hey, there's enough people reading it that there ought to be something good to get (statistically speaking)! ;-)
--
Let's also not forget about shared memory mappings.
--
A registry-like system build with libPropList could be cool, but it could get big enough to become an annoyance if it was in a single file. It's true some packages could use some more consistency.
--
My company web site, http://ludusdesign.com/, is on Media3...
--
HPCwire already has that, they call that something like "industry news" or somesuch...
--
Pierre Phaneuf
I personally think that IMAP access is highly desireable (especially over SSL). The ability to read my mail with all of my folders and stuff identical and syncronized on all my machines (home, work, laptop) is extremely cool.
A web interface alternative is nice too, but be sure it's over SSL.
--
Pierre Phaneuf
Heck, if you want shared memory supercomputing power, get the real stuff, either a NEC, a Fujitsu or a Hitachi (Crays are now way behind in performance for their shared memory offerings, they only have NUMA T3s).
For many (real) applications, a NEC SX-5 with 16 CPUs will kick an AlphaServer GS 32 CPU box right out of the water. Don't look at the Top 500 (they're near the top anyway), their benchmark is so parallel that a distributed.net-style effort could get the top spot...
http://www.hstc.necsyl.com/
--
Pierre Phaneuf
About some talks about memory bandwidth: the data bus is 256 BYTES wide. I repeat: this is not "bits", but "BYTES"!
There is also a crossbar interconnect device that can connect a number of SX-5s together at 8 or 16 gigabytes per second (I think it's 16, but I'm not sure).
E-mail me if you want a sales contact.
--
Pierre Phaneuf
NEC is involved in some Linux strategy, even at the level of its supercomputers. For example, the SX-5 can use an Intel-based Linux machine as a support system.
--
Pierre Phaneuf
Check this out:
http://www.opensource.org/secrets.html--
Pierre Phaneuf
Yeah, but the IBM LVM can resize filesystems on-the-fly, WHILE THEY'RE MOUNTED!
This feature alone has saved my butt on more than one occasion. There is something quite thrilling to transferring a huge amount of data from another machine, noticing that you won't have enough space, to run to the machine room to slip in an additionnal drive, add it to the logical volume, then grow your filesystem, all of this while the transfer takes place!
This, and the very nicely done SMIT configuration program (like Linuxconf, but done much better (or more complicated, depending on who your talk to)), are the only two things I liked about IBM's AIX. I loved the RS/6000 machines, but AIX is awful. Yet, those two features are almost enough to redeem the whole thing! (adding bash, XEmacs and other GNU utilities don't hurt either!)
Pierre Phaneuf
What? What's XCopyArea there for?
"update/validation regions"? What the heck is that? Personally, I only redraw what the Expose events tell me I need to redraw, if that's what you mean.
Pierre Phaneuf
The s/// you had should be made a s///g and should be inserted into most industry journalist with a sledgehammer.
Thank you.