There are a lot of web developers who have mastered Flash. A great many of these will not be learning a new framework unless there are some serious advantages, simply because they already know how to use Flash. As long as Flash is "good enough" developers will continue to use it.
Solution: fire those web developers. They don't produce anything useful anyway, and getting rid of them saves money for things and people that do.
AT&T was allowed to be a monopoly much longer than it should have been, and service stagnated. When it was finally broken up, massive amount of progress happened. Now that the monopoly is being restored (in the form of merger and cartel-like behavior) things are getting worse again.
For most of the history, [relatively diluted] alcohol was the only drug consumed in most of the civilized world, and public drunkenness was severely discouraged by overwhelming majority of society.
Countries with widespread drug use had their development slowed down, what contributed to their eventual colonization by "merely occasionally drunk" members of the Western civilization.
It was "legally sanctioned" after it already gobbled up the whole market with not even a remote chance for competitors to arise. If anything, government planted the seeds of their monopoly destruction by acknowledging it early enough.
I cannot envision a world where there is no demand for recreational drugs.
I can. Stop raising generations of people who believe that the only acceptable state of a human being is happiness, and they will live normal lives doing whatever they will find interesting.
It was only regulated because government acknowledged how dangerous its monopoly position was. "Monopolies exist only because government assists then" is a Conservative talking point that has absolutely no connection to reality. Don't repeat it.
Of course, I'm one of those crackpots that think land ownership should be meaningful, and that if you want to control something, you should have to own it first. Radical, I know.
That worked so well when to be someone's boss you had to own him as a slave.
This is the ONLY real problem with parallel interfaces -- crosstalk is a complete red herring because no one in his right mind will approve a cable, serial or otherwise, that will mess up data on another cable laid in parallel. Parallel interfaces of the past rely on single clock for all lines, so they can fill bus-wide buffer in one cycle.
However with multiple lines, each with its own synchronization and with a larger buffer on the receiving end, clock skew is merely latency -- you have to wait for every bit to complete its cycle before you can push the received data word to the bus. So parallel interfaces are possible, they just require different mechanism of data transfer and synchronization. It is more expensive, but if you really need this speed, it is easily achievable.
ie. for a single connection they will do as much work as they can, then they need to wait for a notification to find out when they can resume
No. It's application's choice if it wants to perform as much work as it can or not every time I/O is available, and how much of this work is done within the same thread as select() -- once triggered, I/O can be delegated to a completely different thread or process. It's not a single connection at a time, either -- select() returns a set of file descriptors available for I/O, and it's kernel's choice how much it will wait before waking up the process and presenting it with multiple file descriptors that became available while the process was sleeping -- priorities and buffers' state can affect this.
Was I too tired when I wrote that and not clear enough, or are you simply using this as an excuse to embrace your inner zealot and mock other developers?
Neither -- you merely assume that all uses of select() are exactly the same as the textbook example of a single-threaded, single-process server where everything happens between calls to select(). While some software is indeed written that way (and there are often good reasons for doing so), there are more complex scenarios that make select() or select()-like syscalls useful for high-performance servers.
select() "linearizes" operations that are linear under any reasonable scenario anyway, and that buys it complete or nearly complete lack of synchronization primitives in the thread or process that performs it. As long as all long-running processing is delegated somewhere else, and I/O or processing triggered after one call to select() is not likely to be complete before the loop calls select() again, the loop is not a bottleneck, and select() sleeps while OS updates its buffers with I/O. As long as loop includes only minimal, low-latency logic (usually just buffers/requests handling), it can act as a layer between OS scheduler (represented by syscalls) and internal application's request handling (represented by worker threads/processes and buffers).
In poorly thought out benchmarks, when workers threads are nonexistent and network I/O is instantaneous, select() loop becomes the bottleneck, however this is not a realistic scenario for a server application.
(obviously, in everything above select() is actually select()/poll()/epoll()/..., as all those interfaces implement the same functionality with different ways of optimizing the access to it -- select() itself is the least scalable of them)
I don't buy that explanation. Without the mechanism you can either have a multithreaded or a select loop based application
You just use select() in place where would use a mutex otherwise. Mutexes are not supposed to be used that way in the first place.
Using pipes for this is rather a hack,
Actually it's one of the most elegant solutions in Unix design. It may look like a hack for a Windows programmer, but so does pretty much everything non-Windows-like.
there's unnecessary data being copied twice,
There is the same amount of data copying as with any other messaging interface. If you don't need a message, you can pass one byte, so you only really spend time performing system calls and context switches.
A futex-fd hybrid could offer the same - select() could return without entering the kernel if the futex was available or wait for it in a system call otherwise.
First of all, the whole point of this is to remove scheduling logic from userspace and place it into the kernel -- then kernel scheduler can optimize everything based on full information about pending I/O and other forms of waiting instead of blindly preempting processes that may or may not be in the best position to be preempted. Of course, this requires sufficiently lightweight scheduler and syscall interface, something that Windows lacks. You seem to be under impression that Linux or other Unix-like systems achieve their high performance by using some thick userspace scheduler that doesn't perform select()/poll()/... syscall on every call to the library select() (or poll(), etc. in any combination depending on the OS) -- this is not true, modern pthreads implementation is very thin, and days of "green threads" libraries are long gone. Java has thick layers on everything anyway but the OS does not.
Second, waiting on futex is a syscall anyway, so if you are trying to wait on anything I/O-related, you can just as well wait on a real file descriptor. Futex is only more efficient if it is used purely for synchronization, and therefore there can't be any file descriptors involved.
Third, if you really care, you can turn futex into a file descriptor and include it into the list of file descriptors you are waiting on.
We rewrote the network IO to work behind the curtain with threads, with the result that the one-socket-per-thread model actually did the I/O completion port thing, with as many as 32k Java threads running in a grand total of about a dozen Windows threads (stacks were small, stacks grew on demand. Certain things were tricky.).
Congratulations, you have achieved what Microsoft is still trying to do with.NET -- conceal messy interfaces behind layers of language/runtime that look like less messy interfaces.
1) got to use the underlying OS's preferred way of doing async IO (on another OS, we might do it differently)
This is exactly what I am talking about. Windows has deeply flawed infrastructure, so it calls for deeply flawed solutions -- software running on Windows has to use the most convoluted interface because this is the only way to get optimal performance.
Fortunately, no one in his right mind would optimize high-performance application for Windows because there are other, better systems, with cleaner interfaces. Not that there is a shortage of crazy people and Microsoft fans, but I really don't care about them and neither should anyone else.
For instance they let you select() both file handles and synchronization primitives, so that you can simultaneously wait for a file operation and a mutex.
In Unix select()/poll()/... works only on file handles because anything worth waiting on IS a file handle. In Windows there is no unified file handle, so Microsoft developers had to add completely foreign entities to this mechanism. In Unixlike OS the purpose of a mutex is data consistency, not I/O or IPC scheduling.
I think there was some work put into Linux wrt. to wrapping mutexes in file descriptors, but it was ultimately abandoned, probably because they couldn't get it right.
It is absolutely unnecessary. Pipes performed this functionality since the very beginning of Unix IPC design, so despite their interface being identical to slower I/O primitives, they are optimized for this use, and on top of that can pass small serialized messages along to the process or thread along with unblocking event.
The homemade DVR is a pleasant - if geekish - project.
And that covers absolutely everyone who will ever bother to copy DRM'ed video. Demise of NTSC and Comcast's eagerness to DRM each and every channel will eventually force everyone who has a homemade box to add such a device, and then HDCP will be about as effective as DVD CSS.
But the father of three kids rents - or more likely buys - the video from Pixar - and that is where the money is.
And he would buy it even if DRM is broken. VHS tapes and CD were sold and rented just fine without DRM, the problem is that content producers want to squeeze more and more profit from the same content.
I just have one question -- is the purpose of all this crap to make a device that only few manufacturers can produce, then make sure that only DRM'ed to Hell version is available on the market?
HDMI DRM is for all practical purpose defeated (YA, RLY) by the use of mass-produced $100-$300 HDCP strippers in homemade DVRs -- now our beloved content providers want hardware companies to build something else, easier to keep out of consumers' hands?
This means async I/O on Linux can use less memory, while on Windows it can give higher throughput.
No. It means that you don't know how pending I/O controls the scheduler. On Linux processes are almost never "notified" by asynchronously arriving signals, they are either processing a request, or sleeping while performing a syscall, so they are awakened by a scheduler when whatever they are waiting for (usually data arrived or buffer is available) happened. Scheduler takes into account the number of pending I/O requests and priorities, so processes are awakened in the optimal order -- I/O priority dictates whether it will be for latency or throughput.
Windows developers followed their typical "Oh!!! It's an event! It looks like a hardware interrupt! I LOVE EVENT HANDLERS! And VMS has the same mechanism! It must be fast!!!" decision-making process instead of trying to actually design a useful data processing mechanism, and predictably ended up with a fine-tuned, convoluted system that works worse than generalized and straightforward one chosen by Unix/BSD/Linux developers.
Aren't you supposed to tell us that GIMP does not support CMYK? I thought, Microsoft doesn't pay those who don't include dissing GIMP and OpenOffice into every comment, or at least proclaim their love for Linux before spewing their anti-open-source propaganda.
Ff you have multiple cores that do nothing otherwise (like all benchmarks happen to act), multithreading will use them and asynchronous nonblocking I/O won't, so maximum transfer rate for static data in memory over low-latency network will be always faster for blocking threads.
In real-life applications if you always have enough work to distribute between cores/processors, your nonblocking I/O process or thread will only depend on the data production and transfer rate, not the raw throughput of the combination of syscalls that it makes. If output buffers are always empty, and input buffers are empty every time a transaction happens, then both data transfer speed is maxed out, and adding more threads that perform I/O simultaneously will only increase overhead. If it is not maxed out, same applies to queued data before/after processing -- that is, if there is processing. So if worker threads/processes do more than copying data, then giving additional cores to them is more useful than throwing them on to be used for I/O.
Re:Should be using Scatter/Gather +IOCP on windows
on
Java IO Faster Than NIO
·
· Score: 1, Troll
When you listen to the technical explanations, the Microsoft way actually IS better - it's just aht it's totally incompatible with evrything else.
No. It's only better if the rest of the system is done according to Microsoft ideology -- that is, never try to look at a bigger picture when designing each piece, then go out of your way to make each piece optimal in face of completely un-cooperating environment. Microsoft developers have no idea how Unix-like systems deal with I/O, scheduler and virtual memory all interacting with each other, so after Microsofties introduce massive amount of complexity everywhere, it seems to them that they have an optimal solution for each and every case they tried to optimize. In reality Unix had system that is more optimized and more straightforward for the developer.
I beg to differ. If you compare them in the areas where they compete -- causing damage to US economy and directing development of US culture, politics and technology toward pointless, counterproductive tinkering -- Microsoft is far ahead of Taliban, al-Qaeda, whole country of Iran, and all organizations that are, or perceived, to be anti-American taken together.
Slashdot Mirror
IE is the #1 browser on the market, so it's kind of a big deal.
The more reasons to write things that don't work in it -- to kill it faster.
It's not like users will abandon something they need if they have to download a free, vastly superior browser to use that.
There are a lot of web developers who have mastered Flash. A great many of these will not be learning a new framework unless there are some serious advantages, simply because they already know how to use Flash. As long as Flash is "good enough" developers will continue to use it.
Solution: fire those web developers. They don't produce anything useful anyway, and getting rid of them saves money for things and people that do.
AT&T was allowed to be a monopoly much longer than it should have been, and service stagnated. When it was finally broken up, massive amount of progress happened. Now that the monopoly is being restored (in the form of merger and cartel-like behavior) things are getting worse again.
For most of the history, [relatively diluted] alcohol was the only drug consumed in most of the civilized world, and public drunkenness was severely discouraged by overwhelming majority of society.
Countries with widespread drug use had their development slowed down, what contributed to their eventual colonization by "merely occasionally drunk" members of the Western civilization.
It was "legally sanctioned" after it already gobbled up the whole market with not even a remote chance for competitors to arise. If anything, government planted the seeds of their monopoly destruction by acknowledging it early enough.
I cannot envision a world where there is no demand for recreational drugs.
I can. Stop raising generations of people who believe that the only acceptable state of a human being is happiness, and they will live normal lives doing whatever they will find interesting.
It was only regulated because government acknowledged how dangerous its monopoly position was. "Monopolies exist only because government assists then" is a Conservative talking point that has absolutely no connection to reality. Don't repeat it.
And your solution to Comcast being crap is... to legislate them into not being crap? That'll totally work.
Why not? (please provide an argument that makes sense to a non-Libertarian).
Of course, I'm one of those crackpots that think land ownership should be meaningful, and that if you want to control something, you should have to own it first. Radical, I know.
That worked so well when to be someone's boss you had to own him as a slave.
Do Anonymous Cowards get mod points? Where are mine?
He meant:
"I came to troll the discussion as an Anonymous Coward and moderate it as a user. And I'm all out of mod points!"
This is the ONLY real problem with parallel interfaces -- crosstalk is a complete red herring because no one in his right mind will approve a cable, serial or otherwise, that will mess up data on another cable laid in parallel. Parallel interfaces of the past rely on single clock for all lines, so they can fill bus-wide buffer in one cycle.
However with multiple lines, each with its own synchronization and with a larger buffer on the receiving end, clock skew is merely latency -- you have to wait for every bit to complete its cycle before you can push the received data word to the bus. So parallel interfaces are possible, they just require different mechanism of data transfer and synchronization. It is more expensive, but if you really need this speed, it is easily achievable.
ie. for a single connection they will do as much work as they can, then they need to wait for a notification to find out when they can resume
No. It's application's choice if it wants to perform as much work as it can or not every time I/O is available, and how much of this work is done within the same thread as select() -- once triggered, I/O can be delegated to a completely different thread or process. It's not a single connection at a time, either -- select() returns a set of file descriptors available for I/O, and it's kernel's choice how much it will wait before waking up the process and presenting it with multiple file descriptors that became available while the process was sleeping -- priorities and buffers' state can affect this.
Was I too tired when I wrote that and not clear enough, or are you simply using this as an excuse to embrace your inner zealot and mock other developers?
Neither -- you merely assume that all uses of select() are exactly the same as the textbook example of a single-threaded, single-process server where everything happens between calls to select(). While some software is indeed written that way (and there are often good reasons for doing so), there are more complex scenarios that make select() or select()-like syscalls useful for high-performance servers.
select() "linearizes" operations that are linear under any reasonable scenario anyway, and that buys it complete or nearly complete lack of synchronization primitives in the thread or process that performs it. As long as all long-running processing is delegated somewhere else, and I/O or processing triggered after one call to select() is not likely to be complete before the loop calls select() again, the loop is not a bottleneck, and select() sleeps while OS updates its buffers with I/O. As long as loop includes only minimal, low-latency logic (usually just buffers/requests handling), it can act as a layer between OS scheduler (represented by syscalls) and internal application's request handling (represented by worker threads/processes and buffers).
In poorly thought out benchmarks, when workers threads are nonexistent and network I/O is instantaneous, select() loop becomes the bottleneck, however this is not a realistic scenario for a server application.
(obviously, in everything above select() is actually select()/poll()/epoll()/..., as all those interfaces implement the same functionality with different ways of optimizing the access to it -- select() itself is the least scalable of them)
I don't buy that explanation. Without the mechanism you can either have a multithreaded or a select loop based application
You just use select() in place where would use a mutex otherwise. Mutexes are not supposed to be used that way in the first place.
Using pipes for this is rather a hack,
Actually it's one of the most elegant solutions in Unix design. It may look like a hack for a Windows programmer, but so does pretty much everything non-Windows-like.
there's unnecessary data being copied twice,
There is the same amount of data copying as with any other messaging interface. If you don't need a message, you can pass one byte, so you only really spend time performing system calls and context switches.
A futex-fd hybrid could offer the same - select() could return without entering the kernel if the futex was available or wait for it in a system call otherwise.
First of all, the whole point of this is to remove scheduling logic from userspace and place it into the kernel -- then kernel scheduler can optimize everything based on full information about pending I/O and other forms of waiting instead of blindly preempting processes that may or may not be in the best position to be preempted. Of course, this requires sufficiently lightweight scheduler and syscall interface, something that Windows lacks. You seem to be under impression that Linux or other Unix-like systems achieve their high performance by using some thick userspace scheduler that doesn't perform select()/poll()/... syscall on every call to the library select() (or poll(), etc. in any combination depending on the OS) -- this is not true, modern pthreads implementation is very thin, and days of "green threads" libraries are long gone. Java has thick layers on everything anyway but the OS does not.
Second, waiting on futex is a syscall anyway, so if you are trying to wait on anything I/O-related, you can just as well wait on a real file descriptor. Futex is only more efficient if it is used purely for synchronization, and therefore there can't be any file descriptors involved.
Third, if you really care, you can turn futex into a file descriptor and include it into the list of file descriptors you are waiting on.
We rewrote the network IO to work behind the curtain with threads, with the result that the one-socket-per-thread model actually did the I/O completion port thing, with as many as 32k Java threads running in a grand total of about a dozen Windows threads (stacks were small, stacks grew on demand. Certain things were tricky.).
Congratulations, you have achieved what Microsoft is still trying to do with .NET -- conceal messy interfaces behind layers of language/runtime that look like less messy interfaces.
1) got to use the underlying OS's preferred way of doing async IO (on another OS, we might do it differently)
This is exactly what I am talking about. Windows has deeply flawed infrastructure, so it calls for deeply flawed solutions -- software running on Windows has to use the most convoluted interface because this is the only way to get optimal performance.
Fortunately, no one in his right mind would optimize high-performance application for Windows because there are other, better systems, with cleaner interfaces. Not that there is a shortage of crazy people and Microsoft fans, but I really don't care about them and neither should anyone else.
For instance they let you select() both file handles and synchronization primitives, so that you can simultaneously wait for a file operation and a mutex.
In Unix select()/poll()/... works only on file handles because anything worth waiting on IS a file handle. In Windows there is no unified file handle, so Microsoft developers had to add completely foreign entities to this mechanism. In Unixlike OS the purpose of a mutex is data consistency, not I/O or IPC scheduling.
I think there was some work put into Linux wrt. to wrapping mutexes in file descriptors, but it was ultimately abandoned, probably because they couldn't get it right.
It is absolutely unnecessary. Pipes performed this functionality since the very beginning of Unix IPC design, so despite their interface being identical to slower I/O primitives, they are optimized for this use, and on top of that can pass small serialized messages along to the process or thread along with unblocking event.
Mass-produced by who and at what risk?
http://www.hdfury.com/
The homemade DVR is a pleasant - if geekish - project.
And that covers absolutely everyone who will ever bother to copy DRM'ed video. Demise of NTSC and Comcast's eagerness to DRM each and every channel will eventually force everyone who has a homemade box to add such a device, and then HDCP will be about as effective as DVD CSS.
But the father of three kids rents - or more likely buys - the video from Pixar - and that is where the money is.
And he would buy it even if DRM is broken. VHS tapes and CD were sold and rented just fine without DRM, the problem is that content producers want to squeeze more and more profit from the same content.
I just have one question -- is the purpose of all this crap to make a device that only few manufacturers can produce, then make sure that only DRM'ed to Hell version is available on the market?
HDMI DRM is for all practical purpose defeated (YA, RLY) by the use of mass-produced $100-$300 HDCP strippers in homemade DVRs -- now our beloved content providers want hardware companies to build something else, easier to keep out of consumers' hands?
This means async I/O on Linux can use less memory, while on Windows it can give higher throughput.
No. It means that you don't know how pending I/O controls the scheduler. On Linux processes are almost never "notified" by asynchronously arriving signals, they are either processing a request, or sleeping while performing a syscall, so they are awakened by a scheduler when whatever they are waiting for (usually data arrived or buffer is available) happened. Scheduler takes into account the number of pending I/O requests and priorities, so processes are awakened in the optimal order -- I/O priority dictates whether it will be for latency or throughput.
Windows developers followed their typical "Oh!!! It's an event! It looks like a hardware interrupt! I LOVE EVENT HANDLERS! And VMS has the same mechanism! It must be fast!!!" decision-making process instead of trying to actually design a useful data processing mechanism, and predictably ended up with a fine-tuned, convoluted system that works worse than generalized and straightforward one chosen by Unix/BSD/Linux developers.
No. And you have no idea what are you talking about.
Aren't you supposed to tell us that GIMP does not support CMYK? I thought, Microsoft doesn't pay those who don't include dissing GIMP and OpenOffice into every comment, or at least proclaim their love for Linux before spewing their anti-open-source propaganda.
Ff you have multiple cores that do nothing otherwise (like all benchmarks happen to act), multithreading will use them and asynchronous nonblocking I/O won't, so maximum transfer rate for static data in memory over low-latency network will be always faster for blocking threads.
In real-life applications if you always have enough work to distribute between cores/processors, your nonblocking I/O process or thread will only depend on the data production and transfer rate, not the raw throughput of the combination of syscalls that it makes. If output buffers are always empty, and input buffers are empty every time a transaction happens, then both data transfer speed is maxed out, and adding more threads that perform I/O simultaneously will only increase overhead. If it is not maxed out, same applies to queued data before/after processing -- that is, if there is processing. So if worker threads/processes do more than copying data, then giving additional cores to them is more useful than throwing them on to be used for I/O.
When you listen to the technical explanations, the Microsoft way actually IS better - it's just aht it's totally incompatible with evrything else.
No. It's only better if the rest of the system is done according to Microsoft ideology -- that is, never try to look at a bigger picture when designing each piece, then go out of your way to make each piece optimal in face of completely un-cooperating environment. Microsoft developers have no idea how Unix-like systems deal with I/O, scheduler and virtual memory all interacting with each other, so after Microsofties introduce massive amount of complexity everywhere, it seems to them that they have an optimal solution for each and every case they tried to optimize. In reality Unix had system that is more optimized and more straightforward for the developer.
I beg to differ. If you compare them in the areas where they compete -- causing damage to US economy and directing development of US culture, politics and technology toward pointless, counterproductive tinkering -- Microsoft is far ahead of Taliban, al-Qaeda, whole country of Iran, and all organizations that are, or perceived, to be anti-American taken together.
Not really. IBM just had a relatively modest computer engineering company in the guts of a massive IT services conglomerate.