What I'm interested in is millisecond accurate *time of day*, or at least one that's properly synchronized across a group of machines sitting in the same LAN.
Windows offers ~5 different timers, with resolution ranging from ~10^-8 to 10^-1, but they are useless when comparing events on _different_ machines along the network, as each of these timers has, on each machine, a totally different origin, and a slightly different rate.
It's possible to compensate for these, and ntpd/xntpd on Linux and FreeBSD does that perfectly well. Windows does not, because of some kernel implementation issue.
I'm not expecting Windows or Linux to be a real-time system. I just expect time-keeping to be reasonably dependable, which on Linux it is and on Windows it isn't.
Wrote a dumbed-down client-server utility, in which one machine tells the other (using a UDP probe, or a previously established TCP connection), "tell me your time up to the millisecond".
In a reasonably loaded LAN, you get responses back after ~0.1ms, which means there is no skew due to latency. When you do that between two Linux machines, you get better than 0.1ms sync. When you do that between Windows machines, you get up to 30ms, no matter how long you allow those two machines to try to synchronize.
My investigations indicate that this is an implementation flaw in the Windows NT kernel which has existed since day one, and as far as I know is not expected to be fixed in Vista.
Windows doesn't do anything useful. That was the point of my rant. I'll probably buy the book sometime soon, but -- as I indicated -- the problem lies with the Windows kernel rather than with NTP. (And yes, I have researched this extensively).
NTP does have a "next check" concept, or at least every NTP client I've used does (usually some form of exponential backoff when things look ok).
The "synchronizing on a schedule" was the only solution that was somewhat close to being reasonable. No "real" NTP client I used seems to work. I didn't try this specific build, but I did try one compiled from the same sources, and it had lousy performance.
Thanks for the link, though.
The solution my "enterprise" finally used, btw, is to write a high precision clock synchronizer that is linked to any of our apps that need it; Thankfully, we _are_ in a position to shove the accurate clock everywhere we need. Our own code manages to keep ~0.5ms of accuracy within our network. It's not truly a general purpose solution though.
For various reasons, I'm trying to synchronize a clock to millisecond accuracy among ~50 Microsoft Windows stations, and it's nearly impossible -- No NTP client for Windows (including AboutTime, 2000's internal client, XP's internal client, and a port of the standard NTP client) appears to be able to keep time reasonably synchronized.
Part of the problem is the Windows Kernel counting time in 10ms or 15ms (depending on whether or not you use an SMP kernel), which automatically says you can't get more than ~30ms precision. But it seems so much worse, with every machine drifting up to ~1 second daily unless they are syncrhonized very frequently -- I get somewhat reasonable results synchronizing them every minute.
On Linux and FreeBSD, this is so trivial it's not even funny; My linux machines manage to keep synchronization to ~0.5 ms over months. Please wake me up when Windows is ready for the enterprise. And, yes, the "enterprise" I work in does need millisecond precision time-of-day synchronization among machine, as does any place that seriously tries to correlate network events (especially those related to security) collected at different points in the network.
Those of us who care to measure for themselves rather than buy Intel's propaganda, have noticed this long ago. I bet the people quoted in the article noticed it long ago as well, but it has only recently become "politically correct" to share that knowledge.
Most likely, your bank is running its mortgage application on a seven year old database. Many of these apps are actually running on much older hardware/software combinations. I don't think mortgages have changed that much in the last 10 years or so.
Newer != Better, New Features != Required Features.
I recommend getting a copy of Horowitz & Hill's "Art of Electronics" book. It's a good read - also get the "laboratory book" (whatever it's called), which is really a fun read and has some experiments listed.
Art of Electronics at Amazon
Matlab style; works everywhere, outputs any format, uses any graphics backend. Look at the beautiful screenshots and how easy it is to generate them. Free (gratis and libre). Python License.
Without copyright law, people would still only release executables. And everything would become gratis, but there is no reason anything would be libre.
Maximum substring sum is written in K tersely as "|/0(0|+)\" and verbosely as "max over 0 (0 max plus) scan". More readable?
It's notation for semantics, and the semantics are what you're missing. K is not readable with just knowledge of C/Java/Lisp; It has a learning curve. But it is well worth it - can you do the max substring sum _efficiently_ in 9 chars?
I know K is never going to be popular. Neither will Lisp or for that matter, any other language that has a nontrivial learning curve.
I'm really sorry that you have to smell the Java. Really. I hope you'll be able to enjoy programming once more some day.
Readability is relative to knowledge. How much is "1/2"? In C it is 0; In Pascal it is 0.5. What happens in "if (peace or destroy(world))"? In Pascal, it is undefined. In C (assuming "#define or ||"), it will destroy the world only if there is no peace.
is "object.frobnicateOtherObject(other)" more readable than "+/v" ?
When it is comparable to this example (yes, the text above is both the logic and the GUI for the game below), or this calculator or spreadsheet implementations I'll be impressed.
It may look obfuscated, but it isn't to one versed in K programming. And if you aren't versed, anything (except perhaps Python) looks obfuscated.
Reference counting can interact nicely with multithreading on modern (post `96) hardware - most modern CPUs have this nice "compare-and-swap" atomic operation, which can be used to manage refcounts without any form of locking. Yes, it is a little less efficient and a little more intricate, but it's doable; In Windows, for example, it's called "InterlockedIncrement()" and "InterlockedDecrement()".
Also, in many environments you DON'T modify the reference count every time you copy a pointer; there's a concept called "borrowed references" which is used in Python, COM, and many other ref count schemes to avoid some useless refcounts.
Python (pre 2.0) used to do only refcount, and did it much better than Java (using GC) in all respects except thread friendliness. Modern python (2.0 and beyond) does both -- but it's extremely rare for the gc to be needed at all.
Data Warehousing for Cavemen by Phil Greenspun (of ArsDigita fame) -- some background, and implementation using SQL. A dimensional modelling manifesto.
Wikipedia has good coverage.
MDX is the query language (look it up in MSDN). Personally, I don't like the syntax but who cares.
And then, when you're convinced OLAP is complex, have a look at Stevan Apter's Drilldown example. The source code is here, all of one printed page, including the GUI and generation of random data.
You might want to encrypt it. Windows has EFS built in, which some people recommend. I'm a happy customer of Jetico BestCrypt, which i highly recommend.
Encryption might be helpful against a physical break-in or computer theft. It might also aid against _some_ successful hacking, provided that you do not keep an encrypted volume mounted (thus accessible) when not necessary. This won't help if you've been rooted and keylogged, though.
The hard work is right-to-left support, for which there is, potentially, a huge market - Arabic speakers. If you have Arabic working, Hebrew is much simpler (replace the font, disable context-sensitive-glyphs, and you're done).
Furthermore, Israel exports a lot of "bleeding edge" technology to the rest of the world. If Israelis don't use Microsoft, it will have noticable effects, especially with early adopters which are often also trend setters.
The problem is with the high level languages, not with the intermediate ones. If the source is C, efficiently vectorizing it is a problem that will not be solved by a new intermediate representation. And if it's a new high level language - well, we'll need to design the IL with respect to that HLL.
APL, J and K are three languages that could be considered IL for high performance computing. But they're high enough to write code directly in them; And so far, it seems, implementations do a wonderful job without compiling down to machine language.
Use disks.
See : Mike Rubel's rsync backup system.
You can't beat IDE disks on price/gb - Tapes are MORE expensive; They're fast, available on-line, and you'll probably be able to mount them on any machine in the next 10 years (which is not true for many tape drives).
(It's slashdot, afterall - I wouldn't want to be thrown out for actually _reading_ the article).
All of the participants come from a business administration perspective. It's not really a wonder they think moving elements around in a gantt chart is "higher level work" than writing lines of code.
It would be a much easier world for the Business Administration guys if software development actually _was_ a low skill job. If it can be specified well enough to be automated by human drones, it will be automated by machines - and then we'll need a higher skilled developer to supervise these machines.
They should discuss outsourcing management - it's the next logical step.
I still play (emulated) C64 games on my PC - they're lots of fun, and are very addictive, though somewhat lacking in the graphics department.
I find it strange that the Palm, and generally all modern PDAs and cell phones have 20 times as much power as the old C64 in every measurement, yet most of the games suck.
Most home users copy, and even that is starting to change lately, with awareness promoted by the BSA, mostly.
Most businesses buy. And when you buy, it costs here the same as it does in the US (even though salaries are lower).
MS Office support for hebrew is bearable (not perfect, but it gets the job done). Compatibility of office versions with respect to Hebrew is horrible - numbers and punctuation are reordered, randomly it seems, when you save in one version and open in another.
I must say it warms my heart - but I'm a bit pessimistic, you see... Israel has got some of the best politicians money can buy; And, judging from the enthusiastic appearance of two of our ministers in Microsoft's latest "microsoft in the government expo" in Italy, I think Microsoft Israel is well aware of the commodity status of Israeli politicians.
Before Microsoft Windows was considered stable enough to run servers, the norm was to pack as much as possible onto each server. On SPARC/Solaris servers, that's still the case, and is the reason why 64-way servers are actually reasonable - every processor can access the file system, and they share the load. To get comparable (in the CPU power sense) from independent computers, you'd need much more CPUs, perhaps 4 times as much, because an idle CPU in one box can't take over an overloaded CPU in another.
But Microsoft has made it standard practice to set up a box for every service (And making you pay lots of microsoft tax in the process). And at the NT4 days, that was a good practice, because the O/S wasn't stable enough to be trusted with more than one task at a time - Exchange, SQL-Server and other services could all bring the entire O/S down when a malformed user request came in.
Nowadays, the Windows 2000 and 2003 systems are stable enough to run more than one service at a time. They don't use their resources efficiently, but they are capable of the level of separation needed, and are generally stable enough. Yet, many applications are built with the older mindset, and make demands that are not reasonable for a machine that hosts many services. For example, they might require a specific service pack, a specific version of another tool (Database, Exchange, whatever). A batch text processing system might require a specific Word version to be installed on the server, and a different system might require another.
If two such demands are incompatible, you're out of luck because Microsoft doesn't let you install two versions of the software at the same time. Most Unix packages can have multiple version co-existing at the same time.
So, to sum up - on modern operating systems, there's no real inherent technical argument against packing all the services on one machine. However, due to historical reasons (tracing back to days when this was unpractical), you often have no choice but dedicate different boxes to different services when running Microsoft O/S and its tools. There is rarely such a problem on Unix systems and their like.
Slashdot Mirror
What I'm interested in is millisecond accurate *time of day*, or at least one that's properly synchronized across a group of machines sitting in the same LAN.
Windows offers ~5 different timers, with resolution ranging from ~10^-8 to 10^-1, but they are useless when comparing events on _different_ machines along the network, as each of these timers has, on each machine, a totally different origin, and a slightly different rate.
It's possible to compensate for these, and ntpd/xntpd on Linux and FreeBSD does that perfectly well. Windows does not, because of some kernel implementation issue.
I'm not expecting Windows or Linux to be a real-time system. I just expect time-keeping to be reasonably dependable, which on Linux it is and on Windows it isn't.
Wrote a dumbed-down client-server utility, in which one machine tells the other (using a UDP probe, or a previously established TCP connection), "tell me your time up to the millisecond".
In a reasonably loaded LAN, you get responses back after ~0.1ms, which means there is no skew due to latency. When you do that between two Linux machines, you get better than 0.1ms sync. When you do that between Windows machines, you get up to 30ms, no matter how long you allow those two machines to try to synchronize.
My investigations indicate that this is an implementation flaw in the Windows NT kernel which has existed since day one, and as far as I know is not expected to be fixed in Vista.
Real NTP does a whole lot more.
Windows doesn't do anything useful. That was the point of my rant. I'll probably buy the book sometime soon, but -- as I indicated -- the problem lies with the Windows kernel rather than with NTP. (And yes, I have researched this extensively).
NTP does have a "next check" concept, or at least every NTP client I've used does (usually some form of exponential backoff when things look ok).
The "synchronizing on a schedule" was the only solution that was somewhat close to being reasonable. No "real" NTP client I used seems to work. I didn't try this specific build, but I did try one compiled from the same sources, and it had lousy performance.
Thanks for the link, though.
The solution my "enterprise" finally used, btw, is to write a high precision clock synchronizer that is linked to any of our apps that need it; Thankfully, we _are_ in a position to shove the accurate clock everywhere we need. Our own code manages to keep ~0.5ms of accuracy within our network. It's not truly a general purpose solution though.
For various reasons, I'm trying to synchronize a clock to millisecond accuracy among ~50 Microsoft Windows stations, and it's nearly impossible -- No NTP client for Windows (including AboutTime, 2000's internal client, XP's internal client, and a port of the standard NTP client) appears to be able to keep time reasonably synchronized.
Part of the problem is the Windows Kernel counting time in 10ms or 15ms (depending on whether or not you use an SMP kernel), which automatically says you can't get more than ~30ms precision. But it seems so much worse, with every machine drifting up to ~1 second daily unless they are syncrhonized very frequently -- I get somewhat reasonable results synchronizing them every minute.
On Linux and FreeBSD, this is so trivial it's not even funny; My linux machines manage to keep synchronization to ~0.5 ms over months. Please wake me up when Windows is ready for the enterprise. And, yes, the "enterprise" I work in does need millisecond precision time-of-day synchronization among machine, as does any place that seriously tries to correlate network events (especially those related to security) collected at different points in the network.
Those of us who care to measure for themselves rather than buy Intel's propaganda, have noticed this long ago. I bet the people quoted in the article noticed it long ago as well, but it has only recently become "politically correct" to share that knowledge.
Ahhmm.
Most likely, your bank is running its mortgage application on a seven year old database. Many of these apps are actually running on much older hardware/software combinations. I don't think mortgages have changed that much in the last 10 years or so.
Newer != Better, New Features != Required Features.
I recommend getting a copy of Horowitz & Hill's "Art of Electronics" book. It's a good read - also get the "laboratory book" (whatever it's called), which is really a fun read and has some experiments listed. Art of Electronics at Amazon
Matlab style; works everywhere, outputs any format, uses any graphics backend. Look at the beautiful screenshots and how easy it is to generate them. Free (gratis and libre). Python License.
matplotlib site
That's entirely wrong.
Without copyright law, people would still only release executables. And everything would become gratis, but there is no reason anything would be libre.
Maximum substring sum is written in K tersely as "|/0(0|+)\" and verbosely as "max over 0 (0 max plus) scan". More readable?
It's notation for semantics, and the semantics are what you're missing. K is not readable with just knowledge of C/Java/Lisp; It has a learning curve. But it is well worth it - can you do the max substring sum _efficiently_ in 9 chars?
I know K is never going to be popular. Neither will Lisp or for that matter, any other language that has a nontrivial learning curve.
I'm really sorry that you have to smell the Java. Really. I hope you'll be able to enjoy programming once more some day.
Readability is relative to knowledge. How much is "1/2"? In C it is 0; In Pascal it is 0.5. What happens in "if (peace or destroy(world))"? In Pascal, it is undefined. In C (assuming "#define or ||"), it will destroy the world only if there is no peace.
is "object.frobnicateOtherObject(other)" more readable than "+/v" ?
Will probably converge to K or APL eventually.
When it is comparable to this example (yes, the text above is both the logic and the GUI for the game below), or this calculator or spreadsheet implementations I'll be impressed.
It may look obfuscated, but it isn't to one versed in K programming. And if you aren't versed, anything (except perhaps Python) looks obfuscated.
Reference counting can interact nicely with multithreading on modern (post `96) hardware - most modern CPUs have this nice "compare-and-swap" atomic operation, which can be used to manage refcounts without any form of locking. Yes, it is a little less efficient and a little more intricate, but it's doable; In Windows, for example, it's called "InterlockedIncrement()" and "InterlockedDecrement()".
Also, in many environments you DON'T modify the reference count every time you copy a pointer; there's a concept called "borrowed references" which is used in Python, COM, and many other ref count schemes to avoid some useless refcounts.
Python (pre 2.0) used to do only refcount, and did it much better than Java (using GC) in all respects except thread friendliness. Modern python (2.0 and beyond) does both -- but it's extremely rare for the gc to be needed at all.
Recommended reading:
Data Warehousing for Cavemen by Phil Greenspun (of ArsDigita fame) -- some background, and implementation using SQL.
A dimensional modelling manifesto.
Wikipedia has good coverage.
MDX is the query language (look it up in MSDN). Personally, I don't like the syntax but who cares.
And then, when you're convinced OLAP is complex, have a look at Stevan Apter's Drilldown example. The source code is here, all of one printed page, including the GUI and generation of random data.
You might want to encrypt it. Windows has EFS built in, which some people recommend. I'm a happy customer of Jetico BestCrypt, which i highly recommend.
Encryption might be helpful against a physical break-in or computer theft. It might also aid against _some_ successful hacking, provided that you do not keep an encrypted volume mounted (thus accessible) when not necessary. This won't help if you've been rooted and keylogged, though.
Yes, it is cool. Check out Dan Bernstein work on package management at http://cr.yp.to/slashpackage.html
The hard work is right-to-left support, for which there is, potentially, a huge market - Arabic speakers. If you have Arabic working, Hebrew is much simpler (replace the font, disable context-sensitive-glyphs, and you're done).
Furthermore, Israel exports a lot of "bleeding edge" technology to the rest of the world. If Israelis don't use Microsoft, it will have noticable effects, especially with early adopters which are often also trend setters.
The problem is with the high level languages, not with the intermediate ones. If the source is C, efficiently vectorizing it is a problem that will not be solved by a new intermediate representation. And if it's a new high level language - well, we'll need to design the IL with respect to that HLL.
APL, J and K are three languages that could be considered IL for high performance computing. But they're high enough to write code directly in them; And so far, it seems, implementations do a wonderful job without compiling down to machine language.
Use disks. See : Mike Rubel's rsync backup system. You can't beat IDE disks on price/gb - Tapes are MORE expensive; They're fast, available on-line, and you'll probably be able to mount them on any machine in the next 10 years (which is not true for many tape drives).
(It's slashdot, afterall - I wouldn't want to be thrown out for actually _reading_ the article).
All of the participants come from a business administration perspective. It's not really a wonder they think moving elements around in a gantt chart is "higher level work" than writing lines of code.
It would be a much easier world for the Business Administration guys if software development actually _was_ a low skill job. If it can be specified well enough to be automated by human drones, it will be automated by machines - and then we'll need a higher skilled developer to supervise these machines.
They should discuss outsourcing management - it's the next logical step.
I still play (emulated) C64 games on my PC - they're lots of fun, and are very addictive, though somewhat lacking in the graphics department.
I find it strange that the Palm, and generally all modern PDAs and cell phones have 20 times as much power as the old C64 in every measurement, yet most of the games suck.
Most home users copy, and even that is starting to change lately, with awareness promoted by the BSA, mostly.
Most businesses buy. And when you buy, it costs here the same as it does in the US (even though salaries are lower).
MS Office support for hebrew is bearable (not perfect, but it gets the job done). Compatibility of office versions with respect to Hebrew is horrible - numbers and punctuation are reordered, randomly it seems, when you save in one version and open in another.
I must say it warms my heart - but I'm a bit pessimistic, you see ... Israel has got some of the best politicians money can buy; And, judging from the enthusiastic appearance of two of our ministers in Microsoft's latest "microsoft in the government expo" in Italy, I think Microsoft Israel is well aware of the commodity status of Israeli politicians.
Before Microsoft Windows was considered stable enough to run servers, the norm was to pack as much as possible onto each server. On SPARC/Solaris servers, that's still the case, and is the reason why 64-way servers are actually reasonable - every processor can access the file system, and they share the load. To get comparable (in the CPU power sense) from independent computers, you'd need much more CPUs, perhaps 4 times as much, because an idle CPU in one box can't take over an overloaded CPU in another.
But Microsoft has made it standard practice to set up a box for every service (And making you pay lots of microsoft tax in the process). And at the NT4 days, that was a good practice, because the O/S wasn't stable enough to be trusted with more than one task at a time - Exchange, SQL-Server and other services could all bring the entire O/S down when a malformed user request came in.
Nowadays, the Windows 2000 and 2003 systems are stable enough to run more than one service at a time. They don't use their resources efficiently, but they are capable of the level of separation needed, and are generally stable enough. Yet, many applications are built with the older mindset, and make demands that are not reasonable for a machine that hosts many services. For example, they might require a specific service pack, a specific version of another tool (Database, Exchange, whatever). A batch text processing system might require a specific Word version to be installed on the server, and a different system might require another.
If two such demands are incompatible, you're out of luck because Microsoft doesn't let you install two versions of the software at the same time. Most Unix packages can have multiple version co-existing at the same time.
So, to sum up - on modern operating systems, there's no real inherent technical argument against packing all the services on one machine. However, due to historical reasons (tracing back to days when this was unpractical), you often have no choice but dedicate different boxes to different services when running Microsoft O/S and its tools. There is rarely such a problem on Unix systems and their like.