A few days ago, The Iowa Department of Commerce Utilities Board put a stop to traffic pumping in Iowa. It seems that a number of small telcos like "The Farmers and Merchants Mutual Telephone Company of Wayland, Iowa" were overcharging long distance carriers for "terminating" large numbers of long distance calls that were actually shipped elsewhere.
(Unlike the Internet, there is inter-company billing within the telephone system.) This service was used mostly for conference bridges and dial-a-porn. Sprint, which offers flat-rate long distance service within the US, was losing money on calls to those numbers. So Sprint blocked them and filed a complaint with the Iowa authorities.
Iowa ruled this week that the telcos were overcharging, had to stop it, and had to give the money back.
Sprint also had to stop blocking, which won't be a problem once the rates come down.
The FCC is working on this problem nationally, but the worst offenders just got shut down.
Is "http://www.project10tothe100.com/" actually associated with Google? The site looks phony. The "about" page is an off-site link to Google. The code contains links to "appspot.com", so Google is hosting an application, but that doesn't mean Google is behind it. There's suspicious Javascript that constructs a domain name. There's no SSL cert. The "robots.txt" file blocks everybody.
The domain is registered to Google, though. And it's registered through MarkMonitor. (MarkMonitor is the "if you have to ask, you can't afford it" domain registrar. They register domains like "gm.com", "google.com", "hp.com", and "ford.com".) That's the only indication of legitimacy on the site, though.
The FTC and the Justice Department's antitrust unit were out to lunch during the Bush administration, but that seems to be changing. Stay tuned for enforcement.
First off, Jamie Zawinski no longer programs much. He runs a nightclub.
Second, Joel Spolsky isn't exactly a big name on programming. He's better known as a blogger than a developer. He runs a little company that makes a desktop project tracking tool. That's not rocket science. We're not hearing this "duct tape" stuff from people like Dave Cutler, who designed VMS and Windows NT. Or lead developers on MySQL. Or big names in game development.
Spolsky is taking potshots at the template framework crowd. He has a point there. I've been very critical in that area myself; I think the C++ standards committee is lost in template la-la land. The real problem with C++ is that the underlying language has a few painful flaws for historical reasons, and attempts to paper those flaws over with templates never quite work. (Read up on the history of auto_ptr to understand the pain.) But that's almost a historical issue now. Newer languages such as Java and Python aren't as dependent on templates as is C++. If you get the basic language design right, you don't need templates as much.
Demonstrate how Wikipedia locks. Edit something in the Wikipedia sandbox from two adjacent computers, demonstrate what an "edit conflict" looks like, and show how you resolve it.
Demonstrate how a hard-lock version control system, like Microsoft Visual SourceSafe, locks. Show what happens when you try to check out something on one machine that's already locked on another machine.
Point out what happens if someone leaves something checked out.
Get your management to decide which approach they want.
Quit whining and user test. The "I'm l33t because I can use the command line" crowd should just shut up. The sysadmins whose ego is wrapped up in being able to edit config files with "vi" need to grow up.
User testing is straightforward. You need a setup which lets you record synchronized video from both the user's screen and the user's face and voice. Then you give people with various levels of experience various tasks, and record the results. You go through the video, and note when they got stuck, when they couldn't find what to do next, and when they had to back up and undo something.
Then check the notes for any problem that occurs more than once. This isn't rocket science.
User interface design is not about eye candy. The key issue is avoiding user dead ends, backup, and redo. Nor are "wizards" papered over command line programs the answer. In such systems, there's usually too little info coming back; the "wizard" doesn't understand problems reported by the lower levels.
Apple used to be fanatical about this, before they had to bolt the Mac model onto a UNIX model. One of Apple's original rules was "You should never have to tell the computer something it already knows." If you find yourself typing in serial numbers or IP addresses, the interface is broken.
The single wheel on the U3-X is made up of many tiny motor-controlled wheels, packed inside the bigger wheel, allowing the device to swerve in any direction.
A wheel made of up of smaller wheels is one of the classic bad ideas of robotics. Back in the 1980s, when robot motion planning software barely worked, many mobile robots were "holonomic" or "omni-drive": they could move in any direction without turning first. One of the popular geometries was three big wheels on axes 120 degrees apart (that robot is in a display case in the lobby of the computer science building at Stanford), with each big wheel composed of little wheels around the rim. This mechanism can execute any rotation or translation.
The problem is that the little wheels only work on hard, flat terrain. Shag rugs are a problem. Grass, dirt, and mud, no way.
If you know this is coming, get extra air filters, use the absolute kind (like 3M Filtrete), and be prepared to change them frequently.
With absolute filters, the filters will gradually stop letting air through as they clog, so you
must inspect them regularly or have clogged-filter sensors. The usual fibreglas filters don't
even try to stop 100% of the particles above the filter's size limit, but they tend to still pass
air even when clogged, so neglecting them doesn't stop airflow.
And use duct tape to fix any leaks around the filters.
Now that the US has been operating in the sandbox for years now, keeping gear going during sandstorms is well understood.
BIND, like Sendmail, is one of those legacy pieces of Berkeley software from the 1980s that should have been retired a long time ago.
A basic problem with both of those packages is that they're database applications without a database. Back in the 1980s, there were no good database programs available for UNIX, and some apps had to roll their own. We're way past that.
There are open-source database-based alternatives.
Qmail is a database-based replacement for Sendmail, and it's generally considered to be much more stable and secure. (At this late date, nobody should be running Sendmail.) There's MyDNS, which is a MySQL-based DNS program, but that's never really caught on. The big commercial DNS systems are all database-based.
Let's take a look at the patch history of QNX. QNX is a message passing microkernel mostly used for embedded systems. But it can be run with a full GUI, runs on multiprocessors, and can be run as a server. Millions of "headless" embedded systems have QNX inside. I used it in a DARPA Grand Challenge vehicle. BigDog, the legged robot, runs QNX.
Drivers are outside the kernel. All drivers. File systems are outside the kernel. Networking is outside the kernel. And they're all application programs, not some special kind of loadable kernel module.
There have been 14 patches to QNX in the last two years. Only one is an actual kernel patch: "This patch contains updates to the PPCBE version of the SMP kernel. You need this patch only for Freescale MPC8641D boards." Only one is security-related: "This patch updates npm-tcpip-v6.so to fix a Denial of Service vulnerability where receipt of a specially crafted network packet forces the io-net network manager to fault (terminate)." Neither Linux nor Windows comes close to that record.
There's little "churn" in a good microkernel. Since little code is going in, new bugs aren't going in.
Good microkernels tend to slowly converge toward a zero-bugs state.
QNX generally has a "there's only one way to do it" approach, like Python. Linux supports three completely different driver placement - compiled into the kernel, loadable as a kernel module at boot time, and run as a user process. QNX only supports one - run as a user process "resource manager". That simplifies things. A "one way to do it" approach means that the one best way is thoroughly exercised and tested. There are few seldom-used dark corners in critical code.
When QNX boots, it brings in an image with the kernel, a built-in process called "proc", any programs built into the boot image, and any shared objects ".so" wanted at boot. These last two run entirely in user space; they're just put in the boot image so they're there at startup. That's how drivers needed at startup get loaded. They don't have to be in the kernel. (In fact, you can put the whole boot image in ROM, and many embedded systems do this.)
A QNX "resource manager" is a program which has registered to receive messages for a certain portion of pathname space. The QNX kernel has no file systems; part of the initial "proc" process is a little program which keeps an in-memory table of "resource managers" and what part of pathname space they manage. This is similar to "mounting" a driver under Linux, but it doesn't require a file system up during boot. File systems are user programs which start up and ask for some pathname space, after which "open" messages are directed to that file system.
Another QNX simplification is that the kernel doesn't load programs. "exec" is implemented by a shared library. That library is loaded with the boot image, to allow things to start up. "exec" runs entirely in user space, with no special privileges, so if there's a bug in "exec" vulnerable to a mis-constructed executable, that program load fails and everything else goes on normally.
The price paid for this is some extra copying, since all I/O is done by message passing. This isn't
much of a cost any more, because you're almost always copying from cache to cache. That's an important point. Message passing kernels used to be seen as expensive due to copying cost. But today, copying recently used material is cheap. On the other hand, some early microkernels (Mach comes to mind) worked very hard to mess with the MMU to avoid big copies, moving blocks from one address space to another by changing the MMU. This seems to be a lose on modern CPUs; the cache flushing required when you mess with the address space on recently used data hurts performance.
I used to pump uncompressed video through QNX message passing using 2% of a Pentium III class CPU. Message passing, done right, is not a major performance problem.
You'll need a proper desk to display that teletype.
Now that it's working, the plan is to build a case of brass and glass, with lights inside, to show off the mechanism and make it look Victorian-era. The California Steampunk Exhibition is on for spring 2010 (2009 was canceled due to the recession), and I want to have it there.
The software for this is on SourceForge, if anybody else has a Baudot teletype machine. It not only does RSS feeds, but you can send SMS messages from the Teletype. The idea is to support modern communications with very retro technology.
This doesn't need to end once you hit the hard limit of silicon, because then the technology for making things that small will mature and you will still be able to get the same number of transistors but for half the price.
We're starting to hit some fundamental limits. The ultimate one, of course, is that at some point atoms are too big, and you need at least one electron per bit of data. We're not quite there yet, but we're getting close. There's serious work on single electron memory cells.
The current big problem is getting rid of the heat. Smaller transistors are possible, but not too many of them can be active at one time. This is why flash memory currently leads in density; at any one time, only a small fraction of the transistors are using power. Even RAM now has serious heat dissipation problems. With CPUs, heat removal dominates design. Chip designers are worried about MIPS/watt, rather than MIPS/cm^2. 3D designs have it worse; getting the heat out of the intermediate layers is very tough.
There's another limit worth bearing in mind - electromigration. After fabrication has put all the atoms where they're supposed to be, they don't always stay there. Some, pulled by electrical and thermal forces, do come off edges and move around. This causes failures over time. The smaller the device, the worse the effects of this process.
As a steampunk project, I've restored a Teletype Model 15, which, running uncased so you can see the insides, is an impressive piece of machinery. I have this connected to a program that polls any desired collection of RSS feeds and prints any new items that come in.
Running off Reuters or the NPR news feed, it emulates a classic news ticker.
I've tried giving it a list of Twitter RSS feeds. Works fine, looks stupid. For each new twit,
the motor winds up to speed, 50 pounds of machinery grinds into life, and with much clattering and banging, the machine hammers out some banal twit on a long roll of yellow paper. Twitter content is just too lame for this. Hooked to the Reuters feed, at least you get the feeling that you're keeping up with what's going on in the world. Not with Twitter.
(Incidentally, Twitter's server-side RSS implementation sucks. RSS feed servers are supposed to accept a query with a number obtained from the previous query, and if the numbers match, it means nothing changed and no new text is transmitted. Twitter implements that so badly that every poll results in transmitting the entire RSS content again, even if nothing changed. Most other RSS feeds, such as Reuters, more or less get this right.)
Of course it's a dead skill. Along with copperplate, chancery cursive, Bell's Visible Speech, and Pitman shorthand. Those were all industrial-strength methods intended to solve the problem of getting information onto paper at a reasonably high rate of speed.
It may take longer for the ideographic languages to give in. Japanese and Chinese have the problem that
the manual typewriters for a 3000-character font were really slow and clunky, so much business paperwork was handwritten until computers came in. Typing on those things was slower than writing the ideographs. Even today, none of the keyboard input systems are that great. Drawing ideographs on a touchscreen isn't unusual. (There is, incidentally, a neat touch-screen kanji input program for the Nintendo DS.)
Stenotype keyboards live on, faster than anything else. I'm surprised they've never become popular for chat.
I mean, the language reference explicitly defines member lookup as string lookup in a dictionary associated with an object! This is going to be an order of magnitude slower than vtable dispatch in a statically typed language, there's no way around it.
Sure there is. See Shed Skin. You have to do global analysis.
It helps if you apply some restrictions. For example, it's reasonable to require that you can't add a new member to a class dynamically unless the module defining the class does so, in at least one place.
So class member functions can self-modify, but must do so visibly.
(You can still subclass from another module if you really need to override something.)
Then you can define "slots" for all the fields, and for many of them, determine their static type.
The real problem for optimization is "hidden dynamism" - places where the dynamic binding features are invoked, but that's not obvious from the source code. "Hidden dynamism" is rare - if you eval a string, it's not usually because you want to load a new function definition that replaces an existing one. (And if the string does that, the odds are that it's a security problem comparable to an SQL injection attack.) All I'd ask is that if you're going to have hidden dynamism, you must also have some explicit dynamism for the same member. (You might have to write something like "self.fn = self.fn", so the optimizer picks up on the concept that ".fn" is subject to being redefined.
But such requirements should be rare. I'm not proposing to add declarations to the language.
Not even via "decorators".)
Good implicit typing requires program-wide analysis, because you need to see all the calls to a function to find out what types are used in the calls. If parameter 1 is always an integer, and parameter 2 is always a string, then considerable optimization becomes possible.
It's also useful to determine at compile time which functions "keep" references to their arguments. If an argument comes in, and nothing in the function ever keeps a reference to it beyond function exit, then you don't need to update reference counts for that argument, because the function can't extend its life.
There are lots of other potential optimizations for Python. For example, if the compiler determines that an argument is always a small immutable (int, float, a small object like a complex number
or a 3D point) and is never "kept", it may be cheaper to pass it by value. This is a generalization of "unboxing".
The compiler has to recognize the 0.01% of the time that something funny is going on and generate the inefficient code for the general dynamic case. The rest of the time, more static code can be used.
If you really like JIT systems, you can even fix it so that when the inefficient case comes up unexpectedly, it's properly handled, by recompiling the relevant code on the fly. I think the Unladen Swallow people are headed in that direction, and the Javascript JIT people definitely are. (The Javascript people have a much worse legacy code problem than the Python people, though. Python code is generally part of coherent programs, which are maintained as a unit. Javascript code is a collection of snippets of dubious provenance spread all over the Web.)
The big issue is that this model is not compatible with most existing threaded Python code. The single resource-space assumption is pretty deeply embedded...
It's not quite that bad. Python guarantees memory safety, and the atomicity of some sequence operations, like "append". But it's not a run-to-completion thread system. Fine-grained locks wouldn't break many pure Python applications.
The big issue is "who owns what" for locking purposes. I discussed some ways to deal with that issue on the Usenet Python group a few weeks ago. If you're willing to introduce "synchronized objects" (like Java) and require that all other mutable objects be linked to either by one thread or one synchronized object at a time, it can all be made to work without extensive programmer effort.
Immutable objects don't have sharing problems, except for memory allocation purposes. (That's what concurrent garbage collection is for.)
Or you can punt and use large numbers of individual processes, but that eats memory. I have an application which works that way. It works fine, but each process takes 10MB or so, while a thread to do the same job would need only 100K.
In an era where the typical new server has 8 processors, this really needs to be solved.
Because it IS easier to use and hardware is always getting cheaper.
On the server side, 10x to 30x slower means building entire buildings full of servers. Or more expensive hardware in the cell phone. Or using another language.
Python is actually a good general-purpose programming language, not just a "scripting language". The big problem is slow execution.
The basic problem with CPython is that, being a naive interpreter, it has to check for
the hard cases every time. "n = n + 1" ought to be a few machine instructions, maybe only one.
But Python has to check for n being a float, n being a string, n being an object, etc., every time.
Shed Skin, with a type inference systems, does analysis at compile time and determines that n is always an integer, then generates code for integer arithmetic only.
There are all sorts of dynamic things one can do to a Python program while it's running. You can add a function to an object. You can replace existing functions. You can load new modules on the fly. But most of the time, for most of the objects, you don't do that. An efficient implementation needs to separate out the cases where something unusual might occur, and the far more common cases when the program is doing routine stuff that needs to go fast. The common cases can then be handled with much simpler, and faster, code.
Python is quite a good language, but the implementations suck. This is holding back widespread
use of Python. It's too slow, typically 10x to 30x slower than C. That's far worse than Java.
There have been several attempts at other implementations. But because Guido Rossum fights formal standardization of the language, treating his CPython implementation as a de-facto standard,
everyone else has a moving target to hit.
Google (who hired Guido) likes Python, but they don't like the low performance.
CPython is a "naive interpreter" (very little optimization). Worse, with the rather lame implementation the Global Interpreter Lock, not only can't it use a multi-core CPU effectively, multi-thread programs run slower on multi-core CPUs. (The threads fight over the lock in an embarrassingly inefficient way.)
Google is doing "Unladen Swallow", which is an attempt to bolt CPython to a just-in-time compiler to a virtual machine. It's not clear how well that will work out, but the end result will have more layers than seems to be indicated. The goal is 5x faster than CPython, which won't beat Java, let alone C/C++.
It's cute that Python to JavaScript translation is possible, but it's not going to help much on the performance front.
For a few years, the great hope of the Python community was PyPy, but that had too many goals, was being developed in "sprints", and after five years, the European Union pulled the plug on funding after it was slower than CPython.
Shed Skin, which is a Python to C++ hard-code compiler, is currently the lead in Python performance, but it doesn't yet implement the whole language. Still, with about two people working on it, Shed Skin is doing better than most of the bigger projects. Shed Skin does automatic type inference. Python doesn't have declarations, but with enough analysis, the compiler can figure out what types each variable can hold and generate hard types, which makes for much faster code.
First, what's needed is a site that collates failure reports and includes unit serial numbers. Then you can get a handle on the problem. If you're seeing soldering failures, it's probably a process problem at the assembly plant. If the serial numbers cluster, there's a process variation failure. If they're randomly distributed, there may be a design problem. Process variation problems are usually
easy to fix if you have the data. Well-run plants log all that data (far easier now than in the manual era) and can go back in the logs to find the problem.
Japanese companies used to have a good handle on this; it's what Deming was all about. Consumer electronics plants in China, not so much.
The most secure OS in the world, not even Linux nor OSX, isn't going to be able to protect you when you decide to authorize and run an.exe file you downloaded.
Actually, no. It's quite possible to have a system where the downloaded.exe file is in an untrusted security compartment of a mandatory security system, such as SELinux provides. You can then run it, but it can only work on other untrusted data. That's good enough for a game.
For historical reasons, UNIX, Linux, and Windows tend to give applications the access privileges of the user using them. This is the real problem.
This patent seems to be about a way to keep big distributed game systems from choking on O(N^2) traffic.
There are other approaches; it's not clear that this one is widely used. Current practice is, I think, that clients send to the servers, and the servers send back updates to the clients that only contain information relevant to that client. The method described in the patent works more like a LAN party, where everybody is multicasting to everybody else.
The mechanism described in claim 1 might apply to multiway voice chat. That's exactly this type of problem. If you can only talk to the people "near" you, you need just that kind of distribution system. If player are in positions where A can hear B, and B can hear C, but A can't hear C, you need the kind of aggregation described.
Better voice systems are way beyond that. I went to a demo at Dolby Labs where they had multiplayer positional audio with proper attenuation and occlusion as the players moved around. As a demo, they had a player move entirely around the audience, while talking to someone who wasn't moving, and it all sounded right. Thinking about that, though, the audio processing was being done in the receiving client (to construct five channel audio, phase-synchronized, you have to do it there), so probably the message traffic was simple aggregation, plus positional info.
Multics was open sourced two or three years ago, but I haven't heard of anybody taking advantage of that to try using it again.
If the open sourcing had happened before the last machine went down, it would probably still be running in emulation. But the "open source" is a collection of listing files, and the PL/I compiler isn't available.
DOCKMASTER, the NSA's externally visible machine, was one of the last Multics machines. They
wanted a secure machine for outside access, and only Multics was considered acceptable. It shut down around 2000; they couldn't get repair parts any more.
Slashdot Mirror
A few days ago, The Iowa Department of Commerce Utilities Board put a stop to traffic pumping in Iowa. It seems that a number of small telcos like "The Farmers and Merchants Mutual Telephone Company of Wayland, Iowa" were overcharging long distance carriers for "terminating" large numbers of long distance calls that were actually shipped elsewhere. (Unlike the Internet, there is inter-company billing within the telephone system.) This service was used mostly for conference bridges and dial-a-porn. Sprint, which offers flat-rate long distance service within the US, was losing money on calls to those numbers. So Sprint blocked them and filed a complaint with the Iowa authorities.
Iowa ruled this week that the telcos were overcharging, had to stop it, and had to give the money back. Sprint also had to stop blocking, which won't be a problem once the rates come down.
The FCC is working on this problem nationally, but the worst offenders just got shut down.
Is "http://www.project10tothe100.com/" actually associated with Google? The site looks phony. The "about" page is an off-site link to Google. The code contains links to "appspot.com", so Google is hosting an application, but that doesn't mean Google is behind it. There's suspicious Javascript that constructs a domain name. There's no SSL cert. The "robots.txt" file blocks everybody.
The domain is registered to Google, though. And it's registered through MarkMonitor. (MarkMonitor is the "if you have to ask, you can't afford it" domain registrar. They register domains like "gm.com", "google.com", "hp.com", and "ford.com".) That's the only indication of legitimacy on the site, though.
We know why most audio CDs cost $17.99. Illegal price fixing.
We know why video games cost $60. Illegal price fixing.
The FTC and the Justice Department's antitrust unit were out to lunch during the Bush administration, but that seems to be changing. Stay tuned for enforcement.
First off, Jamie Zawinski no longer programs much. He runs a nightclub.
Second, Joel Spolsky isn't exactly a big name on programming. He's better known as a blogger than a developer. He runs a little company that makes a desktop project tracking tool. That's not rocket science. We're not hearing this "duct tape" stuff from people like Dave Cutler, who designed VMS and Windows NT. Or lead developers on MySQL. Or big names in game development.
Spolsky is taking potshots at the template framework crowd. He has a point there. I've been very critical in that area myself; I think the C++ standards committee is lost in template la-la land. The real problem with C++ is that the underlying language has a few painful flaws for historical reasons, and attempts to paper those flaws over with templates never quite work. (Read up on the history of auto_ptr to understand the pain.) But that's almost a historical issue now. Newer languages such as Java and Python aren't as dependent on templates as is C++. If you get the basic language design right, you don't need templates as much.
OK. Here's what to do.
Quit whining and user test. The "I'm l33t because I can use the command line" crowd should just shut up. The sysadmins whose ego is wrapped up in being able to edit config files with "vi" need to grow up.
User testing is straightforward. You need a setup which lets you record synchronized video from both the user's screen and the user's face and voice. Then you give people with various levels of experience various tasks, and record the results. You go through the video, and note when they got stuck, when they couldn't find what to do next, and when they had to back up and undo something. Then check the notes for any problem that occurs more than once. This isn't rocket science.
User interface design is not about eye candy. The key issue is avoiding user dead ends, backup, and redo. Nor are "wizards" papered over command line programs the answer. In such systems, there's usually too little info coming back; the "wizard" doesn't understand problems reported by the lower levels.
Apple used to be fanatical about this, before they had to bolt the Mac model onto a UNIX model. One of Apple's original rules was "You should never have to tell the computer something it already knows." If you find yourself typing in serial numbers or IP addresses, the interface is broken.
The single wheel on the U3-X is made up of many tiny motor-controlled wheels, packed inside the bigger wheel, allowing the device to swerve in any direction.
A wheel made of up of smaller wheels is one of the classic bad ideas of robotics. Back in the 1980s, when robot motion planning software barely worked, many mobile robots were "holonomic" or "omni-drive": they could move in any direction without turning first. One of the popular geometries was three big wheels on axes 120 degrees apart (that robot is in a display case in the lobby of the computer science building at Stanford), with each big wheel composed of little wheels around the rim. This mechanism can execute any rotation or translation.
The problem is that the little wheels only work on hard, flat terrain. Shag rugs are a problem. Grass, dirt, and mud, no way.
If you know this is coming, get extra air filters, use the absolute kind (like 3M Filtrete), and be prepared to change them frequently. With absolute filters, the filters will gradually stop letting air through as they clog, so you must inspect them regularly or have clogged-filter sensors. The usual fibreglas filters don't even try to stop 100% of the particles above the filter's size limit, but they tend to still pass air even when clogged, so neglecting them doesn't stop airflow.
And use duct tape to fix any leaks around the filters.
Now that the US has been operating in the sandbox for years now, keeping gear going during sandstorms is well understood.
I'd rather have text menus than vast numbers of tiny obscure icons.
And if we have to have tiny obscure icons, someone as good as Susan Kare is needed to design them.
BIND, like Sendmail, is one of those legacy pieces of Berkeley software from the 1980s that should have been retired a long time ago.
A basic problem with both of those packages is that they're database applications without a database. Back in the 1980s, there were no good database programs available for UNIX, and some apps had to roll their own. We're way past that.
There are open-source database-based alternatives. Qmail is a database-based replacement for Sendmail, and it's generally considered to be much more stable and secure. (At this late date, nobody should be running Sendmail.) There's MyDNS, which is a MySQL-based DNS program, but that's never really caught on. The big commercial DNS systems are all database-based.
Let's take a look at the patch history of QNX. QNX is a message passing microkernel mostly used for embedded systems. But it can be run with a full GUI, runs on multiprocessors, and can be run as a server. Millions of "headless" embedded systems have QNX inside. I used it in a DARPA Grand Challenge vehicle. BigDog, the legged robot, runs QNX.
Drivers are outside the kernel. All drivers. File systems are outside the kernel. Networking is outside the kernel. And they're all application programs, not some special kind of loadable kernel module.
There have been 14 patches to QNX in the last two years. Only one is an actual kernel patch: "This patch contains updates to the PPCBE version of the SMP kernel. You need this patch only for Freescale MPC8641D boards." Only one is security-related: "This patch updates npm-tcpip-v6.so to fix a Denial of Service vulnerability where receipt of a specially crafted network packet forces the io-net network manager to fault (terminate)." Neither Linux nor Windows comes close to that record.
There's little "churn" in a good microkernel. Since little code is going in, new bugs aren't going in. Good microkernels tend to slowly converge toward a zero-bugs state.
QNX generally has a "there's only one way to do it" approach, like Python. Linux supports three completely different driver placement - compiled into the kernel, loadable as a kernel module at boot time, and run as a user process. QNX only supports one - run as a user process "resource manager". That simplifies things. A "one way to do it" approach means that the one best way is thoroughly exercised and tested. There are few seldom-used dark corners in critical code.
When QNX boots, it brings in an image with the kernel, a built-in process called "proc", any programs built into the boot image, and any shared objects ".so" wanted at boot. These last two run entirely in user space; they're just put in the boot image so they're there at startup. That's how drivers needed at startup get loaded. They don't have to be in the kernel. (In fact, you can put the whole boot image in ROM, and many embedded systems do this.)
A QNX "resource manager" is a program which has registered to receive messages for a certain portion of pathname space. The QNX kernel has no file systems; part of the initial "proc" process is a little program which keeps an in-memory table of "resource managers" and what part of pathname space they manage. This is similar to "mounting" a driver under Linux, but it doesn't require a file system up during boot. File systems are user programs which start up and ask for some pathname space, after which "open" messages are directed to that file system.
Another QNX simplification is that the kernel doesn't load programs. "exec" is implemented by a shared library. That library is loaded with the boot image, to allow things to start up. "exec" runs entirely in user space, with no special privileges, so if there's a bug in "exec" vulnerable to a mis-constructed executable, that program load fails and everything else goes on normally.
The price paid for this is some extra copying, since all I/O is done by message passing. This isn't much of a cost any more, because you're almost always copying from cache to cache. That's an important point. Message passing kernels used to be seen as expensive due to copying cost. But today, copying recently used material is cheap. On the other hand, some early microkernels (Mach comes to mind) worked very hard to mess with the MMU to avoid big copies, moving blocks from one address space to another by changing the MMU. This seems to be a lose on modern CPUs; the cache flushing required when you mess with the address space on recently used data hurts performance.
I used to pump uncompressed video through QNX message passing using 2% of a Pentium III class CPU. Message passing, done right, is not a major performance problem.
You'll need a proper desk to display that teletype.
Now that it's working, the plan is to build a case of brass and glass, with lights inside, to show off the mechanism and make it look Victorian-era. The California Steampunk Exhibition is on for spring 2010 (2009 was canceled due to the recession), and I want to have it there.
The software for this is on SourceForge, if anybody else has a Baudot teletype machine. It not only does RSS feeds, but you can send SMS messages from the Teletype. The idea is to support modern communications with very retro technology.
This doesn't need to end once you hit the hard limit of silicon, because then the technology for making things that small will mature and you will still be able to get the same number of transistors but for half the price.
We're starting to hit some fundamental limits. The ultimate one, of course, is that at some point atoms are too big, and you need at least one electron per bit of data. We're not quite there yet, but we're getting close. There's serious work on single electron memory cells.
The current big problem is getting rid of the heat. Smaller transistors are possible, but not too many of them can be active at one time. This is why flash memory currently leads in density; at any one time, only a small fraction of the transistors are using power. Even RAM now has serious heat dissipation problems. With CPUs, heat removal dominates design. Chip designers are worried about MIPS/watt, rather than MIPS/cm^2. 3D designs have it worse; getting the heat out of the intermediate layers is very tough.
There's another limit worth bearing in mind - electromigration. After fabrication has put all the atoms where they're supposed to be, they don't always stay there. Some, pulled by electrical and thermal forces, do come off edges and move around. This causes failures over time. The smaller the device, the worse the effects of this process.
As a steampunk project, I've restored a Teletype Model 15, which, running uncased so you can see the insides, is an impressive piece of machinery. I have this connected to a program that polls any desired collection of RSS feeds and prints any new items that come in. Running off Reuters or the NPR news feed, it emulates a classic news ticker.
I've tried giving it a list of Twitter RSS feeds. Works fine, looks stupid. For each new twit, the motor winds up to speed, 50 pounds of machinery grinds into life, and with much clattering and banging, the machine hammers out some banal twit on a long roll of yellow paper. Twitter content is just too lame for this. Hooked to the Reuters feed, at least you get the feeling that you're keeping up with what's going on in the world. Not with Twitter.
(Incidentally, Twitter's server-side RSS implementation sucks. RSS feed servers are supposed to accept a query with a number obtained from the previous query, and if the numbers match, it means nothing changed and no new text is transmitted. Twitter implements that so badly that every poll results in transmitting the entire RSS content again, even if nothing changed. Most other RSS feeds, such as Reuters, more or less get this right.)
Of course it's a dead skill. Along with copperplate, chancery cursive, Bell's Visible Speech, and Pitman shorthand. Those were all industrial-strength methods intended to solve the problem of getting information onto paper at a reasonably high rate of speed.
It may take longer for the ideographic languages to give in. Japanese and Chinese have the problem that the manual typewriters for a 3000-character font were really slow and clunky, so much business paperwork was handwritten until computers came in. Typing on those things was slower than writing the ideographs. Even today, none of the keyboard input systems are that great. Drawing ideographs on a touchscreen isn't unusual. (There is, incidentally, a neat touch-screen kanji input program for the Nintendo DS.)
Stenotype keyboards live on, faster than anything else. I'm surprised they've never become popular for chat.
I mean, the language reference explicitly defines member lookup as string lookup in a dictionary associated with an object! This is going to be an order of magnitude slower than vtable dispatch in a statically typed language, there's no way around it.
Sure there is. See Shed Skin. You have to do global analysis.
It helps if you apply some restrictions. For example, it's reasonable to require that you can't add a new member to a class dynamically unless the module defining the class does so, in at least one place. So class member functions can self-modify, but must do so visibly. (You can still subclass from another module if you really need to override something.) Then you can define "slots" for all the fields, and for many of them, determine their static type.
The real problem for optimization is "hidden dynamism" - places where the dynamic binding features are invoked, but that's not obvious from the source code. "Hidden dynamism" is rare - if you eval a string, it's not usually because you want to load a new function definition that replaces an existing one. (And if the string does that, the odds are that it's a security problem comparable to an SQL injection attack.) All I'd ask is that if you're going to have hidden dynamism, you must also have some explicit dynamism for the same member. (You might have to write something like "self.fn = self.fn", so the optimizer picks up on the concept that ".fn" is subject to being redefined. But such requirements should be rare. I'm not proposing to add declarations to the language. Not even via "decorators".)
Good implicit typing requires program-wide analysis, because you need to see all the calls to a function to find out what types are used in the calls. If parameter 1 is always an integer, and parameter 2 is always a string, then considerable optimization becomes possible.
It's also useful to determine at compile time which functions "keep" references to their arguments. If an argument comes in, and nothing in the function ever keeps a reference to it beyond function exit, then you don't need to update reference counts for that argument, because the function can't extend its life.
There are lots of other potential optimizations for Python. For example, if the compiler determines that an argument is always a small immutable (int, float, a small object like a complex number or a 3D point) and is never "kept", it may be cheaper to pass it by value. This is a generalization of "unboxing".
The compiler has to recognize the 0.01% of the time that something funny is going on and generate the inefficient code for the general dynamic case. The rest of the time, more static code can be used.
If you really like JIT systems, you can even fix it so that when the inefficient case comes up unexpectedly, it's properly handled, by recompiling the relevant code on the fly. I think the Unladen Swallow people are headed in that direction, and the Javascript JIT people definitely are. (The Javascript people have a much worse legacy code problem than the Python people, though. Python code is generally part of coherent programs, which are maintained as a unit. Javascript code is a collection of snippets of dubious provenance spread all over the Web.)
That video is the kind of crap which people with an animation program and no talent produce.
The big issue is that this model is not compatible with most existing threaded Python code. The single resource-space assumption is pretty deeply embedded...
It's not quite that bad. Python guarantees memory safety, and the atomicity of some sequence operations, like "append". But it's not a run-to-completion thread system. Fine-grained locks wouldn't break many pure Python applications.
The big issue is "who owns what" for locking purposes. I discussed some ways to deal with that issue on the Usenet Python group a few weeks ago. If you're willing to introduce "synchronized objects" (like Java) and require that all other mutable objects be linked to either by one thread or one synchronized object at a time, it can all be made to work without extensive programmer effort. Immutable objects don't have sharing problems, except for memory allocation purposes. (That's what concurrent garbage collection is for.)
Or you can punt and use large numbers of individual processes, but that eats memory. I have an application which works that way. It works fine, but each process takes 10MB or so, while a thread to do the same job would need only 100K.
In an era where the typical new server has 8 processors, this really needs to be solved.
Because it IS easier to use and hardware is always getting cheaper.
On the server side, 10x to 30x slower means building entire buildings full of servers. Or more expensive hardware in the cell phone. Or using another language.
Python is actually a good general-purpose programming language, not just a "scripting language". The big problem is slow execution.
The basic problem with CPython is that, being a naive interpreter, it has to check for the hard cases every time. "n = n + 1" ought to be a few machine instructions, maybe only one. But Python has to check for n being a float, n being a string, n being an object, etc., every time. Shed Skin, with a type inference systems, does analysis at compile time and determines that n is always an integer, then generates code for integer arithmetic only.
There are all sorts of dynamic things one can do to a Python program while it's running. You can add a function to an object. You can replace existing functions. You can load new modules on the fly. But most of the time, for most of the objects, you don't do that. An efficient implementation needs to separate out the cases where something unusual might occur, and the far more common cases when the program is doing routine stuff that needs to go fast. The common cases can then be handled with much simpler, and faster, code.
Python is quite a good language, but the implementations suck. This is holding back widespread use of Python. It's too slow, typically 10x to 30x slower than C. That's far worse than Java.
There have been several attempts at other implementations. But because Guido Rossum fights formal standardization of the language, treating his CPython implementation as a de-facto standard, everyone else has a moving target to hit.
Google (who hired Guido) likes Python, but they don't like the low performance. CPython is a "naive interpreter" (very little optimization). Worse, with the rather lame implementation the Global Interpreter Lock, not only can't it use a multi-core CPU effectively, multi-thread programs run slower on multi-core CPUs. (The threads fight over the lock in an embarrassingly inefficient way.)
Google is doing "Unladen Swallow", which is an attempt to bolt CPython to a just-in-time compiler to a virtual machine. It's not clear how well that will work out, but the end result will have more layers than seems to be indicated. The goal is 5x faster than CPython, which won't beat Java, let alone C/C++.
It's cute that Python to JavaScript translation is possible, but it's not going to help much on the performance front.
For a few years, the great hope of the Python community was PyPy, but that had too many goals, was being developed in "sprints", and after five years, the European Union pulled the plug on funding after it was slower than CPython.
Shed Skin, which is a Python to C++ hard-code compiler, is currently the lead in Python performance, but it doesn't yet implement the whole language. Still, with about two people working on it, Shed Skin is doing better than most of the bigger projects. Shed Skin does automatic type inference. Python doesn't have declarations, but with enough analysis, the compiler can figure out what types each variable can hold and generate hard types, which makes for much faster code.
First, what's needed is a site that collates failure reports and includes unit serial numbers. Then you can get a handle on the problem. If you're seeing soldering failures, it's probably a process problem at the assembly plant. If the serial numbers cluster, there's a process variation failure. If they're randomly distributed, there may be a design problem. Process variation problems are usually easy to fix if you have the data. Well-run plants log all that data (far easier now than in the manual era) and can go back in the logs to find the problem.
Japanese companies used to have a good handle on this; it's what Deming was all about. Consumer electronics plants in China, not so much.
eBay doesn't even have a source code license. They're just a binary licensee with a bulk buy deal, like any other end user of commercial software.
They bought the customer base, but not the software.
The most secure OS in the world, not even Linux nor OSX, isn't going to be able to protect you when you decide to authorize and run an .exe file you downloaded.
Actually, no. It's quite possible to have a system where the downloaded .exe file is in an untrusted security compartment of a mandatory security system, such as SELinux provides. You can then run it, but it can only work on other untrusted data. That's good enough for a game.
For historical reasons, UNIX, Linux, and Windows tend to give applications the access privileges of the user using them. This is the real problem.
This patent seems to be about a way to keep big distributed game systems from choking on O(N^2) traffic. There are other approaches; it's not clear that this one is widely used. Current practice is, I think, that clients send to the servers, and the servers send back updates to the clients that only contain information relevant to that client. The method described in the patent works more like a LAN party, where everybody is multicasting to everybody else.
The mechanism described in claim 1 might apply to multiway voice chat. That's exactly this type of problem. If you can only talk to the people "near" you, you need just that kind of distribution system. If player are in positions where A can hear B, and B can hear C, but A can't hear C, you need the kind of aggregation described.
Better voice systems are way beyond that. I went to a demo at Dolby Labs where they had multiplayer positional audio with proper attenuation and occlusion as the players moved around. As a demo, they had a player move entirely around the audience, while talking to someone who wasn't moving, and it all sounded right. Thinking about that, though, the audio processing was being done in the receiving client (to construct five channel audio, phase-synchronized, you have to do it there), so probably the message traffic was simple aggregation, plus positional info.
Multics was open sourced two or three years ago, but I haven't heard of anybody taking advantage of that to try using it again.
If the open sourcing had happened before the last machine went down, it would probably still be running in emulation. But the "open source" is a collection of listing files, and the PL/I compiler isn't available.
DOCKMASTER, the NSA's externally visible machine, was one of the last Multics machines. They wanted a secure machine for outside access, and only Multics was considered acceptable. It shut down around 2000; they couldn't get repair parts any more.