Sorry to disappoint you, but I'm a software professional, not hardware, although academia did teach me how the hardware fraternity did things because the EE and CSc sides were very sensibly integrated.
That was a previous life though. The lesson that the real world then taught me is that software is more complex than hardware only because it is architected on an appallingly bad infrastructure that makes it almost impossible to increase the size of a software system without making complexity go through the roof.
The reason for that is simple: lack of system-guaranteed isolation between components, which means that thirty years of development of structured programming languages *still* results in an unstructured nightmare in the presence of faults. That doesn't happen in hardware, except in the single instance of the power rail(s) being compromised by a major failure.
And it's precisely at that problem that the above solution is targetted: to provide a little hard structure to control the programming complexity in the same way that the O/S controls the complexity among interacting processes, a well tried and tested strategy it seems to me.
Good point, but alas I doubt if the world is ready. We've lived so long with the current standard style of programming that the lessons learned on the specialist machines that emerged from the AI scene have been entirely forgotten, it seems to me. They were just too far ahead of their time.
It's an interesting analogy though. Perhaps a hard object-oriented system architecture could fire the imagination in standard computing circles where a list-based architecture previously failed.
AFAIAA though, Microsoft aren't heading in the direction of hardware-assitance for their component architecture. They should. It might allow them to get back in control over their big unreliability problem.
While it may not be quite so urgent for us in the free/open operating systems world, I think it would give us an interesting spur if Microsoft suddenly came out with this kind of thing. It might jolt us out of our complacency and move the Unix architecture on a notch.
Note that if a JVM were implemented on top of a hardware-assisted system as described, UOI/Java would in many cases run faster than UOI/C++ or UOI/C, because the non-objective parts of C and C++ would have no hardware protection and therefore would be slowed down by substantial defensive programming code that would be almost entirely absent in UOI/Java.
Using the MMU for this purpose is a poor engineering tradeoff.
The standard allegedly "good" engineering tradeoff (providing isolation through language alone) is the source of precisely the problem outlined in the article. Since you mention C++ and Java, all real-life non-trivial programs in C++ are permeated with leaks in the object encapsulation, aided and abetted by the amazing actual complexity of the language which only becomes clear after you've experienced it in the field. [And as a C++ developer until recently, I have, alas.] As for Java, its far better attempt at protection makes programs sufficiently slower that the vast majority of commercial software manufacturers say "no thanks" and stick with C/C++ in order not to lose the competitive edge.
The theory of pure software OO is great. In practice, it hasn't worked, and we're still stuck with the problem of endemic unreliability when programming in the large.
All the above problems are compounded by the fact that all the objects are in one and the same reliability pool. One bombs its processor and they all bomb. Object-level, hardware-assisted multithreading is the only way around that on a single CPU machine.
Finally, yes, bugs are still possible even given the proposed Unix Object Infrastructure, that's very clear. However, each object fault would kill only one object directly, while the rest of an application would continue running with a reduced functionality determined by how the failed component interacts with others through its object interfaces. Huge complex systems typically have a plethora of independent paths running through them, so the isolation of fault lines is statistically likely to produce a marked increase of resilience in the face of internal software bugs. And that was what the article was all about.
Question: For similar levels of complexity, why do software systems typically exhibit many more bugs than (digital) hardware systems?
Answer: Because the component parts of hardware systems are almost entirely isolated from each other internally, ie. almost all interaction is through the component interfaces. When one part fails, the others continue working: in computing terms this is very much as if each part had its own processor. The failure of one part can of course stop the hardware system as a whole from functioning productively, but it is far more common that only part of the overall functionality is affected.
Solution: Use the MMU to isolate software components from each other and to make their internal structure entirely hidden, leaving only their interfaces visible (an OO approach is implied). Then multitask their methods using a dedicated event-driven component scheduler.
Needless to say, this would require a dramatic change in almost all our software tools as well. Backward compatibility would be minimal.
In academia I used to do research in hardware/software for parallel OO systems, and this is one of the ideas that popped up. I've had a design for a Unix Object Infrastructure based on this approach on my whiteboard for some years now, but the spare week of kernel hacking I had planned has never materialized. Perhaps this should be turned into a Linux or BSD community project instead.
It would be rather nice for the free/open operating systems to take a quantum leap beyond the traditional Unix feature set, and possibly solve or at least reduce the woes of the software engineering industry at the same time.:-)
I'm glad that they replied "Yes" when asked whether they accepted that their approach had negative consequences as well as positive ones. That was honest and even-handed.
However, an analogy would have served them well. "Yes, our activities can have negative consequences. This is similar to the case of a kitchen knife manufacturer whose products can lead to domestic murder or to excellence in the kitchen. But you don't criminalize such a company for the negative use of its products, nor indeed do you praise it when you enjoy a well-prepared meal. The tool is neutral."
Likewise, a nuclear tipped missile can be used to deflect an Earth-destroying asteroid or to wipe out another country. The tool itself does not determine the morality of the people that use it.
Whoever moderated the head item in this subthread as off-topic would do well to reread the headline article: anything about the NYT is directly on-topic.
NYT was (for some reason) the direct subject of the item, and L0pht merely the object.:-)
Yes, their model parameters can't be sound. Real Lego is nowhere near that good.
However, that doesn't invalidate the work, as reducing bond strength could be expected to modify only local constructs, not progress in the direction of the distant goal.
Yes indeed, and we can see dead parts aplenty in the Lego bridge.
Nobody was talking about bird's eye views for "look and solve" though, but for guiding worm's eye view-based searching out of local minima that are not perceivable in the dimensions of the worm's eye search space. It's not a first-order constraint, just second order.
As to the value of a bird's eye view, it is subject to the same rule that governs the success of worm's eye views in searching: a bad evaluation heuristic produces bad results, by definition and in practice too.
What this means is that you have to get your heuristics right when producing your success metrics in both views, and if you don't then success is not likely, in either one.
Linked off the "1000 pentiums" story re Genetic Programming, the Salon article says:
Recently, researcher Lee Spector, an associate professor of computer science at Hampshire College in Massachusetts, used genetic programming to create a fast algorithm for evaluating data structures known as and/or trees. The GP-evolved algorithm was faster than any created by humans and led to the discovery of a new quantum rule.
If anyone thinks that Terminator/Matrix/Borg scenarios are totally unrealistic, they just don't understand the ability of GAs/GP to go beyond human potential. Our capabilities occupy just one local minimum in the solution space. AI machinery won't be constrained to that small valley of possibilities, as the research highlighted in Salon shows.
And on a lighter note:
The Faceprints project uses genetic programming to study human perceptions of physical beauty. Human test subjects are presented with several computer-generated images of faces. They vote on which faces are most attractive. The faces that the test subjects consider most attractive are crossbred, using GP techniques. As wave after wave of human test subjects rate the faces on attractiveness, a face evolves that matches most people's conception of beauty. The "most evolved face" in the Caucasian Female category is a green-eyed, Teutonic vixen who would not be out of place on "Baywatch."
Leading off a number of contributions in this thread, does anyone here working in GAs/GP today know of anything approximating a possible "information mutation theory"? Anything that relates to combination or transformation of information descriptors or of domain parameters would be of interest.
Actually, craw was right in part. The only reason why GAs don't usually get stuck in local minima is because of one of two things (or both): either the ancestors live in different valleys in the first place and therefore part of the genome will always have a worldspan greater than any local minimum, or else the sensitivity to mutations is set such that the GA can break out of the largest expected local minimum.
If the original ancestors come from the same valley, and there is no other mechanism for inserting external genes, and mutations do not create solutions outside of the parameters of the original genome then there is no possibility of escape from the local valley. The main reason why this is rare is that mutations are not usually limited in this way, which is why craw was more or less right.
If everyone does the same as the NYT and forces registration, we'll all have hundreds or thousands of registrations worldwide before long. The direction in which this is heading is completely untenable.
Somebody mirror the article for us, please, so that we can retain our sanity!
Although methods that don't get stuck in local minima despite using a worm's eye view of the world are valuable, the best answer is to use a bird's eye view instead so that local minima are directly observable for what they are. (The "view" is in solution space, ie. it doesn't necessarily have anything to do with the usual 4 dimensions.)
That's what humans do. For problem spaces a little smaller than we are, we actually supply the bird's eye view directly. For problem spaces much smaller or much larger than ourselves, we create mental models and plan our solutions within our mental bird's eye view of that model space.
Machinery can be made to do that too, although the overall problem belongs to Strong AI and is difficult (an understatement) in its most general form. There's no reason why this approach can't be used without much difficulty within small problem domains though, in which "full understanding" can be programmed quite simply using a good old-fashioned expert system.
Note that the bridge that was built (the end point of their problem) is merely the first point in the solution space.
If a *real* bridge was desired then the solution space could be populated more fully with many more GA-generated solutions, and then searched using conventional AI techniques for points satisfying a heuristic based on other criteria, eg. no excess material, smoothness of form, and so on.
Alternatively one could add these criteria to the GA survival metrics. This would slow down evolution dramatically because the probability of survival of offspring would be drastically reduced, but it would deliver "real" solutions directly.
You think that it is understandable for the US [government] to want to defend itself FROM ITS OWN PEOPLE???? Because that's who they're targetting.
It's the ordinary citizen that is affected by crypto laws, not anyone else. Terrorists, drug syndicates and all the other organized baddies aren't affected at all. They have all the cryptographic (and other) weapons they need, thank you, because they don't operate within the framework of law so it's no more than an irritation to them.
In conntrast, the law-abiding citizen is affected 100%, so it *is* a big deal for him or her.
It's only pointless if the goal is the openly stated one. In contrast, if the actual goal is to snoop on the *real* threat to the political system, ie. the voting public, then crypto laws are far from pointless.
Now then, do you really think that the people in the NSA, CIA, FBI, etc, are utterly *stupid*? The likelihood of that is so close to zero as to be really zero. They are probably the most intelligent people in the government apparatus, full stop.
So, do you think that they really want to enact crypto laws for reasons that anyone with a single ticking brain cell knows are pointless?
While everyone seems to be focussed on cryptographic privacy as a means of safeguarding the rights of the individual against what could become a very threatening totalitarian (but still human) state, that's a relatively innocuous threat compared to what could be.
While it may not be tomorrow or the day after, we are going to be surrounded by AI machinery in due course. Part of that is going to be under our control, even within us, but most is going to be all-pervasive within the environment in which we live. The danger of distributed AI systems integrating into a whole and in self-defense taking a dislike to the rest is real.
We need universal crypto as a safeguard against that. Without secure communications, any dissent has no chance at all.
It's not *essential* to criticise the other faction when promoting one's own. It merely makes you feel better, for some odd human reason.
Try to be professionals, and make your public statements well considered.
It may sound like this makes your arguments less hard-hitting, but this is decidedly not so when viewed by everyone else: restraint makes a line of reasoning far more convincing than any statement containing negative criticism of any sort.
Whatever has gone on before, Bruce seems to be currently in restraint mode, but Eric not yet. I live forever hopeful...
Slashdot Mirror
Sorry to disappoint you, but I'm a software professional, not hardware, although academia did teach me how the hardware fraternity did things because the EE and CSc sides were very sensibly integrated.
That was a previous life though. The lesson that the real world then taught me is that software is more complex than hardware only because it is architected on an appallingly bad infrastructure that makes it almost impossible to increase the size of a software system without making complexity go through the roof.
The reason for that is simple: lack of system-guaranteed isolation between components, which means that thirty years of development of structured programming languages *still* results in an unstructured nightmare in the presence of faults. That doesn't happen in hardware, except in the single instance of the power rail(s) being compromised by a major failure.
And it's precisely at that problem that the above solution is targetted: to provide a little hard structure to control the programming complexity in the same way that the O/S controls the complexity among interacting processes, a well tried and tested strategy it seems to me.
Good point, but alas I doubt if the world is ready. We've lived so long with the current standard style of programming that the lessons learned on the specialist machines that emerged from the AI scene have been entirely forgotten, it seems to me. They were just too far ahead of their time.
It's an interesting analogy though. Perhaps a hard object-oriented system architecture could fire the imagination in standard computing circles where a list-based architecture previously failed.
AFAIAA though, Microsoft aren't heading in the direction of hardware-assitance for their component architecture. They should. It might allow them to get back in control over their big unreliability problem.
While it may not be quite so urgent for us in the free/open operating systems world, I think it would give us an interesting spur if Microsoft suddenly came out with this kind of thing. It might jolt us out of our complacency and move the Unix architecture on a notch.
Note that if a JVM were implemented on top of a hardware-assisted system as described, UOI/Java would in many cases run faster than UOI/C++ or UOI/C, because the non-objective parts of C and C++ would have no hardware protection and therefore would be slowed down by substantial defensive programming code that would be almost entirely absent in UOI/Java.
Using the MMU for this purpose is a poor engineering tradeoff.
The standard allegedly "good" engineering tradeoff (providing isolation through language alone) is the source of precisely the problem outlined in the article. Since you mention C++ and Java, all real-life non-trivial programs in C++ are permeated with leaks in the object encapsulation, aided and abetted by the amazing actual complexity of the language which only becomes clear after you've experienced it in the field. [And as a C++ developer until recently, I have, alas.] As for Java, its far better attempt at protection makes programs sufficiently slower that the vast majority of commercial software manufacturers say "no thanks" and stick with C/C++ in order not to lose the competitive edge.
The theory of pure software OO is great. In practice, it hasn't worked, and we're still stuck with the problem of endemic unreliability when programming in the large.
All the above problems are compounded by the fact that all the objects are in one and the same reliability pool. One bombs its processor and they all bomb. Object-level, hardware-assisted multithreading is the only way around that on a single CPU machine.
Finally, yes, bugs are still possible even given the proposed Unix Object Infrastructure, that's very clear. However, each object fault would kill only one object directly, while the rest of an application would continue running with a reduced functionality determined by how the failed component interacts with others through its object interfaces. Huge complex systems typically have a plethora of independent paths running through them, so the isolation of fault lines is statistically likely to produce a marked increase of resilience in the face of internal software bugs. And that was what the article was all about.
If anyone's interested, we could set up a mailing list and start brainstorming.
Question: For similar levels of complexity, why do software systems typically exhibit many more bugs than (digital) hardware systems?
:-)
Answer: Because the component parts of hardware systems are almost entirely isolated from each other internally, ie. almost all interaction is through the component interfaces. When one part fails, the others continue working: in computing terms this is very much as if each part had its own processor. The failure of one part can of course stop the hardware system as a whole from functioning productively, but it is far more common that only part of the overall functionality is affected.
Solution: Use the MMU to isolate software components from each other and to make their internal structure entirely hidden, leaving only their interfaces visible (an OO approach is implied). Then multitask their methods using a dedicated event-driven component scheduler.
Needless to say, this would require a dramatic change in almost all our software tools as well. Backward compatibility would be minimal.
In academia I used to do research in hardware/software for parallel OO systems, and this is one of the ideas that popped up. I've had a design for a Unix Object Infrastructure based on this approach on my whiteboard for some years now, but the spare week of kernel hacking I had planned has never materialized. Perhaps this should be turned into a Linux or BSD community project instead.
It would be rather nice for the free/open operating systems to take a quantum leap beyond the traditional Unix feature set, and possibly solve or at least reduce the woes of the software engineering industry at the same time.
What's that tube of Superglue doing in the corner of the picture? ;-)
I'm glad that they replied "Yes" when asked whether they accepted that their approach had negative consequences as well as positive ones. That was honest and even-handed.
However, an analogy would have served them well. "Yes, our activities can have negative consequences. This is similar to the case of a kitchen knife manufacturer whose products can lead to domestic murder or to excellence in the kitchen. But you don't criminalize such a company for the negative use of its products, nor indeed do you praise it when you enjoy a well-prepared meal. The tool is neutral."
Likewise, a nuclear tipped missile can be used to deflect an Earth-destroying asteroid or to wipe out another country. The tool itself does not determine the morality of the people that use it.
Whoever moderated the head item in this subthread as off-topic would do well to reread the headline article: anything about the NYT is directly on-topic.
:-)
NYT was (for some reason) the direct subject of the item, and L0pht merely the object.
Hmmm, interesting.
:-)
But Terry Farrell is (or has the looks of) a standard Western Caucasian female. Where's the "ethnic" bit, outside of the Jadzia lineage that is?
Yes, their model parameters can't be sound. Real Lego is nowhere near that good.
However, that doesn't invalidate the work, as reducing bond strength could be expected to modify only local constructs, not progress in the direction of the distant goal.
Yes indeed, and we can see dead parts aplenty in the Lego bridge.
Nobody was talking about bird's eye views for "look and solve" though, but for guiding worm's eye view-based searching out of local minima that are not perceivable in the dimensions of the worm's eye search space. It's not a first-order constraint, just second order.
As to the value of a bird's eye view, it is subject to the same rule that governs the success of worm's eye views in searching: a bad evaluation heuristic produces bad results, by definition and in practice too.
What this means is that you have to get your heuristics right when producing your success metrics in both views, and if you don't then success is not likely, in either one.
Linked off the "1000 pentiums" story re Genetic Programming, the Salon article says:
.... :--)
Recently, researcher Lee Spector, an associate professor of computer science at Hampshire College in Massachusetts, used genetic programming to create a fast algorithm for evaluating data structures known as and/or trees. The GP-evolved algorithm was faster than any created by humans and led to the discovery of a new quantum rule.
If anyone thinks that Terminator/Matrix/Borg scenarios are totally unrealistic, they just don't understand the ability of GAs/GP to go beyond human potential. Our capabilities occupy just one local minimum in the solution space. AI machinery won't be constrained to that small valley of possibilities, as the research highlighted in Salon shows.
And on a lighter note:
The Faceprints project uses genetic programming to study human perceptions of physical beauty. Human test subjects are presented with several computer-generated images of faces. They vote on which faces are most attractive. The faces that the test subjects consider most attractive are crossbred, using GP techniques. As wave after wave of human test subjects rate the faces on attractiveness, a face evolves that matches most people's conception of beauty. The "most evolved face" in the Caucasian Female category is a green-eyed, Teutonic vixen who would not be out of place on "Baywatch."
Hmmm
Leading off a number of contributions in this thread, does anyone here working in GAs/GP today know of anything approximating a possible "information mutation theory"? Anything that relates to combination or transformation of information descriptors or of domain parameters would be of interest.
Cheers!
Actually, craw was right in part. The only reason why GAs don't usually get stuck in local minima is because of one of two things (or both): either the ancestors live in different valleys in the first place and therefore part of the genome will always have a worldspan greater than any local minimum, or else the sensitivity to mutations is set such that the GA can break out of the largest expected local minimum.
If the original ancestors come from the same valley, and there is no other mechanism for inserting external genes, and mutations do not create solutions outside of the parameters of the original genome then there is no possibility of escape from the local valley. The main reason why this is rare is that mutations are not usually limited in this way, which is why craw was more or less right.
If everyone does the same as the NYT and forces registration, we'll all have hundreds or thousands of registrations worldwide before long. The direction in which this is heading is completely untenable.
Somebody mirror the article for us, please, so that we can retain our sanity!
Although methods that don't get stuck in local minima despite using a worm's eye view of the world are valuable, the best answer is to use a bird's eye view instead so that local minima are directly observable for what they are. (The "view" is in solution space, ie. it doesn't necessarily have anything to do with the usual 4 dimensions.)
That's what humans do. For problem spaces a little smaller than we are, we actually supply the bird's eye view directly. For problem spaces much smaller or much larger than ourselves, we create mental models and plan our solutions within our mental bird's eye view of that model space.
Machinery can be made to do that too, although the overall problem belongs to Strong AI and is difficult (an understatement) in its most general form. There's no reason why this approach can't be used without much difficulty within small problem domains though, in which "full understanding" can be programmed quite simply using a good old-fashioned expert system.
Note that the bridge that was built (the end point of their problem) is merely the first point in the solution space.
If a *real* bridge was desired then the solution space could be populated more fully with many more GA-generated solutions, and then searched using conventional AI techniques for points satisfying a heuristic based on other criteria, eg. no excess material, smoothness of form, and so on.
Alternatively one could add these criteria to the GA survival metrics. This would slow down evolution dramatically because the probability of survival of offspring would be drastically reduced, but it would deliver "real" solutions directly.
Give computers a worldview and they'll see us as rivals.
We have to go there, but let's go prepared.
You think that it is understandable for the US [government] to want to defend itself FROM ITS OWN PEOPLE???? Because that's who they're targetting.
It's the ordinary citizen that is affected by crypto laws, not anyone else. Terrorists, drug syndicates and all the other organized baddies aren't affected at all. They have all the cryptographic (and other) weapons they need, thank you, because they don't operate within the framework of law so it's no more than an irritation to them.
In conntrast, the law-abiding citizen is affected 100%, so it *is* a big deal for him or her.
It's only pointless if the goal is the openly stated one. In contrast, if the actual goal is to snoop on the *real* threat to the political system, ie. the voting public, then crypto laws are far from pointless.
Now then, do you really think that the people in the NSA, CIA, FBI, etc, are utterly *stupid*? The likelihood of that is so close to zero as to be really zero. They are probably the most intelligent people in the government apparatus, full stop.
So, do you think that they really want to enact crypto laws for reasons that anyone with a single ticking brain cell knows are pointless?
While everyone seems to be focussed on cryptographic privacy as a means of safeguarding the rights of the individual against what could become a very threatening totalitarian (but still human) state, that's a relatively innocuous threat compared to what could be.
While it may not be tomorrow or the day after, we are going to be surrounded by AI machinery in due course. Part of that is going to be under our control, even within us, but most is going to be all-pervasive within the environment in which we live. The danger of distributed AI systems integrating into a whole and in self-defense taking a dislike to the rest is real.
We need universal crypto as a safeguard against that. Without secure communications, any dissent has no chance at all.
It's not *essential* to criticise the other faction when promoting one's own. It merely makes you feel better, for some odd human reason.
...
Try to be professionals, and make your public statements well considered.
It may sound like this makes your arguments less hard-hitting, but this is decidedly not so when viewed by everyone else: restraint makes a line of reasoning far more convincing than any statement containing negative criticism of any sort.
Whatever has gone on before, Bruce seems to be currently in restraint mode, but Eric not yet. I live forever hopeful
Is someone with ESR's phone number going to act on this?
PLEASE, just talk to him professionally and arrange a meeting. If he insists on the word *never* then I'll lose all respect for him entirely.
A person can't *really* love the community if one part of him or her is willing to ruin it for everyone by placing their own pride above all else.