Slashdot Mirror
Autonomic Computing
pvcpie writes: "The New York Times has a story today about Autonomic Computing, which is described as "a biological metaphor suggesting a systemic approach to attaining a higher level of automation in computing;" and they published a paper (pdf) on the topic. Apparently there are already some universities signed up on Autonomic Computing projects, more info was available on the website and in the nyt article. It also appeared in CNET."
IBM has done quite a bit of research on autonomic computing. IBM Just keeps getting cooler and cooler IMO. Although I have to say that this wquote frightens me:
"Civilization advances by extending the number of important operations which we can perform without thinking about them." - Alfred North Whitehead"
Anyways, here is the link to their Autonomic Computing R&D site:
http://www.research.ibm.com/autonomic/
Execute? [Y/N] _
Can you say "SysAd"?
The more that can be done automatically, the more of the IT staff's precious time can be dedicated to more complex tuning tasks, and/or new development. This will make IT more effective, not obsolete.
Using the human body as an analogous self-maintaining system, would there be a coresponding lifespan to such a system? A point that is reached when the body can no longer repair and regulate itself and it simply quits.
For that matter would there be analogous doctors, hospitals and life support systems? How about gymnasiums for keeping in shape? (and I ask that last one only half-jokingly...)
Gordon
What do you think you are doing, Dave?
It reboots itself.
Look out, guys, they're performing a paradigm shift!
While I think that designing system components with feedback capabilities is a neat idea, remember that doing this in a safe way requires actual computer science. Or at least extensive modeling. It's cool, sure, but it's going to be a while.
Oh, go on, check out my job.
My high school biology teacher must've said a thousand times, "Evolution proceeds towards what works, not towards what is best."
The body works so well because it's highly highly highly redundant at the cellular level, not because there is a brilliant master control program controlling the most efficient implementation. You can't even imagine a number as big as the number of hormone receptors in your body.
That kind of duplication in a computer system costs real money, and while a noble goal, people don't spend money on reliable systems, they buy Windows.
This is a terribly useful approach in the battlefield, and the right thing to do once bandwidth and computational power are practically unlimited, but we're still in the stage of computing where people just want more features, reliability be dammed. After all, Nimda follows an autonomic behavior.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I would worry about these systems if I had one. I for one enjoy the fine-grained control I have over systems I manage.
I would hate to see my web server decide to bump up the number of allowed simultaneous connections in response to a denial of service attack, or decide that the ogg encoder in the background is indeed more important than domain control services.
... and of course the manditory gripe - that my system decides that it doesn't like my pirated MP3s and deletes them automatically.
If computers become smarter than the people who design their software, how are they any use as a tool anymore?
I think you read into that one a little wrong.
They're saying that they want the computers to handle the mundane tasks so that the IT people can have more time for the more important tasks, like the person-to-person tasks. Do you know any IT worker who isn't overworked? This is designed to help alleviate their stress and give them more time for the more important things in life.
At least in theory =)
If God gave us curiosity
In the end it turns out that the most complex problem arise in trying to coordinate a collection of "autonomic" (?) components. Distributed systems with unrully objects... This is what the autonomous agent community is mainly concerned with ( see the UMBC agents page or this very useful overview paper for example).
Of course IBM pushing this it might mean a kick up rear for the academic to actually get some of this potentially cool stuff working. Chances are you never want the end user to know how it works anyway.
.sig
This means that we would need modular units in a network, say, that would be autonomic. The desktop PCs would have to be autonomic before you could bet the network to truly be so.
It would be a whole new way of computer design for software, and I doubts that some of the OSs out there would have code bases that could be viable in this regard.
Note that you can do this sort of thing as an optical illusion. You can pretend that everything is doing all right, when in fact it is going to hell in a hand basket. The vaporware diagnostic that merely pretends everything is all right, or the repair that cause more damage than was present in the first place.
But I think we have had enough of that over the past decade or two to know to avoid it. And, of course, the guilty have not been named because everyone knows who they are already.
"It is a greater offense to steal men's labor, than their clothes"
A program/database is only only as good as the knowledge of the person who has input and wrote it. If a PC thinks that to wipe it's hard drive is the best way to fix itself, I'd blame the author who wrote the maintenance program. No expert would wipe a hard drive as a general fix for a PC, and would not write these instructions into the PC.
In my opinion, part of a autonomous PC is to be self-sufficient, not act like a lemming and follow other PC's just to follow.
Plus, just as human have a basic survial instinct to survive, I think you'd write this instinct into the PC as well and not have it destroy itself (unless it was doing major harm to its master, etc. Remember Asimov's rules for robots).
Finally, I agree humans will never be replaced as the final decision maker in fixing and running PC's, servers, networks, etc., but when I was a sys admin, I'd have killed for PC's to be smart enough to do some of the basic, mundane, man-hour, labourious tasks such as upgrading Service Packs if I told them all to do it, install programs, etc. Then I could have done more fun stuff. Plus when I had to fix a problem, people weren't glad to see me, because I was only there when something went wrong. Granted they were happy someone was there to fix the problem, but all would have preferred that there was no problem in the first place (PC fixed itself).
--- RFC 1149 Compliant.
"Open the pod door, Hal" ...
utter rubbish
sorry for the run-on, but that was my immediate thought when i read Paul Horn's statement that the creation of "computer systems and software that can respond to changes in the digital environment, so the systems can adapt, heal themselves and protect themselves" is the only thing which will reduce the need for "constant human maintenance, fixing and debugging of computer systems." freeing humans for higher pursuits sounds good, but is probably only likely in a utopia. Horn goes on to say "The only way to get efficiency gains in information technology is to take some of the people out." This trend sounds like the steel industry - we'll have more cost efficient processes in providing IT services, but all those educated in that field will end up working at mc donald's.
so what happens when we all forget exactly how this "autonomic software" regulates itself? i guess this is the final word in proving the importance of documentation! : ) ** begging for a flamebait mod**    or we could skip the documentation and just kidnap the children of visiting alien starships when we eventually start dying of radiation poisoning from our super-self-configuring systems.
the infamous penn state stalker server!
p.l.u.r.
Of when Crab and Tortise were involved in a phonograph player duel, where Crab was on a quest to find a phonograph upon which any sounds could be played, and Tortise kept making "This cannot be played on Phonograph X" records.
"The only way to get efficiency gains in information technology is to take some of the people out."
They're called managers.
this says one thing to me
ka-ching!![$$]
lots of overtime
-- www.globaltics.net
Political discussion for a new world
Yeah...but does the University know about it? :-)
This is not to be confused with anatomic computing, which is an entirely different concept.
You can accomplish anything you set your mind to. The impossible just takes a little longer.
Hmmm... a computer that's supposed to run perfectly all the time and doesn't let the user get in and mess with anything. It sounds vaguely familiar, like a fleet of old Macs booting up at the same time. One thing I know, don't make the power switch software-dependent, or we'll never be able to turn the bastards off without unplugging them... or throwing them out of the window.
The problems I discovered were:
* The building blocks of the software itself, are human optimized algorithms and datastructures;
* In order to improve human optimized algorithmns (meta-optimization?), one could develop some form of trial and error optimization algorithmn, but this would complicate things even more (it's hard to determine whether the searchdirection makes any sense); designing such algorithmn is very hard, because, how long do we search before we give up? This is like the chess game, certain move may look silly in the first place, but it could be a very good move in the end...
* If the program is to optimize smart, it will need to use *known* optimizations, and be unable to improve human optimized algorithmns... Introducing the factor of meta-optimization gives the problem of CPU-time distribution: how much CPU time may content optimization take, and how much time may met-optimization take??
* If only known algorithmns are used, the program is bound to a limited level of complexities. Meaning that: lot's of human comprehension has high complexity, which is yet not very well understood by science; the "Perfect Human Interface" is likely to fail in this area - it's the area the user (again) needs to adapt to the machine.
But if these guys actually succeed in their quest... brilliant!!
Bizar technology?
Anyway, the idea of Autonomic Computing is hardly new (consider plug-and-play and autoinstallers). The really, really hard part of it is to impose autonomic computing on a system that was not designed for it. It is very difficult to make a complex system "simple" without redesigning the complex system.
I think most of you, so far, are missing the idea here. I also think the good Dr. from IBM is too, but that is beside the point. The point here in redesigning the way systems work from the ground up is to make them more capable of doing what YOU as users/admins actually want them to do. The idea being that YOU set the policy and the computer learns how best to implement it.
I, personally, don't like this very much. It sounds like the next step in closing off the workings of the "operating system" from the user. What happens to Linux and open source when Windows starts to dynamically rearrange it's code to optimize for your preferences and specific uses? It gets left behind is what.
I've been thinking about where operating systems are headed and what I want in an operating system, lately. I had pretty much defined what I wanted, when I started to run across projects like this: TUNES, and ideas like this: Flow-Based Programming. I then realized that I wasn't entirely original. People have been thinking about the same things and trying to work them out for some time. But there has been little mainstream work done to get things to happen.
In my opinion, the design of TUNES and the ideas expressed about Flow-Based programming are a perfect fit for open source programming. And, there's no reason that autonomic computing couldn't fit right into the mix as well, as long as it's an open-source feature rather than a built in proprietary unified piece of the system.
The new system I'd like to see would be completely dynamically restructurable, and reprogrammable from the ground up. I think this would be a prerequisite for full-blown autonomic computing, but I have a feeling that the corporates are going to slip it into Windows in such a way that Windows stays the same on the surface, but just tells you less and makes more decisions for you than it already does. Problem is, that's what most users think they want. What I suggest is doing it in such a way that each user has total choice about how his system is designed and operated. Of course there would be predefined templates for certain types of systems (web servers, web/e-mail clients, gaming system, desktop publishing workstation, etc). So a user could pick one or more open source templates on which to base his system and then modify it to his needs as he goes. These templates would define what optimum scheduling and resource allocation should be done for specific tasks and merge this at the lower level with the needs of other tasks and the priorities set by the user or learned dynamically by the system.
I think we'll see some very interesting advances in the next 10-15 years. Let's hope the open-source community doesn't miss the boat. Microsoft sure as hell won't.
This is some very cool stuff. Of course, I am reminded of a joke related to this:
Joe, the world's leading Cyberneticist boards a plane bound for Athens, Greece. This flight is the maiden flight of American Airline's first totally automated flight.
As he walks to his seat, he is greeted by a slick looking robotic flight attendant of his design. After sitting down, another attendant of the same design brings him a scotch and water (just the way he likes it) and says in a tinny voice "Good Morning Dr. Davidson, I hope you enjoy the flight."
Settling back in his seat with his drink in hand, he thinks about the many thousands of hours he has put into the autonomic systems that entirely control this plane.
As he goes to give his empty glass back to the robotic attendant, the plane pushes back from the gate. After a short while, he hears a much smoother robotic voice come over the intercom; "Good Morning, ladies and gentlemen and welcome to American Airlines Flight 1644 from Los Angeles to Boston. This will be a 2 1/2 hour flight. We at American Arlines would like to take a moment and point out that this is the first trip made completely under the control of the latest IBM 36000 Autonomic Robotic Piloting Computer. Every aspect of this flight, from the attendants serving you drinks to myself, the pilot, have been developed with safety in mind."
As the flight trundles down the runway, picking up speed, the voice continues on; "So you can sit back, relax and enjoy the flight, secure in the knowledge that absolutely nothing can go wrong, go wrong, go wrong, go wrong......"
Seriously, we're not very far from this. Flights routinely take off and land with only the most minor human intervention, and cars are being developed which use visual cues to pilot themselves down the road (a company in Australia has converted a Humvee for a test bed).
The Dopester
"Yes, I'm a Karma Whore, but I'm doing it to pay my way through school."
> Virus writer have a lot to teach us about self-maintaining and
> -tuning programs - while despising the destruction they cause, I can't help but
> admire their design prowess.
It seems, you never looked at the code of virus programs. They are not self-maintaining or seld-tuning. Most of the time, they are even written very badly and tend to crash in unknown environments. But for a virus, that doesn't matter, as long as it performs well enough in a commmon environment. The author usually doesn't care it it crashes the current host, ruins it or just doesn't work if the virus gets confused. If a virus happen to find enough friendly computing environments, that offer it the exact condition it needs, we'll hear about it on CNN. If not, it just won't spread.
All in all, calling a virus self-tuning or self-maintaining is utter crap, usually the kind found in many articles about artifical life by Katz.
In reading this article I'm beinning to wonder if AI is going to arrive not in the form of software implementation, but in the form of a computer that's "autonomic". Think about it, the article discusses a computer that is "self healing" with no need for human maintinence. I would imagine such a system would also need limited regenitive capabilities in case hardware was damaged on a physical level, this might fall under the bracket of self replication. Think about the amount of information these machines would be handling and "autonomicly" distributing themselves.
Even limited self replication coupled with the ability to process information so rapidly and powerfully seems like borderline sentience to me. What happens when you attempt to replace an autonomic router and the computer as a whole deceides thats really something you shouldn't do, because the router is so useful. Can this be coded around or avoided altogether? The people that develop this technology are going to have to be weary of creating something that cares more about its own processes than the user trying to make use of them.
Abandon All Hope, Ye Who Enter Here
Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any--no inventions that will do for software productivity, reliablity, and simplicity what electronics, transistors and large-scale integration did for computer hardware. We cannot expect ever to see two-fold gains every two years.
It is curious that the paper mentioned in the article does not deal explicitly with Brooks' objections since they are the best known statements of the problem of complexity in software.
Among Brooks solution for reducing complexity is to use great design: Whereas the difference between poor conceptual designs and good ones may lie in the soundness of design method, the difference between good designs and great ones surely does not. Great designs come from great designers. Software construction is a creative process. Sound methodology can empower and liberate the creative mind; it cannot enflame or inspire the drudge.
I would also add that simplicity engenders complexity. Simplified systems become subsystems for more complex designs. When the complexity of a system becomes a barrier to its further enhancement, simplifying it only allows its complexity to continue to increase.
It is the objectives of the system that creates complexity, not the development techniques. The only way ultimately to reduce complexity is to artificially constrain the requirements.
Two possibilities rear their ugly heads here:
1) An autonomic virus, written with the capability to "heal itself" once installed. Does this make sense? It seems to me that some existing virii already have some self-healing properties, such as those that hide a copy of themselves on a user's HD and insert a registry key in a Winodws registry to have themselves restored at reboot time. Thoughts?
2) A virus designed to insert itself into an autonomic system would conceivably be able to use the system's "self-healing" properties to protect itself (a funny memory springs to mind. I went to remove Outlook Express from my Win2K box at work, and discovered that Win2K does not have the option to uninstall Outlook Express. Undaunted, I went into the folder the executable was in and deleted it. Within five seconds, the system detected my "user error" in deleting a system file, and restored it. It took me a while to figure out how to prevent this, but it really threw me for a loop when I first saw it happen).
"Can't you see that everyone is buying station wagons?"
Every so often what is a reasonable, modestly interesting idea gets high high-jacked by the snake oil peddlers in whose hands it becomes the solution to the world problems.
The popular press quickly grasps on to it, as these magic bullets increase the circulation of OMNI and Scientific American. Eventually the politicians hear about them and allot untold amounts of money to these efforts.
After 5-10 years nothing much comes out of this, and the snake oil peddlers move on to another area.
Among the thusly overinflated areas we have:
- AI
- neural networks
- expert systems
- nanotechnology
- chaos theory
- e-commerce
- parallel computing
- distributed computing
- complexity (a la Santa Fe Institute) theory
- logic programming
the latest two additions are
- the semantic web
- autonomic systems
/.ers are well advised to apply a healthy dose
of skepticism to any such magic bullet claim.
While the idea has the power of lucidity, it's not clear whether it can be implemented effectively. The development of autonomic responses in complex animals developed over millions of years using sexual reproduction as a means of pruning less viable branches while introducing sufficient variation to ensure species-level survival.
At the organism level, autonomy is accomplished through a combination of neural, hormonal, and physiological responses to external stimuli and internal state changes. How many proteins and hormones, etc are required for this? How many combinations of signals are there? Clearly the number of combinations is large enough to be considered "countless". That an organism as complex as a human being works at all is an impressive feat of biochemical integration and regulation.
"...we don't really need... sentient machines and androids programmed to love and laugh -- to overcome the largest obstacle standing in our way."
That's an assumption which remains to be validated! How does IBM know that it's not the other way around? Perhaps love and laughter (i.e. higher emotions) are a natural and inevitable byproduct of the ultimate expression of the ideas of autonomy. Put another way, it may be possible to provide some sort of low-level homeostasis without emotions, but the maximum expression of those concepts might lead to a deeper philosophical awareness. At that point, look for IBM's business systems to call in sick with "mental health days" once in a while.
Richard Powers' novel Galatea 2.2 was an interesting examination of the relationship between self-awareness, emotion, and intelligence.
That was in the original Apple "Inside Macintosh".
The hardware side is reasonably close on this. All the newer interfaces (USB, IEEE-1394, PCI, PCMCIA) have identity info on all devices. And it's been that way for a few years now. It's time to pull the plug on the old stuff and insist that everything autoconfigure.
This is key. And again, Apple almost had it right, once. The original Apple model was that the system had two main repositories of system state - the Desktop file, and application preferences. The Desktop file could be regenerated if needed (and had to be, due to lousy database design), and application preferences were cosmetic only - you could delete preferences at any time, and just went back to the defaults.
Apple never faced up to checkability, though. And it hurt them, because they were running an unprotected OS with a tendency to trash its internal data structures.
Broken things must not contaminate other things.
It's unacceptable to ever get bad data from a disk. Reported errors, yes; undetected errors, no. Everything must have error checking. Memory parity must always be on. (And ECC ought to be standard.)
Only 14 amino acids form the basis for tens of thousands of proteins and billions of combinations of genes. Those amino acids combine with one another in exactly three ways (covalence, hydrogen bonds, Van Der Waal bonds) to form those proteins. Maybe this is an example of what you mean by "Simplicity engenders complexity."
Of course, there are other important chemicals in the body (both organic and inorganic) -- lipids, carbohydrates, salts, etc. But the "simplicity" of the protein "alphabet" is a starting point.
The first is, will IT workers embrace this? I know quite a few Oracle DBAs who are none to happy about turning over the optimization of their database to the computer. Maybe these people are just afraid of change and having to learn a higher art than what they've got, but if they tell their manager that they can do much better than the computer can, who will the manager believe? For this to progress, you must find a way to not scare the geeks.
Secondly, how do we accomplish this without advancing machine tecnology too far? If a machine becomes self aware and protective of itself, what happens when we want to shut it down? What are you doing Dave? I know there are ways of preventing this, but will they work, and will we be able to find out if they work before it is too late, so to speak. I'm not trying to be paranoid, but this is something that is a real concern.
Another piece of this that someoneelse mentioned is if the computer is maintaning the basic stuff, what happens when the computer dies and no one knows exactly how it did what it did? A very real example is the ubiquitus (sp?) of calculators. How many of you can still do long division in your head? There was some story I read in High School where this guy who could do simple math without a computer was such an oddity that he became a king or something like that.
Keep doing those math problems.
THIS SPACE FOR RENT
Actually, it's all supply and demand. Programming, engineering, and medical are common fields yet the pay is good.
Art is a fairly rare field, and the pay generally sucks. Acting, music too. Sucks, unless you bedome popular, then you become a natural monopoly on "you" and your pay skyrockets.
I am for the complete Trantorization of Earth.
It tells your heart how fast to beat, checks your blood sugar and oxygen levels, and controls your pupils so the right amount of light reaches your
There's an OO principle called dimeter which advocates as few dependancies as possible between objects. This sounds like a lot of hooks all over the place which is not a model of simplicity. It would be better for "it" to step out of the way and let each object adjust itself based on its surroundings just as in natural systems. Nature has a tremendous advantage over computers. It is far more efficient because everything is happening literally in parallel. Computers can really only do a very limited number of things at a time although sometimes the user perceives concurrencey due to very rapid time-slicing.
As a result, programmers are forced to make tremendous compermises given the comparatively limited medium with which they have to work. It will take well established techniques and objective analysis to determine the be way to utilize bits on silica.
Over the years I have recognised one principle that transcends this issue -- the issue of dealing with complexity. Oversimply it is Recursive Composition. This "pattern" or OO construct as it is sometimes referred to does not have a Class or particular set of relationships between objects. It's completely arbitrary. The idea, is to recursively delegate the responsibilty of another part of the system to yet another module. At the leaves of this tree you have the primative operations and at the root you have one simple instruction for triggering a potentially very complex cascade of instructions. Thus you have reduced the complexity of the overall system. The key difference between this and just another group of functions calling one another (and thus target to reduceing complexity of programs and in real-life systems) is parameterization.
As a simple example, imagine trying to encode or decode a database file. The database file has a header, a record list, and data chunks. Like this one on PalmOS PDB files. If one were to apply the principle of Recursive Composition the API for this PDB codec would be, at the top level, PDB_decode(char *src). At the next level down you have operations like Hdr_decode(char *src) and Record_decode(char *src). At the leaves you have dec_uint32be(char *src) to decode an unsigned 32 bit integer in Big Endian byte order.
If you can parameterize cleaning exactly what is required to perfrom a task and delegate it to another module you have broken the problem into at least two smaller problems which reduces the order of complexity. Simple!
Homeostatsis and self-regulation are not properties that you implement once in some abstract data type and that henceforth works for everything, or that require breakthrough new technology, they are design goals that you need to take into account when you design each and every part of a system. Biological organisms have been forced from day one to deal with these issues. The reason real software systems don't do this is not that people don't know how to, it's that software developers don't bother and aren't trained to do it, and that they can get away with it because there are always smart humans around to help it.
So, next time you write a new piece of software, think about how you can make it more self adapting and less reliant on numerous environment variables and other arguments supplied by the user. The pathsearch library is a simple example of this.
More networking + more devices + more subtle, automatic behaviour of devices - human oversight = many more security holes.
Quattuor res in hoc mundo sanctae sunt: libri, liberi, libertas et liberalitas.
The article lists 8 characteristics of autonomic systems:
- To be autonomic, a computing system needs to "know itself " and comprise components that also possess a system identity.
- An autonomic computing system must configure and reconfigure itself under varying and unpredictable conditions.
- An autonomic computing system never settles for the status quo -- it always looks for ways to optimize its workings.
- An autonomic computing system must perform something akin to healing -- it must be able to recover from routine and extraordinary events that might cause some of its parts to malfunction.
- A virtual world is no less dangerous than the physical one, so an autonomc computing system must be an expert in self-protection.
- An autonomic computing system knows its environment and the context surrounding its activity, and acts accordingly.
- An autonomic computing system cannot exist in an hermetic environment.
- Perhaps, most critical for the user, an autonomic computing system will anticipate the optimized resources needed while keeping its complexity hidden.
The features indicate a system several orders of magnitude more complex than the one it is intended to correct. Given the fact that these autonomic systems are supposed to deal with the shortage of IT professionals, where are the IT professionals supposed to come from that will implement these systems? Are they solving the problem of complexity or just creating more complex systems to maintain?When they develop an autonomic programing language it will be time to give it some serious consideration.
Sounds great! Count me in!
Of course, the Morlocks will get hungry from time to time...
- - - -
The real Tetsujin 28 is a giant robot.
Gene Roddenbery (spelling?) came up with fully integrated autonomic computing with hardware (ie., the Enterprise) ages ago, best expressed for the first time in The Next Generation. Plainspeak computing, graphical/physical interfaces, monitoring systems paramters, generating data and executing instructions at voice-command, the whole scha-bang, all on the bridge of the Enterprise. Of course this is where computing is going. The fine writers at IBM should watch more TV. I am not sure that this is where computing SHOULD go as the paper contends; it puts that much more power in the hands of people wishing to commandeer a business or endeavour by way of its computer control. Doubtless security measures will improve but there is ALWAYS a way to adapt to an obstacle; we see this in nature as we do in computing. Overcoming obstacles is why computers exist in the first place.
wakka smakka
When I wrote a B-Tree, as an improvement on a Insertion sort, I wondered whether I could think of an algorithm, fed with meta-semantics on the insertion sort, and designing the B-Tree as a evolutionary successor. It took me quite some time to understand the algorithm myself, and implementing this was even harder - semantic comprehension of the building blocks "in english" resulted in many more lines of C++ code. Because of the recursion, the B-Tree code needed, except for the actual B-Tree code, the following elements:
* execution environmental code - organisation and anticipation of the recursive characteristics of the algorithmn; for example a messaging system to tell the calling function what we'd expect from it...
* datastructure environmental code - understanding the benefits of using multiple data structures to be able to perform the algoritmn; but this would be related to the B+ tree even more; using 1 data structure (BTree) was harder than using 2 data structures (B+Tree)...
* state determination code - needing to know where we stand and what to do when a (recrsive) function call returns; for the trees: do we need to rebalance?
* borderline code - dealing with those borderline cases; what if I'm in a left node, having no left sibbling, while the right sibbling is not an option?
Stepping from the B-Tree to the B+Tree, the data structures are quite simular to those used in the B-Tree, slight differences make the B+Tree algorithmn easier to implement, and better in throughput for table-indexing.
Ofcourse, this is just a simple example where we could try to think of a meta-optimizer. One can extrapolate this to R-Trees and X-Trees, still, B, B+ R and X trees won't have to be the panacea to organize complex information, perhaps a computed "tree" or indexing structure would even perform better in that case. Oracle's database (and others) are optimized to detect their environment and to act as effectively as possible, although their algorithmns are basically human optimized. Even using "Fuzzy Logic" to determine the best balance between the tasks of caching, defragmenting, reorganizing (ie hashtables and 'clusters'), the main way to store the data is predetermined. In pure (non ORDBMS) SQL based systems, the IT-specialist has a hard time designing flexible datastructures with hierarchy; one meets that point-of-no-return very quickly. So in a system which has the elements of organizing hierarchy and repeatance, the perfect storing algorithm should be computed, perhaps going beyond human comprehension (I agree with you).
Another problem with meta-optimization is that the system must try to model the way the user thinks - which is the only way for a system to act whether it understands the user. If you've ever studied the discipline of "Neuro Linguistic Programming" (I believe they call these guys 'motivation-guru's' in english), then you understand that everyone has personal sub-concious truths. A schema of reasoning is very personal, because every individual has learnt different things. The point is, that the system will be able to adapt to one person only would that be userfriendly? The system wouldn't be very suitable for public use.
Still, as I said, I'm playing the mind game, and if someone is to overcome these problems... brilliant!
Bizar technology?