Slashdot Mirror
Patent Invention Machines
kryzx writes: "Here's one to tickle your imagination: using genetic programming to come up with new, patentable solutions to problems. Could be happening very soon. Here's an article
at MIT Technology Review. This work, being done at Stanford
and Genetic Programming Inc. by
John Koza and company has already succeeded at reproducing quite a few ideas for existing patents, ranging from old to very recent. It's apparently much easier to compare against existing patents than sift through hundreds of surviving algorithms to determine if they are useful, original, and patentable.) Also, this company is a good target for your tech envy, with their 1,000-node Beowulf-style
cluster
of Pentium II 350's and 70-node cluster of 533 MHz DEC Alpha's. (There are pix, too. PII cluster on the main page, Alphas here.) Wanna play with the toys? They have
job openings for programmers. :-)"
I feel the reason for running this project is wrong. "Let's start a project so we can register loads of patents" should instead be "Let's start a project to benefit humanity"
Omigosh! You mean... the guys who run this experiment are... ALIENS?!
In all seriousness, some of the *worst* disasters have been brought about by people trying to "benefit humanity". Crack-dealer infested housing projects, Eugenics etc...
That's not to say that capitalists don't screw up too, but I maintain a healthy skepticism towards altruists. So these guys want to patent stuff. Big deal. At least we know where we stand with them. When the idealists get a hold of things, you never know how they are going to f*** you.
This reminds me of teachers in school. There were many different kinds of teachers, but I'm thinking of two specific types. The first was the stern old man with the ruler. On the first day of class you thought it was going to be miserable, but then you got to know him, learned a lot, and by the last day you missed him. The second was the hippy teacher with the tie-die and the long hair. On the first day, you thought it would be an easy class, but then he had to lay down the law and it turned out that he wasn't very competent either.
Such is the way with capitalists and idealists.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
However, the evolved solutions were limited in their scope. If the chip created through natural selection was tested in an environment of a moderately different humidity or temperature, it's tricks wouldn't work. Just like real-life animals, their niche tricks worked only in a niche environment. In a way, it's all a problem of chaos theory -- sensitivity to initial conditions.
I expect these evolved patents will face similar difficulties. Yes, they'll work great, but only in the specific environment or parameters for which the software originally was programmed. Change things just a little bit, and it has to start all over again. Which makes for great demonstrations of the software, but poor practial, patentable solutions that people will buy.
The idealists may not start out wanting to f*** you, but all too often they enter what I like to call the "dictator's delusion".
It works something like this:
Step 1. "The World would be great if everybody (blank)ed". This is the essence of idealism.
Step 2. A few people will (blank) but not everybody. Laws are passed to encourage people to (blank).
Step 3. Encouraging people to (blank) becomes a proxy for making the world a better place.
Step 4. The delusion is complete. The leaders of the revolution totally forget about making the world a better place, and cling stubbornly to the ideal that (blank) will do that. They forget the fundamental laws like "love thy neighbor" and cling to their preconceived notions of what is best, even when it is demonstrably flawed. Two famous examples: The Pharisees in the Bible and the leaders of any "communist" nation refusing to reform while the proletariat starves. Both groups started out with an ideal. In the first case it was religious, and in the second place political. The result is the same though.
I hope you get this. I can't tell you how many times I've frustrated people who try to pigeon-hole me into either a "leftist" or "rightest" category. Both are "ideals" and flawed because There is no algorithm for right living.
I think I probably summed it up best in this rather off-the-wall essay.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
There's no point in questioning authority if you aren't going to listen to the answers.
I recall reading an interesting article from FeedMag (now on ice) about a "simple" GA that produced an amazingly fast sort algorithm, but that no human could decipher how it worked. Call me crazy, but that's an example of something that simply should not be patentable.
At least I can rest easy in the knowledge that nanotech and AI-assisted design/engineering will probably negate the need for patents a few decades down the road anyway. Who needs a monopoly on an idea -- of human orgin or not -- once the rat race has ended and everyone is able to live like a king? (answer: only the regressively selfish--but that's a whole other rant.)
Power to the Peaceful
I'd rather keep my soul.
honestly, does it matter anyway? They are giving away patents left and right. Other countries don't accept them. They are basically worthless.
I say we need to do some work and improve the system before we start expoiting it.
This article must be one of the more fascinating I have read the last month. This might be due to my over worked imagination, but this seems like the idea of making computers do the work for us is taken to the next level. (oh long sentence)
One thing I am agains though are all those (b)anal patents around. Americans seem to be the worst, but Europe is getting there.
Why are they making the computers invent things to patent? I feel the reason for running this project is wrong. "Let's start a project so we can register loads of patents" should instead be "Let's start a project to benefit humanity"
Find nice cocktail recipes @ www.spitzy.net
It's about time the patent office had a DOS attack.
Donate background CPU time to fight cancer.
That's the real test.
The real question is whether they're going to try to pantent the idea of running a computer program to generate patentable ideas. That meta-patent would be the really valuable idea to come from this research.
There's no point in questioning authority if you aren't going to listen to the answers.
But can they make a machine that'll come up with creative ways to train your wife for you?
Koza has a (controversial) patent on genetic programming.
The owner of a small consulting firm I used to work for patented a method of software assembly using genetic algorithms about two decades ago
It'd be in the public domain by now. The term for the patent monopoly was 17 years after the patent is granted or 20 years after it's filed; those are about the same because it typically takes 3 years to approve a patent.
Just thank goodness Sonny Bono[?] never touched patents.
Will I retire or break 10K?
If a computer can 'invent' something by simply extrapolating rules, the devices it produces are clearly not innovative and fail 'Obviousness' test.
Maybe they could contest pending patent applications on grounds of obviousness, based on the inventions the computer managed to duplicate. Trouble is, their investors probably wouldn't support such an agenda.
Or, as the article puts it:
"I imagine we have done that but we don't know it," laughs Koza. To identify valuable, original results rather than simply matching patents, he explains, a human expert in the given field would need to evaluate tens of thousands of survivors.
In other words, it turns out that an infinite number of monkeys really will stumble into everything given infinite time. Frequently, solutions are simple -- it's identifying the need and fitting a solution to it that' s worthy of a patent.
I always thought that for something to be patented it had to be non obvious, and the product of creative thought.
Non-obvious, yes. More precisely unobvious to one with ordinary skill in the art.
Product of creative thought? No. Aside from the difficulty of coming up with a definition of something so metaphysical as 'creative thought', (an who can say this computer was not in fact engaging in such) there is specific language in the patent statutes that says the method of coming up with the patent is not to be considered when judging if the material is patentable. In other words, it's the result that counts, not how you got there. Sensibly pragmatic IMHO.
If you want more information on genetic programming, check out these articles posted on slashdot a couple days ago.
The fact that a machine can come up with patentable software only goes to show how banal most software patents are.
BH
Fools! They laughed at me at the Sorbonne...!
Okay, listen. The Genetic Algorythm is NOT merely the act of analysing absolutely everything that is possible. That is not feasable. It is not iteration, either. The GA is a process that sets very specific 'fitness' functions and models a limited set of variables that can be changed randomly in order to work towards the highest fitness possible. Each genetic (should be called evolutionary, but it seems that I have already lost that battle) program is usually tailored for a specific problem. If you think that you can write a program that models absolutely every possible invention, then go ahead and try. I have been working on a limited algorythm as a hobby project, and a decent fitness function is a very difficult creature to track down.
Everything above may well be poorly-thought out / spelled. Blame the beer, not me.
One of the things that was not mentioned in the article and as far as I know never made it into Koza's writing is that for some of their older stuff, they came up with some particular patent so regularly that they started to write a patent avoidance filter.
What the thing does it take some existing patents and when the evolution hits on something that is close to it, the solution get punished for that. In think this is on par with patent invention, particularly as it seems you can circumvent any patent by just entering it in the machine and end up with a different device that circumvents it. Might be good idea for some GNU projects to have.
true, thats a better way, relize i was just trying to show an example of how you couldn't use GP for something.
maybe text to speach isn't a good example.
thanks,
-Jon
this is my sig.
I agree. A decade ago, I remarked in Koza's lunch discussion group that we're missing something in understanding how evolution works, and there's a Nobel Prize for whomever finds it. This sparked some interest, since the people in attendance were academic researchers. But a decade later, there's been no big insight.
Observing that most of the variety in biological evolution appeared during a short (in evolutionary terms) period, I have a speculation of my own. Perhaps during the period after viruses emerged, but before self/nonself recognizing immune systems, evolution worked much faster. With viruses to carry information around, some forms of evolution could take place during the organism's lifetime. This is vaguely plausible, but unconfirmed.
It's very frustrating trying to do AI via hill-climbing techniques. Almost invariably, you knock off some of the easy problems, then hit a wall. Your algorithm churns and churns but doesn't improve. Searching on a broad front, or introducing randomness, helps a little, but mostly just gets you out of local minima.
I tell people working on broad-front hill-climbing methods like genetic algorithms to validate them by comparing them against the two extremes - simple greedy hill climbing, and totally random search. If simple greedy hill-climbing works, you didn't need a GA or a neural net in the first place. If totally random search works, the search space is so small you don't need anything fancy.
So if a computer is able to determine the algorithm, How can it be argued that this is not something that someone trained in the art would not come up with?
If someone could program a computer to iterate over possible solutions, this SHOULD NOT meet the requirements for Patentability.
Looking for any old 8-bit Heathkit/Zenith software/hardware - http://heathkit.garlanger.com
Let's face it, if you can't put chains on it, if you can't put a fence around it, or lock it up in some manner, it does not belong to you. Software, ideas, drawings, music, plans, it makes no difference. Once you release it, it becomes like the air that we breathe. It belongs to nobody and to everybody.
The only way that IP laws will be enforceable in the age of the internet is to institute Orwellian laws whereby the governement is given fascist powers to spy on everybody. Already, hundreds of millions of copies of copyrighted software are being used freely around the world and there is nothing the manufacturers can do about it, short of instituting full blown fascism.
Wait a minute, aren't they doing that already? Aren't all ISPs in the US and Europe already keeping a log of all user activities? A government that finds it necessary to spy on its own people, not only does not deserve to last, but cannot possibly last. A house divided and all that...
Hit them where it hurt the most, the pocket book. It's our money that is being used to enact stupid laws like the DMCA. Without money, they have no power. There is only one solution against fascist IP laws: Download it all and copy it all! And use it all for free! Don't give them your money so they can turn around and use it against you.
On the philosophical ground, these Stanford folks seem to have created a device capable of sifting the Akashic data banks. After all, where do these ideas come from? But like any chaotic concept, we reform it along the lines we expect to see.
So we have a machine that we can give a problem, and it will give us an answer, as long as we know what we want to hear.
This sounds an awful lot like most human-made software development (especially "community" development) in that it's really good at optimization, but it slows down significantly once new ground begins to be charted.
It of a conversation I recently had about this very idea- that GNU projects (GNOME was the example) tend to have this habit of just mulling around and looking funny for a long while before jelling into something usable- but once that happens, the thing created is a war-hardened program capable of getting the job done. Who knows, though, if it would have even existed had it not had previous environments' headlights to chase?
There's definitely a parallel here worthy of more observation.
What we call folk wisdom is often no more than a kind of expedient stupidity.-Edward Abbey
I thought I did a fair job expositing the flaws of both capitalists and idealists, the implication being that they will both f*** you, just in different ways and at different times. How is that idealist?
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
There's a class of problems for which such algorithms work. And there are problems for which it doesn't do much. Koza's main contribution has been to find more useful problems for which this approach works. Analog circuit design is a good choice, because tweaking on circuit parameters and connections works in that domain.
Koza's system works on some of the same types of problems as Lenat's Eurisko, from 1978. Eurisko was a search system that worked on LISP S-expressions. It was used for simple program creation, digital circuit design, and trouncing humans in the Traveller strategy game. The basic concept was to take a representation of the solution, apply various plausible operations to it, and see what made things better. Many of the same concepts recur in genetic programming, although the search strategy is very different. Eurisko used heuristics thought to be clever. Genetic programming just bashes on the problem with compute power. (That's why Eurisko worked on a single time-shared DEC-10 and genetic programming needs a Beowulf cluster.)
It's worth noting that the re-invention of early electronic circuits is easy today because you can now use a simulator (typically SPICE) to test them. This makes automated brute-force searches possible.
It's not clear that this approach leads to strong AI. But it's a big hammer that can definitely crack some problems.
Suppose we had a GNU project to do something similar; and published each and every solution which resulted - distributing this effort among thousands of GNU advocates.
These solutions could be used to blunt future patents based upon the resulting "prior art" from this effort. And software would be free to progress as it once did.
Anything that this array of GP computers can create by a process of natural selection must by definition be an obvious consequence of the basic building blocks. After all, no creative thought it being exercised here, just a purely mechanical process.
If so, those patents must have been granted in error (not an uncommon thing for the patent system), and so should be revoked immediately.
That goes for anything else that can be evolved in a similar way.
Maybe we have an automatic way of weeding the many bad patents out of the system. I wonder if the powers that be will be interested ...
Bahh, I just woke up and it's to early for me to post.
About 2 years ago I started getting interested in genetic programming, all one had to do was define the problem well enough give the computers the right pieces and boom you have the next perfect speech to text engine, or Pac-man ghost logic.
The first think you'll realize when you get into this field is that there are two type of genetic programming; "Genetic Algorithms" and "Genetic Programming". The first is really just defining an answer set, like trying to re-arrange a DNA strand so it fits a certain criteria, the latter is actually having the computer create some "code" or an actual algorithm. Naturally I was more interested in "Genetic Programming".
The first thing I set out my test program to do was find an algorithm to find prime numbers, just a little test before I moved on to bigger and better things (like Pac-man ghost logic). I plugged in a few operations for the GP to work with, Add, Sub, Mul, Dev, Mod etc.. and gave it a fitness saying "your fit if you have an algorithm that given N you return Nth prime number". Well this didn't work.
The GP would come up with the most retarded algorithms; long lines of Add/Sub etc.. Some of them worked moderately well, all could have been reduced to a more finite set. Either way it wasn't going to work. The problem was that I needed to give the GP better tools then basic arithmetic, it needed for loops, if branches, variables! Maybe if I could figure out how to give the GP the right set of tools, and defined the problem we'll enough it could have gotten further.
You'll find GP successful in a limited range of problems. This is defined by the tools you can give them to work with, and by how well you can grade its "fitness. Fitness becomes very limited when the user most be evolved, imagine trying to have a GP come up with a good text to speech, ever time a new "generation" of code was produced you could have to personally grade the static garble it came out with.
-Jon
this is my sig.
im actually more interesting in the solutions they are coming up with rather than if the results are patentable. this does bring up quite a few interesting AI problems (solutions?) on making computers duplicate (or perhaps surpass) human problem-solving methods (kind of reminds me of wargames).
--Ks9