Slashdot Mirror
Deriving Semantic Meaning From Google Results
prostoalex writes "New Scientist talks about Paul Vitanyi and Rudi Cilibrasi of the National Institute for Mathematics and Computer Science in Amsterdam and their work to extract meaning of words from Google's index. The pair demonstrates an unsupervised clustering algorithm, which 'distinguish between colours, numbers, different religions and Dutch painters based on the number of hits they return', according to New Scientist."
These kinds of articles never seem to get a very basic problem--natural languages. English is full of words that trip even humans. "Right" the direction versus "right" the judgement is a good example. In wartime something as simple as that may have lead to death. It's the elephant in the living room. Huge, important problem that nobody wants to talk about. There are alternatives, such as lojban which can be parsed like any computer program.
The article mentions English-Spanish translation. When one language is ambiguous (from a bit of Spanish I had in HS I'm guessing English is far more ambiguous), there is no hope of easy translation. And it's worse because the bigger application may be translating the many English pages (ambiguous) to Spanish.
Transcend Humanity. Please.
I though semantic meant "meaning".
If you tell someone to take the right path, they could mistake it as going the opposite of left and walking into a minefield, when in fact you meant going the correct direction.
Is this in any way related to the way that Google was able to decide all on it's own that Scientology was crap, and thus bring Operation Clambake up to the top of the search results? (Until they Scientology people got pissed, anyway.)
:)
Google is already starting to show signs of intelligence higher than some people.
"Everything you know is wrong. (And stupid.)"
Moderation Totals: Wrong=2, Stupid=3, Total=5.
"We attack the enemy on their left flank?"
"Right."
It turns out that the right flank was the dangerous one, and attacking on the left would've guaranteed a victory for the army that unfortunately spoke English.
This obviously has its failings, but theoretically, you could use a sufficiently large database of common human language coupled with simple algorithms to perform operations like grammar checking.
An internet search would not be quite so useful for that, but I would really be interested in what would be possible with full digital access to the library of congress. I would imagine you could do things like automatically generate books based on existing material.
When things get complex, multiply by the complex conjugate.
...as opposed to 'non-semantic meaning' or just 'semantic meaning' as in 'I don't know what semantic means but using it here will make me look intelligent'?
Doesn't it make you feel good to know that our freedoms are protected by politicans, lawyers and journalists.
This is basically what I was referring to in my response to "Using The Web For Linguistic Research" when I said:
followed by the explanation: andSeastead this.
If the context is not there then not even a human can make that distinction.
Once past that the natural language problem or elephant you speak of is really just a machine that define words in context.
Google is actually a pretty powerfull means for deriving context by some means of comparing word frequencies around various results.
I don't ask you to be me. I only ask you not expect me to be you.
After consulting with the elephant in my living room, I have only one thing to say. semantic Pronunciation Key (s-mntk) also semantical (-t-kl) adj. 1. Of or relating to meaning, especially meaning in language. 2. Of, relating to, or according to the science of semantics.
Warning: Could be fatal if taken seriously
This is a pretty nice approach. Quoted from the news article "The technique has managed to distinguish between colours, numbers, different religions and Dutch painters based on the number of hits they return, the researchers report in an online preprint.", it shows that common terminology can be drawn. In the end though, this is a refined search routine for Google IMHO. This would be good for scholar searches perhaps, or even a dynamic thesaurus. But when using terms such as: does windows use linux, the derived results would be broken down into: "linux" "windows" "use" . Google cached pages containing these terms vary so greatly in content. But, if searching for something along the lines of "dutch painters favorite colors", would produce desired results like the control method used in the news article
My Thoughts, Kyndig
Google has determined the meaning of life:
l e+Search
http://www.google.com/search?hl=en&q=42&btnG=Goog
the AC actually isn't far off.
In english what they are trying to do is define a word by context. For example go input a single word in google and you will get all the various contexts in which it is used.
Then by using some algo. ( serious academic handwaving here ) you place the meaning of word via context as determined for google. Thus effectively you create the potential for a program that could distinguish between there and their and do it across languages. It could also translate sayins like, the spirit is willing but the flesh is weak, dumb as a doorknob, I Rocked her World, Looney Tunes and Looney Toons.
The impressive part if done would be that you would not have to program in these various means of determining context as that could be gleaned via Google Results.
I don't ask you to be me. I only ask you not expect me to be you.
wow! = conveying amazement
WoW! = I can't believe I paid $50 for this crap and can't even logon!
Well, when I was in the army, it was very strict that whatever was said over a network DIDN'T have an ambigous meaning. That's why the army language sounds kinda weird at times, because you are not supposed to misunderstand anything.
My understanding is that it is a multiple query system.
Let's say you are just learning to ride a horse, and you want to know of positions to sit on the horse. You'd search for something like:
'good riding positions'
A current search return for this statement would deliver you everything from: Xaviers House Of Pleasures, to Yokels Horse Taming Ranch.
What this system does is refine your query for you, based off cached google pages, and using: page popularity and keyword algorithms.
The search would return results like:
"how to ride your woman"
"positions open on the dude ranch"
"popular riding positions for horses"
A tree is built with all these returns, and their system filters out the "junk", and returns the nearest pages you are looking for "popular riding positions for horses"
It's a smarter smart search engine (filter)
My Thoughts, Kyndig
First off, I am not an "AI" expert nor do I claim to be, however, this is how I see it.
...
Since it seems that so few really understand the term "intelligence", it is really not surprising that even fewer grasp the meaning of the term "artificial intelligence", is it?
One: intelligence is not awareness.
Although we cannot prove the existence of or even seem to really define self-awareness, it seems self-evident, at least to me, that intelligence is clearly defined and can be measured.
Therefore, I believe that we will have "artificial intelligence" soon, in fact, I'd bet Google may well be the first AI or "self intelligent' engine.
However, I suspect it will be quite awhile before we are mature enough to build a self-aware engine.
Lastly, in regards to some of the other comments, it seems to me that this paper is about using the "intelligence" included in the language we use, that Google crawls. This repository is the single largest collection of semantic weighting, therefore, algorithms could be developed that reflect this "intelligence", therefore appear themselves intelligent, even though they themselves are simply deterministic.
Whew
Words to men, as air to birds.
Like you never ask someone to "repeat" something ?
I want a new world. I think this one is broken.
W3C's OWL standard is a "language" to mark-up information to make it more meaningful to machines. Machines can draw conclusions to what a word means by context. So even if two words are the same, they may not mean the same and the computer can draw that conclusion based on the context. It's all a part of W3C's Semantic Web initiative. There is research dedicated to query languages for these kinds of files.
I will give you an example. If you search news (i.e., either Google News or Yahoo! News) for stories about the recent federal action (by Washington) involving Chinese companies and Iranians weapons improved by Chinese technology, you will discover that one of the popular news articles about this topic comes from the "New York Times". Several other newspapers redistributed the Times article, written by David Sanger (spelling?).
I read that article, but I also read articles from less popular Web news sites: e.g. "Taipei Times". The "Taipei Times" article does mention that a Taiwanese company was also implicated in the sale of weapons technology to Taiwan. Yet, "New York Times" article made no mention of this fact.
Is the "Taipei Times" telling the truth? It claims that Ecoma Enterprise Company, a Taiwanese company, was one of the culprits.
At this point, I fired up both Yahoo! Search and Google. Only on Google was I successful in locating the the ORIGINAL source of the information about American penalties against the 7 Chinese companies and the 1 Taiwanese company. The information is on page 133 of the "Federal Register" (volume 70, number 1). So, I discovered that the "Taipei Times" was telling the truth.
Guess how long I took on Google to find this information? 5 minutes. I kid you not. Even though I hate Google's employment practices, I am quite impressed with their technology.
Using Yahoo! Search, I was not able to locate the desired information.
Apparently, Google has an algorithm that, although it is unsupervised (i.e. without the kind of human interaction that corrupts Yahoo! Search), it captures the notion of what the typical person wants to find. The Google algorithm, dare I say "it", is on the verge of acquiring human sentience. THAT is, indeed, impressive.
Pray to Buddha that the middle name of the CEO is not "666" or Beelzebub. Just kidding.
Although very clever, NGD (Normalized Google Distance) misses alll higher-order relationships and does not even distinguish between different categories of pairwise relationships. For example, NGD might assume that "Bush" & "Iraq" had the same relationship as "Slashdot" & "Geek" because the two word pairs co-occur with similar frequencies.
More interesting are analyses on n-Tuples (co-occurences and orderings of n-words at a time). Anyone who does ER (Entity-Relationship) diagrams for relational databases will appreciate that many relationships involve multiple entities that are decomposable into pairwise relationships.
Another limit is that Google is atrocious on its estimates of the number of hits. The actual number of hits is only fraction (about 60%?) of the estimated from my experience. This suggests that Google has a pairwise estimator built in that may be only partially empirical. If Google simply reports an estimated number of hits based on products of probabilities, then their is no information about the pair in the NGD. Obviously, these scientists have gotten useful results, but NGD may not be as good an estimate of the co-occurence of the words as the scientists assume.
Two wrongs don't make a right, but three lefts do.
Google has determined the meaning of life: http://www.google.com/search?hl=en&q=42&btnG=Googl e+Search
a nswer+to+life%2C+the+universe%2C+and+everything%3F
I think you might've meant:
http://www.google.com/search?hl=en&q=what+is+the+
Is it my imagination or is this essentially a new flavor of Kabbalah with the same strengths and weaknesses?
Just need to fit that in googlefs to get better results on queries :)
Intelligence does, however, imply the ability to perform self-directed learning. Without that, all you have is preprogrammed behavior, which is not intelligence. Given the ability to learn, an intelligent entity is likely to draw conclusions about its own existence ("I think therefore I am"), and will thus essentially be self-aware.
Of course, the builders of an artificially intelligent machine might restrict its ability to gather facts about itself - it wouldn't necessarily have the ability to "see" its "body", for example - so this may limit the scope of the AI's self-awareness, at least at first. However, that's an artificially-imposed external constraint, which says nothing about the AI's ability or potential for self-awareness.
Sometimes in a dicucssion somebody might say
"I don't want to get into semantics".
I always want to yell - "why worry about the meaning of things - it'll just cloud things".
I wrote a program that gathered, analyzed and used word pair frequency data (various situational pairings). It needs more raw data, but shows a lot of promise. I opted to not use literature, as that often has archaic and purposefully awful word usage. Some of the issues involved include case, like Fall vs fall, I chose to ignore case, grammatical structure, needs to integrate with a grammar checker. Coupling this with a thesaurus is my eventual goal, this leads to some obvious difficulties, though it has potential rewards. I had considered google, and have run a few tests using it, but that solution was too simple, and not quite as powerful in the long run. Just had to share, sorry to waste your time.
The lore also contains an interesting anectode about the '92 riots in LA. Apparently a group of Marines were dispatched to assist the police. Two officers were approaching a house when someone opened up with a shotgun at them. One officer shouted "cover me" -- so the Marines proceeded to lay down covering fire on the house -- more than two hundred rounds were fired into that house.
I want a new world. I think this one is broken.
I'm a sci-fi vegan: I don't want the aliens to think we have as much right to live as the fried chickens we eat.
This is the mearning of this article for me.
Someone can explain it for me on a human language?
http://www.michel.eti.br
(whispering)
"!=" = NOT equal to
My company develops a data mining program for OS X (theConcept) that uses Google (or other search engines) to provide links to data for mining.
For example, searching on Google for "tom cruise" brings up pages upon pages of links, but -- from a cursory glance at the results -- it is impossible to learn anything about Tom Cruise unless one visits those results.
Our software visits each of those results (for example, the first 100) and looks for the most significant keywords and phrases used over all the data. As you might expect, these typically end up being the names of people (e.g. Nicole Kidman, Penelope Cruz) or movies (e.g. Top Gun, Color of Money) that are associated with Tom Cruise. As far as our software goes, this is ample for doing keyphrase analysis.
But the problem with deriving any additional meaning from the Internet web space is this: the biases that exist due to the very reasons for mentioning Tom Cruise (namely those things he is famous for) simply outweigh -- by a wide margin -- any other quite relevant interesting data about Tom Cruise. So, in fact, the web, in general, is an awful corpus of valid semantic data.
If you want a rough model of popular ideas then perhaps Google and the web en masse is useful (it is for our software). But if you want any real meaning at all you come to the same conclusion that has given rise to sites like Wiki: the web, to be blunt, has a whole lot of shit in it. Coming up with a perfect (and rational) filter is quite a task.
They are developing an open source tool http://complearn.sourceforge.net/ that will hopefully integrate the algorithm they describe. Right now it's only supporting one of their previous algorithms. More about this in the above link and section 5 of the paper.
I've perused the abstract and skimmed the body of the paper. They're fine. But the title is misleading: Automatic Meaning Discovery Using Google.
Their software has discovered meaning no more than paper has when the lexicographer is done writing her dictionary. Meaning is not the grouping of symbols.
For systems that step towards encoding meaning as human brains do, consider the Neural Theory of Language.
You're quite correct that cowboy-loose definitions of terms make this a very difficult discussion to have. For example, when you say "self awareness," it's unlikely that you actually mean "self awareness" in the literal sense; after all, if a computer is capable of detecting when its processor is overheating (and perhaps turn on a fan in response), it is basically "self aware," though we wouldn't confuse that with itelligence.
Rather, I think by "self awareness" here you mean, possessing narrativity; that is, the ability to construct a narrative of itself in relation to the things of which it is aware. In simpler words, consciousness. Now, it is possible to be intelligent without being conscious (everyone thinks they have the smartest dog in the world, but that doesn't make the poor beast conscious). But is it possible to be conscious without being intelligent?
Consciousness is fundamentally linguistic in origin (and I'm tired of arguing that point with people who haven't done a day of cognitive studies in their lives; there's no way around it: without language, consciousness does not evolve). So, for example, in the course of human evolution, first a linguistic parsing system was evolved, humans got language, and then, once this substrate was established, consciousness evolved as an epiphenomenon which rode on top of it. This substrate proved to be a fertile breeding ground on which memetic evolution could take place, as well, and since that is broader than any one particular human component in the system, it's almost more proper to say that we are the tools memes use to propogate, and not vice versa. (This argument is fairly well established with genes; same rules apply.)
So, any artificial system which contains "consciousness" will have to first handle language. If you don't have that linguistic substrate for narrativity and memetic evolution, there is nothing for consciousness to occur in. Maybe the information is there, but it would be like me pointing to an empty spot in the room and saying, "That's a balloon full of air; I just forgot the balloon." So, let's do this in the proper order: language first, then consciousness.
What he wants is more important that what I want. What he wants is also more important that what you want.
Thank you for the reply. I'm glad your work generalizes to longer search-term lists. Like so many other /. readers, I did not take the time to read your preprint before posting.
I've often wondered if one can use simple pair-wise distance estimates to reconstruct a polytope or distorted simplex for the set of items within a multidimensional space. In theory, an N-object system, with non-zero pairwise distances, requires (N-1) dimensions. But in practice, many real systems don't fill the space -- being M-dimensional (M less than N-1) and having only negligible (perhaps noise-induced) thickness in the other dimensions.
For semantic systems, the total number of semantic dimensions may be far less than the number semantic terms or tokens. A simple example dimensional flattening is the existence of synonyms -- the second word does not expand the space because it does not encode a new dimension of meaning. (Synonyms would also be negatively correlated in Google searches, but that's another issue). Also, the fact that each word can be defined in terms of other words suggests that the semantic nebula does not actually fill the space.
Accomplishing this would require a true distance metric. I notice that NGD does not satisfy the triangle inequality. Perhaps some minor transform or alternative formulation of NGD would yield a true metric.
The reason that estimating semantic dimensionality is useful is two fold. First, it says something about the cognitive complexity of humans and human systems. Second, it provides some insight into the required cognitive sophistication of autonomous learning systemd that need to interact "intelligently" with humans. How many words does a system need to truly understand to pass the Turing test?
Creating a full reconstruction, a more challenging task, would provide insight into the structure of human language and human language usage patterns. The dimensionality of clusters of words might provide insight into the complexity of subdomains of knowledge.
I wish you every success in creating better autonomous learning systems.
Two wrongs don't make a right, but three lefts do.
Good point. However it is difficult to value time in a single competitive metric whereas compression ratio (where the initial and compressed sizes include the size of the algorithm/knowledge of the AI) is a single number.
Perhaps the way around this is to have different prizes for different time classes, varying by an exponential. You'd have, say, 3 competitions with timeouts of one unit of time, 10 time units and 100 time units. This could make the contest run in a reasonable period of time at a reasonable cost.
Seastead this.
If given perfect information about the relationships between concepts, you could derive a very intellegent machine. TAke a human for example..
A baby hears the word mom spoken by his mom. Gradually, the baby knows there is a relationship between that sound and a smily face.
The child, growing up, starts to see relationships. Intense pain, which is rare, when correlated with a hot stove, has strong meaning in his mind.
Everything is learned initially through correlations. The advantage of human beings is that there are many more data points for correlation. Google's correlations are weak and don't give nearly as much information.
The NAVY would turn out the lights and lock the doors.
The ARMY would surround the building with defensive fortifications, tanks and concertina wire.
The MARINE CORPS would assault the building, using overlapping fields of fire from all appropriate points on the perimeter.
The AIR FORCE would take out a three-year lease with an option to buy the building.
I want a new world. I think this one is broken.
You, um, seem to be implying that blind people cannot use language.
You may want to seriously reconsider that.
Yes, when we use words, we're using a symbol to stand for the meaning. This is obvious.
However, we can in fact use words we don't understand, and we can extract some sense of understanding from the way it's used.
Consider the concept of, oh, "semantic". This has little-to-no basis in physical reality, and yet is perfectly understandable all around.
An AI, even without physical interactions, could understand such concepts.
And a physical interaction is a vague concept when it comes to an AI; George Lakoff suggests we understand things via metaphor of a very few basic concepts; the directions (up/down/left/right/forward/back) being one of them.
As a result, we sometimes understand the concept of "more" via analogy from "up".
Why could a computer not go the other way? It is not at all difficult for a computer to get the idea of more; why not have a vague idea of what "up" means to us from that?
Obviously this is beyond what is done now, but it does tend to imply that there are methods other than direct physical experience to produce an understanding of the same concepts we get from such experiences. And as AI is geared towards achieving that understanding, even if it would likely not be the most appropriate "intelligence" (such a VAGUE word, sometimes) for a computer, it's still something we will likely end up with.
Anyone whose driven a "radio taxi" would have to agree, but for less dramatic reasons :)
Don't know about the US but over here in Australia they dropped taxi dispatching by voice about 15yrs ago. Now instead of arguing with the dispatcher they listen to music....
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
"the ability to "see" its "body"" - Individual ant's do not learn, they are very much like small robots. An ant's nest on the other hand can display a modicum of intelligence in the way that it forages and protects itself.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Here's the original thread from June 2004: http://forums.searchenginewatch.com/showthread.php ?t=48
Here are the writings from Dr. Garcias own web site: http://www.miislita.com/semantics/c-index-1.html - see especially parts three and four.
Intelligence is not just knowing _absolute_ semantic meaning - e.g. that a cow is a cow and grass is grass. And being able to group grass with other grasses and cows with other bovine animals.
:)
It's being able to understand the statement that cow is to grass in a similar way that balleen whales are to krill.
And then now knowing something about krill from that even if you didn't know what krill was at all.
It's not just knowing the "absolute value" of the meaning - or even that these two objects are linked or close in the same area ( which seems to be the level which most current AI are at).
It's more like kind of knowing the "vector/direction" they are linked, and being able to organize other objects that are related in similar ways in a similar vector. Thus you can learn about things by analogies and metaphors AND even create new things with those.
Would pump out more BS but I have to go for dinner
Compression is a stricter test for AI than Turing
By stricter do you mean narrower and incomplete? Do you think that taking something overly terse and compressed and explaining it simply with examples and analogies etc is a greater intellectual acheivement?
Intelligence can be seen as the ability to take a sample of some space and generalize it to predict things
It would be myopic to see it as such. The ability to communicate an idea is a closer description of what it is to be intelligent as captured by the Turing test which rightly leaves the problem domain beyond this undefined. I know many intelligent people who are incontinently verbose and cannot summarise in a techie/scientific mode but can communicate a feeling or a subtle insight by complex layered descriptions. Compression and prediction might be an optional string of intelligence but by no means the whole instrument.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
That's so true. For (a somewhat related) example, yesterday I've listened to an mp3 recording of radio conversations of Moscow metro traffic controller when one of the machinists was incapacitated by a large dose of vodka. :) The machinist stopped the train at the station and left the train, only to be stopped by the police. Eventually when another machinist stopped on the opposite track at the opposite track at the same station, the traffic controller asked him to takeover the first train and park it in one of the station tonnels, then go back to his train and continue.
:) I imagine how bad it could get if they were on the battlefield.
Even though he was speaking quite clearly and in good Russian, even though the station-master went to the machinist to explain the situation in person, he was unable to comprehend what exactly he was asked to do.
Future Wiki -- If you don't think about the future, you cannot have one.
A slug is not conscious. Nothing without langauge is. Recommended reading: Dr. Daniel C. Dennett, Consciousness Explained and Darwin's Dangerous Idea. Richard Dawkins, The Extended Phenotype. Julian Jaynes, The Origin of Consciousness in the Breakdown of the Bicameral Mind.
Those are all more commercial works, well within the grasp of even people who've done no work in the field. For more sholarly and technical references, check their bibliographies, especially in Dennett.
What he wants is more important that what I want. What he wants is also more important that what you want.
"Sheesh" is a word which normally means, "I'm not very good at actually saying what I mean, so I'll just make strange noises and roll my eyes at someone who won't figure it out for me." (It's also the nick of one of my favorite Internet trolls; what ever happened to the good old days when trolls actually tried to be entertaining, instead of merely annoying?)
Anyway, okay, it's loose definitions of words that are once again getting us into trouble here. That a slug is aware of its environment, as in, capable of responding to environmental stimuli, okay, I'll give you that one. I won't gift wrap it for you, but I'll give it to you.
But that's entirely different from "consciousness" in the sense that we're discussing here. After all, even a computer is capable of detecting environmental stimuli, and responding to them, but as my colleague 2TecTom is pointing out in this same thread, the mere ability to respond to environmental stimuli is not synonymous with consciousness.
Read the source material. It'll give you the weapons you need to overcome your sheeshing.
What he wants is more important that what I want. What he wants is also more important that what you want.
my feeling is that what's often missed in the various AI language research programs is the problem of speech and reference. Any linguist working in the field will tell you that speech recognition is an incredible pain in the arse and then to layer semantic recognition on top of that is doubly painful. Though my real concern is about things like irony and sarcasm. I'm glad somebody stepped up to point out that different languages break concepts and the world up in different ways. But how exactly can you get AI to gather enough circumstantial data to understand something like sarcasm, where the connoted meaning is often different or the exact opposite of the denoted.
And what about signs that are coded to mean multiple things to multiple people simultaneously? For example, you're talking with your friend about this absolutely grotesque hat someone is wearing. You say to this person nice hat; he thanks you and your friend snickers. Same sign, two simultaneous meanings.