Slashdot Mirror
Semantic Web Getting Real
BlueSalamander writes "Tim O'Reilly just did an interview with Devin Wenig, the CEO-designate of Reuters. With no great enthusiasm I started to read yet another interview on how the semantic web was going to make everything great for everybody. Wenig made some good points about the end of the latency wars in news and the beginning of the battle for automatically detecting linkages and connections in the news. Smart news, not just fast news. Great stuff — but just more words? Nope — a little searching revealed that Reuters just opened access to their corporate semantic technology crown jewels. For free. For anyone. Their Calais API lets you turn unstructured text into a formal RDF graph in about one second. I ran about 5,000 documents through it and played with a subset of them in RDF-Gravity. The results were impressive overall. Is this the start of the semantic web getting real? When big names and big money start to act, not just talk, it may be time to pay attention. Semantic applications anyone? The foundation appears to be here."
Next up, semantic spam.
Actually, I think it's beaten the rest of the content to the punch. =(
What good are fancy links if the content still sucks?
I've never understood what the financial benefits for a site joining the semantic web are supposed to me. Reuters may be one thing, but how would you sell this technology to Amazon? Or NewEgg? If commercial sites can't/won't use it, how is it supposed to gain critical mass?
Comment of the year
So I need this WHY?
Most websites have little to say, and take all day to say it.
Having a detailed graphical analysis of the blather seems unlikely to improve the situation. GI,GO.
It would seem spending just a tad more time writing for HUMANS would be way more productive than writing for machines. Having a thousand computers watching your 100 monkeys seems unlikely to bring enlightenment or useful knowledge out of a pile of garbage and human blathering that passes for information on the web these days.
People used to write web pages.
Now they write software to write web pages.
Its not surprising they now need to write software to understand the web pages.
Whats the point?
Sig Battery depleted. Reverting to safe mode.
Semantic webs might be OK for small document sets where you can visualy search tags and click them. Want to look up something about monkeys? Look for the tag that says monkeys (or maybe find primates first, then monkeys) and click it.
But for huge data sets this sucks. After a smallish number of documents & subjects it must be far easier to type monkeys in search box and have Google etc do the search.
This might work for handling some queries, but will suck supremely for complex queries over large data sets (eg. the whole www).
Engineering is the art of compromise.
Semantic Web Getting Real
Just what we need. Yet another version of RealPlayer.
The higher the technology, the sharper that two-edged sword.
Yes -- essentially.
And the only reason we moved from Web 1.0 to web 2.0, and the only reason we need to move from Web 2.0 to Web 3.0 is...
We are still stuck on Search 1.0
Well, ok, to be fair to Google -- Search 1.5
Sorry, but we won't see much improvement in utility until someone rolls out Search 2.0. That is a product LONG overdue.
"Wenig made some good points about the end of the latency wars..."
/.'s 'editorial' habits :\
Mr. Wenig must not be all that familiar with
I read it like this:
Semantic web getting real [player]
and immediately thought "it was bad enough when the original web got it"
"Please note this environment may not be completely safe, so we are going to prevent you from entering. We have also initiated so many system processes that it will simulate a virus on this system."
The links in that article are neat. I am looking forward to watching the maturity of this!
blah, search is great and all, but that shouldn't really be the ultimate purpose of the Semantic Web.
Asking a question and getting a sensible answer, that's the killer app.
How we know is more important than what we know.
If you are like me, and have absolutely positively no dang fucking clue what the summary is talking about: http://en.wikipedia.org/wiki/Semantic_Web
According to the Wikipedia history, this concept has been around since at least 2001.
Write your own Choose Your Own Adventure. http://www.freegameengines.org/gamebook-engine/
IMHO this is not the semantic web. The primary representation is still (just) natural language. Anything in addition to that is really just search engine technology under a different banner. Is that a bad thing? No! I've always said the semantic web was bound to fail because people don't want to spend a lot of extra effort tagging their information so others can slice and dice it; instead, the evolution of natural language processing in search (rather than manual tagging) will solve the problem. Maybe the Reuters idea of exposing the "inferred" metadata will be useful (as opposed to normal searches like google who simply keep this metadata in their own indices), though as yet I don't see why.
When you start aggregating as much text as google does, the semantics just starts popping out, in the form of word relationship statistics.
The massive corpus size, when measured carefully, acts to filter semantic signal from expressive difference "noise".
Combine that kind of latent semantic analysis of global human text with conceptual knowledge representation and inference
technologies (which would use a combination of higher-order logic, bayesian probability, etc) and it should be possible to
create a software program that could start to get a basic semantic understanding of documents and document relationships
in the ordinary "dumb" web.
Could the proponents of the semantic web please tell me what it will add to this?
My basic proposition is that if an averagely intelligent human can infer the semantic essence (the gist, shall we say), of
individual documents, and relationships between documents on the web, why can't we build AI software that does
the same thing, and then reports its results out to people who ask.
Where are we going and why are we in a handbasket?
If by that you mean "a collection of buzz-words that everyone uses without having Clue 1 what the hell they're talking about," yes.
Les Miserables Volume 1 now up with my reading of
Finally, Reuters released OpenCalais as free open-source software. OpenDover will appear any time soon. (someone may then connect both using a Channel, SSH perhaps)
Actually, the story is about a tool which does (a part of) what you are describing.
On first read, I like what they are trying to do, but I see so many problems with what they are thinking, and I am not a web designer in any sense.
First, I don't have a problem finding things to buy on the internet. The problem is, signal to noise ratio. There are TOO MANY google results for something like 'plasma tv.' No matter what kind of RDF is used, it will be abused by people who want their URL to show up in your search for whatever reason. I think someone touched on this earlier a little in this thread, but it deserves repeating.
Second, can you imagine a scenario where, say, best buy or fry's uses some 'semantic web' application to do real time web searchable updates of their inventory? That's what would have to happen for this to work, and do something that isn't already possible.
Right now, I can search for 'plasma tv' in google or ebay. Then I can call my local retailers to see if they carry that item, and have it in stock. In order for this system to make any kind of tangible change in the example given, retail chains would have to update their inventories online, whenever a purchase is made, or new items delivered to the store.
It's an interesting idea. I wonder if the retailers would go for it? All it means for them is fewer people comming into their stores...sounds like that would hurt sales.
I also hate internet hype. I really fouls things up, more than some want to acknowledge. I try to keep my 64 year old dad educated enough to buy coffee beans on ebay, check email, look at news, etc. Every time he sees 'symantic web' or 'web 2.0' in the media, it just confuses him, and I imagine, people like him who just use the net for basics like online bill pay, ebay, etc. He doesn't need a new buzzword to motivated to shop online or whatever.
he has the motivation already...silly contrived 'new meida' buzzwords just waste time and confuse people
Thank you Dave Raggett
I noticed that too... I was looking at the tags to provide an example of what machine-created tagging has to go up against to beat human tagging for a rant up above. I guess I have to thank that idiot for proving my point. Humans do hostile tags, they haven't yet written a subroutine to make a machine act like a jerk.
Why should I be thankful about spending my adult life working because machines aren't up to the task? I'll be thankful when machines take the work and leave us free to do what we want.
Also, it's not yet for "anyone." According to the Calais roadmap, only English documents are accepted: "Calais R3 [July 2008] begins
668.5
I understand being jaded about internet hype and buzzwords but I'm still surprised that after nearly eighty comments there doesn't seem to be anyone who has anything to say other than "vaporware" and "it won't work because of the spammers." Yes, maybe it has been overhyped and yes it is taking a while for the envisioned ideas to come to fruition but that doesn't mean that those ideas aren't worthwhile.
I'll use the following example because I recently had to do this with non semantic tools. Lets say you wanted to see how good or bad a job a transit agency is doing in its city in comparison to other similar cities. A couple of metrics you might use to find similar cities would be population size, population density and land area. Google doesn't do a good job with something like that. You end up needing to search for cities individually and then finding their data points. Or you can find a list of cities ranked by population or population density. If you search on Google for something like that you end up at one of the Wikipedia lists. These lists are helpful but....still lacking. They don't contain all the cities you need or they don't provide a way to look at multiple data sets at the same time. The lists are also compiled by hand and aren't automatically updated when the information on the city page is changed. The data is in wikipedia though. Every city page lists that information in a little box near the start of the article. But how do I take this data that is in Wikipedia from the form that its in into a form that I can use to find what I need to know? Enter the semantic web.
Lets say that wikipedia, or at least the parts dealing with geography, were semantic. Now, there are tens of thousands of pages describing countries, regions, states, counties, parishes, cities, towns and villages. Then those pages are translated into many other languages. Some of the data that these pages contain is of the same type . They all contain the name of the locality, latitude, longitude, size, population size and elevation. For data such as this it would be pretty easy to have a form to enter the data into as opposed using the usual markup and the form could put the data into the proper markup for the page and the proper RDF. Once the data is in proper RDF form it would be easy to automate the process of updating translations of that page with the new data as well as updating any pertinent lists. It would also make it easier for people who want to analyze or use the data because they would be able to access it much more easily.
But nobody really wants machine readable access to this information, you might say, except for the random geek and researcher. I would disagree. Lets say you're using a program like Marble which is similar to Google Earth in some ways but is completely open source. If they wanted to display the population of a city when you hover over it they would currently have to create and maintain their own dataset or they'd have to write a parser to extract it from wikipedia. Neither of those options is particularly easy at the moment but if the information was in semantic form on wikipedia it would be a piece of cake.
The strength of the semantic web isn't, in my opinion, going to be AI like personal agents or anything like that. It'll be things that in many ways are already here. Like Yelp putting geotags on the restaurants they reviews and apps like Google Earth taking that data thats available in machine readable (Semantic!) for to overlay that data on a map so that you can see whats nearby. It'll be applications doing the same with the geotags from flickr. Its really useful mashups like http://www.housingmaps.com/. Its the transit agency putting realtime bus data up in semantic form so you can see on your iphones google map how far away the bus is. So yeah, maybe the semantic web is overhyped but that doesn't mean there isn't a lot of substance there, too.
Cheers,
Greg
There's no more "intelligence" in AI than in a can of Campbell soup. It's basically statistics, linear algebra and (sometimes) handcoded rules for reasoning. It doesn't evolve. It doesn't build upon what it "knows". It has no self-awareness or consciousness and its reasoning capabilities, if present, are extremely weak compared to even children.
We're so early in the development of this field that no one can even define what "self awareness" or "consciousness" really is, let alone how to create it or scale it. Folks try. There's Cycorp, there's Powerset, there are a lot of people in academia who work on NLP, Machine Vision, classification, neuroscience, etc. There is, however, no unifying vision or theory/understanding what is it we're trying to build, and the current methods have nothing in common with "intelligence" per se. They do learn, in a sense that they figure out the hidden structure of a given set of data by approximating it using a mathematical model. Even though this model sometimes closely matches what a human brain does (e.g. in multilayer neural nets), they don't come anywhere close to what one would call "intelligence". What they lack is scale (and speed), and advanced cognitive mechanisms required to become self-learning.
It's also interesting to note, that at this point humans know on a high level how their brain works. Neocortex is a six layer neural net with links going cross-layer and neurons organized into columns. Trouble is, there's hundred billion neurons. We sorta know how vision works, too. Trouble is, we can't work with it in real time (because, naturally, you'd need a chunk of those hundred billion neurons). Heck, even human language is a pain in the ass if you don't have advanced cognition (AKA strong AI), with ability to understand euphemisms, sarcasm and idioms, paraphrase, generalize and specialize. Heck, even anaphora resolution is not solved yet (i.e. what does he/she/it in the current sentence refer to in the previous text). It's as if you had a bunch of parts and no manual and someone asked you to assemble a spaceship out of what you have, warning you that some parts are broken and may require you to make your own replacements. Without blueprints. Blindfolded. With your hands tied behind your back.
I do believe that in 50 years we will have strong AI, though. I work in a science lab, however, and many researchers don't share my optimism.
The company I work for, Garlik has two products that are run off semantic web technology. DataPatrol (for pay) and QDOS (free, in beta).
We use RDF stores instead of databases in some places as they are very good at representing graph structures, which are a real pain to real with in SQL. You often hear the "what can RDF do that SQL can't" type arguments, which are all just nonsense. What can SQL do that a field database, or a bunch of flat files can't? It's all about what you can do easily enough that you will be bothered to do it.
A fully normalised SQL database has many of the attributes of an RDF store, but
a) when was the last time you saw one in production use?
b) how much of a pain was it to write big queries with outer joins?
RDF + SPARQL makes that kind of thing trivial, and has other fringe side benefits (better standardisation, data portability) that you don't get with SQL.
I guess it shouldn't be a surprise to see the comments consisting of the usual round of more-or-less irrelevant jokes and snide commentary - this is Slashdot after all - but I can't help responding.
timeOday >>> "evolution of natural language processing in search (rather than manual tagging) will solve the problem"
..." anyway.
But then if you're creating an addon for joomla (or any template elements really) to display event listings why not add a semantic tag so that a search engine could limit the domain by "tag:events". The extra effort involved is pretty minimal, especially when, if you code well, each event is probably in a "<div class="event eventtype">
Once people realise that search engines can do semantic filtering then it will be worth it.
As for tag-spamming well surely google, et al., won't accept based on tag first but will do their usual contextual/ quantative analyses first and then limit based on tags. So we wouldn't be gaining any spam over what we have now?
I try to keep my 64 year old dad educated enough to buy coffee beans on ebay, check email, look at news, etc. Every time he sees 'symantic web' or 'web 2.0' in the media, it just confuses him, and I imagine, people like him who just use the net for basics like online bill pay, ebay, etc.
I'm afraid whenever I see this argument I immediately tend to discredit all the rest that I've read in that post. Designing technology for those who are least able to uptake it is a losing proposition at best; at worst a total disaster. Technology has always been utilized by those less set in their ways first, less invested in the capital and experience of doing it the 'old way', and is only more broadly adopted once it proves out as a better way to do things. Universal acceptance tends to only come after a generation; when those who are poorly situated to utilize it have passed on.
This speaks to your other concern rather tellingly. Fry's may not put their inventory online. But if Best Buy does, and reaps more rewards, then you can bet eventually all companies will do this as standard practice. Far more likely - a company that is smaller and more mobile will do it first, and then get bought out by a larger company that will adopt it's practices in order to stay potent in a changing marketplace.
But the successful online inventory app is not going to design for Best Buy first. They're going to design for Mom and Pop shop, and scale up to whatever customer they can find. When it proves out or doesn't there will be tangible evidence for others to act on - rather than meaningless hype.
Finally, I think the thing that the semantic web provides is more of the ability of the end user to control results. As we perfect our ability to parse machine language, we perfect our ability to hear clear signal amongst all the noise. I look forward to the day when we have this technology in more than a nascent stage, and think it's silly to dismiss it before then.
Also, I look forward to the day when people stop designing for me. Because presumably I'll be happy with what I have!
[Ego]out