Slashdot Mirror
An App to Boil Down Online User Reviews
An anonymous reader writes "Is this a glimpse at the future of the Semantic Web? A new startup named Pluribo has developed a technology that can auto-summarize user reviews on the internet. It is a Firefox extension that can take a webpage filled with reviews and condense it down into a couple of sentences. Currently, it just works with Amazon electronics, but the potential seems incredible. Ars Technica took an in-depth look."
Somebody fix this so it work's on /. Maybe then I'll RTFA.
I for one welcome our hot grits pouring overlords.
Imagine a Beowulf cluster of Al Gores in Soviet Russia releasing Duke Nukem Forever, you insensitive clod!
In Korea, only old Natalie Portman must be new here.
All your Linux are belong to us Sharks with frickin' laser beams attached to our heads.
1)Stephen King is dead
2)BSD is dying
3)Profit!
There. Fixed that for you.
Enter> The Wicker Man, 2006
Result: "Sucks monkey balls"
Hey, it really does work!
Can we apply this to Slashdot comments? Please? Please? Pretty Please??
I'd love to get a one page summary of all the informative, insightful and interesting comments.
The application seems to assume that the best summary is the one with the most correlation to the other posts, in other words: the most common viewpoint. While that may work fine for user reviews, in most cases the viewpoint of the masses is usually not the best.
I approvied the Pluribo extension for being pushed to public. All I can say is that it gets most of its data from the Pluribo server, and does very little client-cide besides display the data. At the moment it is *extremely* limited, literally to a handful of products. Apparently their system doesn't do the scanning of comments automatically otherwise it would work for everything, whereas it actually has to query the Pluribo server to get results.
== Jez ==
Do you miss Firefox? Try Pale Moon.
...is there any way to have it filter out the obvious astroturfers and trolls?
Seriously, any big-name product or service will have a coterie of fanboys (or paid astroturfers) who will praise something no matter what, and a flock of trolls who will point out everything wrong with it, no matter what.
Do that, and it'd be one hell of an advancement in filtering. :)
Quo usque tandem abutere, Nimbus, patientia nostra?
This is a serious reflection of our current times, where people's eyes gloss over if the concept at hand is not condensed into a convenient sound-byte.
I suppose you could call it the bleeding edge where complacency meets the loss of freedom and the fall of darkness where critical thought once stood.
Now there is enough probable demand to launch a startup designed to remove what minimal labor people are interested in dedicating to the quality of even their leisure time.
I'm sure many fantasize about strangling people this lazy/complacent, but honestly if they're unconscious enough not to care about their own toys, do they really possess a "life" for you to take from them?
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
can someone boil it down to a couple sentences for me?
I Heart Sorting Networks
It's heavily templatized generation of language based on the automatically extracted sentiment data. The important difference here is that the language of the summary does not include phrases from the original user reviews. While this is a new twist on the old problem, automatic extraction of evaluation criteria and sentiment analysis in product reviews are not new. Heck, even Microsoft has a working system for that (electronics only):
http://search.live.com/products/?q=nuvi%20350%20GPS%20-%20Asian%20American%20(City%2FVehicle%2C%203.5%22%20LCD)&p1=%5BCommerceService+scenario%3D%22reviews%22+docid%3D%222BECBBF6F17C98618C2E%22+p%3D%2220df8fe62a9b4e9490993ff7b91032af%22%5D&wf=Commerce&FORM=ENCA
See the bars on the left, and be sure to click through to the individual sentences. It's spooky how accurate that thing seems to be.
The problem with all these systems is that they're heavily domain dependent. You will use different language to write a review of a book than for kitchen appliance. In fact, you may even use different language from different kinds of books or different kinds of kitchen appliances. Worse yet, some things are notoriously difficult to accurately measure sentiment on. Once innuendo and sarcasm become frequent, all hope is lost - you need strong AI to figure that out.
This is not to say these systems are useless - to the contrary, they are very useful in their respective domains. This is just to say that the only new thing I see here is the generated blurb.
Built into Mac OS X is a Summarize Service (click on the application's menu and point to "Services" and choose Summarize) that is pretty cool. Unfortunately, Firefox was never coded to work with that OS X native service, so I had to copy and paste the text of that Ars Technica article into Text Edit and then use that service. But here is the resulting summary (I had to do as short a summary as possible):
Dude, even back in history the sound bite, even if it makes no sense nowadays ("Tippecanoe and Tyler Too!") has been prevalent.
Also, "our current times" include things that simply didn't exist before: near-universal literacy, 24/7/365 media, and a desire to do something with one's time off other than simply sit around in a half-drunken stupor somewhere while catching up from the work-week's exhaustion. The latter part was pretty much what the vast majority of humanity did with what little leisure time they got.
Quo usque tandem abutere, Nimbus, patientia nostra?
This sounds like the opposite of how to get useful information out of reviews, and more like the "consumer products" equivalent to the automatic resume scanner.
You know the resume scanners I'm talking about -- the ones that circular-file the candidate who took three years off work to get his Ph.D. in cognitive science (and whose thesis is a perfect fit for your business plan), preferring, instead, the guy who listed "20 years PROLOG, PL/1, BASIC, C, 10 years C++, 5 years Java, MCSE, A+", because obviously the second guy triggers more buzzwords. Because the HR drone won't understand any of the resumes, he/she just picks whichever one the scanner selects, and that's typically the one with the fewest career gaps and the most buzzwords. ("But that other Ph.D guy only has one or two languages, this guy has six! And that Ph.D guy's been out of work for three years, so obviously nobody would hire him!")
Ten reviews reading "Works. Fast, cheap, lightweight" and three reviews reading "Doesn't work" don't tell me anything, other than that the product might have reliability issues.
One review reading "Didn't work the first time. The manual doesn't mention that you have to make sure the jumper is in the correct position first, and then it works. I own an XYZ-123 and this new product was at least as fast, but at about half the price. Weighs about a pound." tells me everything I need to know -- that the three people who claimed it didn't work almost certainly didn't know how to configure it correctly, and that the first seven reviewers never had a problem because they weren't part of the edge case.
Do you mean 24/7/52? Or perhaps 60/24/365? But 24/7/365 seems a bit redundant?!?
I bet I made a mistake in this post, tis the law of slashdot!
The idea behind the Semantic Web is that content providers tag content with semantic information to allow the creation of an ontology so that programs can easily use the for reasoning. (see http://en.wikipedia.org/wiki/Semantic_Web) The is an example of text summarization which is a classic natural language processing (NLP) task.
I'm concerned how well the applet properly discerns the meaning of words in context. For example, just because I mention that a product is 'a portable laptop' does not mean I am impressed with it's size or weight... it's just the category the product falls in. But judging from the screenshots in the article, this exact error was committed by the plugin.
Reading natural language is hard, and I'm of the opinion that a Firefox plugin just won't cut it for understanding the nuanced opinions given by reviewers.
"I will trust Google to 'do no evil' until the founders no longer run it." Hello Alphabet.
Astroturf is repetitive, so it will get listed in one place. You won't be able to eliminate the talking points but you won't have to read them 50 times to find one genuine opinion. Both will be weighted as original content. You will then have to use your brain as you do now. At the same time, you can eliminate distractions like "hot grits". It took me a while to realize this but now I see that it is true.
Could you boil it down to a word?
Unfortunately, the product will never be able to filter out useless spam reviews that artificially inflate product ratings, such as "OMFG THIS IS THE BEST!!!!" (entire review) (especially for products that aren't even released yet) or paid site-fillers like "this is good I'm so glad [this site] made it available".
I expect most media product reviews will summarize to something like "This is the best ever, if that's your kind of thing."
Comment removed based on user account deletion
Most of us have already figured this out, but apparently some still haven't...
The way to properly get information from online reviews is to sort from lowest rating to highest, skip the trolls, and read until you get to the fanbois.
This document explains how to use the Summarize services available in Mac OS X applications.
If you have a long document, you can use the Summarize service to get a summary of the contents. For example, use this to get a short version of a long page on a Web site.
To get a summary of a document, select the text and choose Services from the application's menu, then choose Summarize.
If the application you are using doesn't support services, copy the text to a TextEdit document to get a summary.
Note: The information is this document comes from Mac OS X Help, the help system included with your computer. It is based on Mac OS X 10.1.2. If a different version is installed on your computer, choose Mac Help from the Help menu. Updated and expanded information may also be available in other Knowledge Base documents.
http://docs.info.apple.com/article.html?artnum=61336
(but i know this feature was in OS 9 or earlier)
Know what I like about atheists? I've yet to meet one that believes God is on their side.
Would be very nice if it could process the reviews on Newegg. Some items have tons of reviews.
So, use this plugin to get your choices from 50 items down to 5-10, and actually read the reviews for those 5-10 items. The best of both worlds.
Not a Twitter sockpuppet... but I wish I was.
By people who think it's slimy for a program to manually scan a lot of pages you aren't going to read anyway.
If Google really cared they would fix Android Chrome to reflow text, instead of discriminating
(Score:2) by plasmacutter (901737) This is a serious reflection of our current times, where people's eyes gloss over if the concept at hand is not condensed into a convenient sound-byte.I suppose you could call it the bleeding edge where complacency meets the loss of freedom and the fall of darkness where critical thought once stood.Now there is enough probable demand to launch a startup designed to remove what minimal labor people are interested in dedicating to the quality of even their leisure time.I'm sure many fantasize about strangling 24/7/365??
(This Slashdot Page)
that do essentially the same thing with text summary (similar to Mac OS X's summarizer). There's an Open Source project that does this too called Summarizer - http://sourceforge.net/projects/summarizer
I run a site that catalogs free books, and accepts user-submitted reviews. (See my sig.) It's a constant source of amazement to me what a low level of morality (and intelligence) some authors have. They'll add their book to the catalog even though it's not free. (The site's UI tells them very clearly that it has to be free online in order to be listed.) Then they'll post their own "review" of the book, which reads exactly like a dust-cover blurb rather than a review. Then I check the email address they used to sign up on the site, and it's the same as the email address of the author of the book -- this despite the fact that the button they had to click on to submit their review was labeled I am not the author, and have no personal, professional, or business relationship with the author. I am submitting my review..
About 50% of the reviews I get are like this, and I have to delete them by hand. I don't actually get that many reviews submitted, which is a good thing in a way, because if the site was really busy I'd never be able to keep up.
I don't think there's any way of solving this problem, since the internet was designed for anonymous use, and even if it was technically feasible to verify identities on the internet, I wouldn't want to do it. Amazon tries fairly hard to deal with this problem. These days they won't let you submit reviews unless you've bought something from them, which is probably a reasonable way to stop sock puppets. They also try to get you to build up a reputation for your online persona, even if it's not publicly tied to a meatspace identity. That doesn't really work that well, though. For instance, there are certain people on amazon who submit something like ten reviews per day, 365 days per year -- obviously they're not really reading all those books. I also don't see any way to stop the phenomenon of the author getting his friends, family, and grad students to write good amazon reviews of his book.
Because of all this, I'm suspicious of any statistical method of analyzing user-submitted reviews. You just have no way of knowing which reviewers are honest. You really have to look at the individual reviews and see if what they say makes sense. Ebay feedback is an example of how silly this can all get, even in a community where people really do have long-term online identities that they have an interest in maintaining good reputations for. What the heck does it tell you if the seller has 99% positive feedback? Absolutely nothing. You have to read the 1% negative reviews and try to evaluate whether they sound reasonable.
Find free books.
Do we really want the internet to represent the people's opinion the same way Congress represents the people's opinion? Let some software compile it and dictate what others read?
Whomever wrote this piece-of-hunk-of-crap really... sucks.
NIST runs yearly evaluations regarding automatic summarization. Some information about that stuff is available at http://www.nist.gov/tac/ (used to be http://duc.nist.gov/).
There are two main approaches: the domain-dependent template filling plus text generation, or the domain independent statistical sentence extraction. And either way, the quality of the generated summaries is far far away from what a human can write.
While machine-learning-powered research systems are much better than Word or OSX summarization, the way to go is still long...
Since they explicitly mention Amazon, heh, my experience with Amazon's user reviews has been pretty bad to start with. Caveat: it's not about electronics, but I do buy games and the occasional DVD movie off Amazon.
My impression is that the amount of fanboyism, astroturfing and bullshit is... epic. Monumental.
E.g., read some reviews for a game that's not released yet. My favourite example was Gothic 3, when it wasn't even in beta yet, or even alpha. The only thing anyone had were some screenshots of what the graphics engine can do. That's it. Nobody had anything playable yet, probably not even the devs.
Well, people were already writing reviews in which it's the greatest game ever, and the gameplay rules, the graphics are the best since Michelangelo, etc.
When released, the game was a buggy mess that didn't even vaguely resemble those "reviews". The graphics had some major glitches. Quests could be broken because the NPC had fucked off, and I know someone who encountered that right in the freaking intro. The game had a nasty memory leak, where eventually it would start to barely crawl and eventually crash... often while saving, leaving you with a corrupt and unusable saved game. Gameplay too was a broken fuckup: e.g., combat was a broken whoever-hit-first-wins affair, because then the other would be continuously interrupted and unable to hit back or change weapons or whatever. Even a flea could probably kill you, if it hit first. Etc.
Most of that stuff _still_ hasn't been fixed, after more than a dozen patches and the publisher giving up on it.
But, of course, going by the user reviews, you'd think it's the greatest game ever.
Now as a human, you can filter out the blatant bullshit, see which reviewers better reflect your taste and didn't post too much bullshit before, etc. I'm skeptical that a program can be too good at doing the same.
But I have an even worse fear: that once people figure out that they only need to game a program, and how, we'll see even more fanboyism, astroturfing and bullshit. Plus an army of sock-puppets to mod each other up, if the bot takes that into account. Basically, think about all the link farms and link spam on the net to game Google's page rank. Now think the same for a bot aggregating reviews. I find that scary.
So, no, I don't want it on Slashdot too. Basically, would you really want 300 goatse links, just so the bot includes it in the digested version?
A polar bear is a cartesian bear after a coordinate transform.
someone is summarizing for me according to his algorithm which i have no clue about as an end user.
Hackers have long memories. It works both ways.
Just googled it and it looks like we got to see the birth of this new term! http://www.google.com/search?q=astrotrolling
Anyone else is getting memories of the original Rollerball movie (the remake sucked donkey balls) where the AIs had summarised all the human literature and books were no longer available?
If anyone laughed at that concept, start trembling.
The submitter commented that this seems to have promise. How so? It's obviously just a filter that's specific to the comment format of Amazon Electronics. It's not like this plugin does any type of natural language processing. The person who developed the extension obviously noticed a pattern that's specific to how people comment at that specific web site. If that's the case, the application has no promise for any wider application. If that's not the case, this person has done some world class work in breaking the barriers between humans and machine, which means they can understand us now!