Slashdot Mirror
Google Assistant Is Smarter Than Alexa, Study Finds (cnet.com)
For the second year in a row, a study found that Google's digital assistant is smarter than Amazon's assistant, Alexa. The study does note that Alexa is catching up and was far and away the most improved from 2017 and 2018. CNET reports the findings: Digital marketing company Stone Temple released the results of its 2018 smart speaker quiz earlier this week. It did a similar test last year in which it asked digital assistants roughly 5,000 questions to see which assistants answer the most correctly. For the first time this year, Stone Temple asked the questions separately to Google Assistant on the company's Home smart speaker, and an Assistant-equipped phone. The study found that Google Assistant attempts the most responses, and gets the most attempted responses correct. Strangely, Assistant performed even better on a phone than on a smart speaker. Surprisingly, Microsoft's Cortana took second place, with Alexa trailing both and Siri lagging far behind the rest. Alexa doubled the number of questions it was able to answer from 2017 and Microsoft's assistant improved as well, with Google holding relatively steady at the top while its competition catches up.
So the companies with their own search engines ranked first? Wow, a shocker!
I'm guessing that's what this comes down to - ask a question, it's sent to a server, parsed, runs a web query, then returns an answer - so it's hardly surprising a device made by Google wins.
The entire point of Google Assistant seems, to me as a user, to be to actually assist me. It gives me answers, plays games, schedules things. It's pretty smart. I'm totally fine with Google mining all that data to target me with ads or whatever.
The point of Alexa, and most of Amazon's technology from phones and tablets to buttons and Alexa, is to make me buy things from Amazon. The other aspects are just as good as they have to be to keep up with the market, sorta.
The difference in design focus is apparent when using these systems.
I think the difference between Google's products results shows that's not entirely true. Otherwise, they would've returned the same results.
AI/Machine Learning is playing the pivotal role here.
Siri's results prove that in my mind, she's dumb af. All she does is web queries.
Comment removed based on user account deletion
I'm going to patent that process. Sounds obvious, but the Patent Office doesn't seem to notice such.
Table-ized A.I.
It doesn't take long to realize that right out of the box Alexa is basically just speech recognition and voice commands. There is not a whole lot of typical "AI" going on. I do think eventually Alexa will get to a point where you are holding a normal conversation with a device that has an endless amount of knowledge, but right now it's not even close.
I recently said to my phone, "OK, Google, is it supposed to rain today? When you answer, keep in mind that you got this question wrong yesterday."
To my surprise, it replied with a list of links to alternative digital assistants I could install.
Alexa brings me beer and booze and everything else.
Good enough for me.
is a question Siri can't answer. Tried that yesterday eight times and had three coworkers try that each at least two times. One of the responses gave us directions to Austin, TX I assume because that is the home of Whole Foods. If AI isn't good enough to give us directions to a store a couple of blocks away, it's just useless and not even worth discussing.
Yes Google's assistant can process natural language far better.
But is it "smarter"? I'd define smarter as "able to do more" and it's clear that not only does Alexa win there, it has an insurmountable lead both in terms of marketshare and with the furious clip that Amazon has people developing "Skills" for Alexa.
Does anyone doubt eventually Alexa will handle natural language too? But then it will ALSO have a wide distribution of users, and vastly more skills... not to mention being wired into something people want, need and use (Amazon purchasing).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
https://www.youtube.com/watch?...
The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
They are stupid. They have comparison bar graphs showing % correct of those attempted and bar graphs with % attempted. Which are both relatively useless metrics on their own, the most useful one for a comparison would be % correct over asked and they don't provide that!
I can't believe anyone with knowledge of statistics, data analysis or a person of sound logic in general had anything to do with this report.
All their graphs are like that, the year-over-year comparisons compare either the % attempted or the % correct of those, never the % correct overall, even their % wrong comparison is just on those attempted! Crazy-frustrating report! How confident can you be that their methodology is sound after that?
Violence is the last refuge of the incompetent. Polar Scope Align for iOS
Are people who don't invite these invasions of privacy into their homes. Alexa, turn up the heat. Get off your fat ass and do it yourself. Alexa look this up. Read a fucking book. FFS.
-- I ignore anonymous replies to my comments and postings.
My guess would be it's:
X% of questions asked are attempted.
Y% of questions attempted are answered fully.
X and Y are thus not percentages of the same thing, and thus Y can be greater than X.
I decided to test Google Home and Alexa devices about 6 months back.
I bought 3 Home devices and 2 Alexa devices along with a few Hue lights and strips for the kids to play with.
Alexa immediately annoyed the crap out of me. Most questions got an answer, and a good number of them also ended with "We have XYZ in stock, would you like to order them now?"
"Alexa, what kind of food does a marmot eat? Mostly vegetation. We have marmot food in stock for $15.95, would you like to order some?"
While Google doesn't seem like an advertising-driven microphone, it is completely incompetent.
I added the lights into the system. I made two rooms "Boy's Room" and "Girl's Room", and then added the lights and strips into each "Johnny's Light" and "Johnny's Strip" as well as "Susie's Light" and "Susie's Strip". Each room also has an overhead light "Girl's Light" and "Boy's Light" (note how it differs from the room names).
Saying any variation of "Turn on/off Johnny's light/strip" causes Google to say they're sorry but they can't help with that. Renaming it from "Johnny's" to "Johnny" and then saying "Turn on/off Johnny light/strip" works perfectly. Apparently Google is confused by a apostrophe.
Saying "Turn on/off girls/boys light" results in the entire room turning on or off.
Saying "Turn on/off girls/boys room light" results in the entire room turning on or off.
Apparently Google gets confused by having a "Boy's Room" and a "Boy's Light" and can't tell the difference.
About 1 out of every 10 commands will ignore the room or light specified and cause all the lights to turn on or off or whatever you were trying to do.
After a while I figured that maybe it wasn't Google--that it was Philips Hue. So I bought a cheap Chinese knock-off MagicHue strip. Nope. Same exact problems.
The very moment I stop naming them using proper names and instead call them "wall strip" and "bed strip" and "superman light" and other such nonsense it works perfectly every time.
Google doesn't understand proper names, commas, or the difference between rooms and lights.
Even worse is the part where I have to manage names and rooms in the Hue app, then tell Google to 'sync devices' and then make sure they are named properly in the Google Home app.
In my personal opinion, IoT sucks. The only IoT product that I like is Google's Nest Protect. Then again, it's not difficult to alert a phone if there's a fire. (The thermostat constantly goes haywire and sets ridiculous temperatures.)
The voice assistant is remarkably responsive when you understand the technology necessary. But please don't neglect ordinary search engines!
Amazon can't even do a product search. I searched for a product by brand name, a name unique among all the products available. I got 10,000 results, only one of which actually had that unique word. I searched for "paper towels" and got 10K results, only a handful with my term and far outnumbered by 'toilet paper' and soaps and totally unrelated products.
Google is almost as bad. The vast majority of results lately don't have any of the words I ask for. When it says it found 10 million results, try this little trick: jump to the last page. You'll discover you are on page 7 or so and that there were really only 700 results. (most of which did not satisfy your search terms) Google gets worse every year.
...omphaloskepsis often...
Surely they mean "less stupid"? Try this with either: "Do NOT give me the weather forecast for tomorrow." Sure enough, they both give you the weather forecast for tomorrow.
If there had been, Siri would have come in first, of course!
And...I'll bet Alexa was the best at ordering stuff from Amazon.
Can't you just say, "Alexa, Enable the Google Assistant Skill" ?
Seems like that ought to fix any problems!
My understanding is that Alexa is still superior for answering questions about whether cats can eat pancakes.
Comment removed based on user account deletion
Google may have a lot of data but Amazon has literally HUNDREDS OF PEOPLE working on just Alexa. I don't think it's crazy to think that Amazon will shoot way, way ahead of Google on this effort. Even if Amazon didn't have a massive lead now I would expect this to put them on top.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Implying that Google's assistant is just some programmers side hobby?
Nope, just that Google does not have nearly as many resources on it, which is 100% true. I know a few people that work at Google...
Implying that throwing people at something solves a data set / deep learning problem?
I've been doing some deep learning stuff over the past few years, your problem is implying deep learning can address this generally and/or better than 500 people thinking of ways they can improve and expand a service TODAY. Applying deep learning only solves the problems you can imagine, and they just have fewer people imagining what and how to solve problems with assistants.
Plus - do you seriously think Amazon does not ALSO have some of those hundreds of people working on deep learning solutions to some problem domains....
The problem you have is you are thinking of the old, very true, adage that you cannot solve a software problem faster by throwing more people at it. HOWEVER in this case they are not solving a single problem, they are developing every domain they can think of in parallel, where they do benefit mightily from the sheer number of different people thinking about it.
Honestly I fail to see how you got to your conclusion
30 years in the software industry and like I said a lot of knowledge of both Google and deep learning tech, along with a lot of thought around the whole area.
You forget that analysing data and serving up results is their core business
And you forget how long they have been doing that, the data stores they have access to vs. Amazon, and how companies ossify over time to which Google is absolutely no exception. You are thinking of the Google that was, not the Google that is...
I don't really feel like saying any more on the topic but I will read a response if you choose to write one... otherwise I think we can just wait and even in a year or two you'll see a lot of what I mean come to more visibility.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Product A attempted to answer 60 / 100 questions.
Product A fully answered 40 questions.
It attempted 60% and fully answered 66.666%.
The "fully answered" percentage is measured against the number attempted, not the number asked.
Solve? It just rearranged the ODE into different forms.
Every end has half a stick.