Google Assistant Is Smarter Than Alexa, Study Finds (cnet.com)

For the second year in a row, a study found that Google's digital assistant is smarter than Amazon's assistant, Alexa. The study does note that Alexa is catching up and was far and away the most improved from 2017 and 2018. CNET reports the findings: Digital marketing company Stone Temple released the results of its 2018 smart speaker quiz earlier this week. It did a similar test last year in which it asked digital assistants roughly 5,000 questions to see which assistants answer the most correctly. For the first time this year, Stone Temple asked the questions separately to Google Assistant on the company's Home smart speaker, and an Assistant-equipped phone. The study found that Google Assistant attempts the most responses, and gets the most attempted responses correct. Strangely, Assistant performed even better on a phone than on a smart speaker. Surprisingly, Microsoft's Cortana took second place, with Alexa trailing both and Siri lagging far behind the rest. Alexa doubled the number of questions it was able to answer from 2017 and Microsoft's assistant improved as well, with Google holding relatively steady at the top while its competition catches up.

Comments

bah

[Blymie]

So the companies with their own search engines ranked first? Wow, a shocker!

Re:bah

[Tablizer]

companies with their own search engines ranked first...

When Jeff Bezos realizes that, he'll enter the search engine biz. Maybe he'll buy Microsoft, keep Bing and Azure cloud, and sell the rest to China.

Re:bah

[UnknownSoldier]

Considering

* back in Feb. 2018 that Amazon's market cap was at $702.5 billion compared to Microsoft's at $699.2 billion (beating MS for the first time),

* but in March 2018 Amazon was at $684.3 billion compared to Microsoft's $692.4 billion ...

... yeaaaah, about that, Amazon buying Microsoft isn't going to happen anytime soon.

Re:bah

[hAckz0r]

No, actually I think its because Google had already analyzed everything that all the Stone Temple employees said and did via email, chat, orders on the web, and past web search history.

Google Assistant already figured out who was doing the study and what questions they were going to ask, so they had the answers ready for them before hand. The web searches were just to make the competition look legit to the judges so they would not catch on.

Alexa on the other hand only knows what Stone Temple employees ordered online from Amazon, and last I checked you still could not order "test questions" from Amazon, even with Prime Membership.

Smarter Web searching

[Larry Lightbulb]

I'm guessing that's what this comes down to - ask a question, it's sent to a server, parsed, runs a web query, then returns an answer - so it's hardly surprising a device made by Google wins.

Zero surprise

[jonnythan]

The entire point of Google Assistant seems, to me as a user, to be to actually assist me. It gives me answers, plays games, schedules things. It's pretty smart. I'm totally fine with Google mining all that data to target me with ads or whatever.

The point of Alexa, and most of Amazon's technology from phones and tablets to buttons and Alexa, is to make me buy things from Amazon. The other aspects are just as good as they have to be to keep up with the market, sorta.

The difference in design focus is apparent when using these systems.

Re:Smarter Web searching

[TFlan91]

I think the difference between Google's products results shows that's not entirely true. Otherwise, they would've returned the same results.

AI/Machine Learning is playing the pivotal role here.

Siri's results prove that in my mind, she's dumb af. All she does is web queries.

Google Assistant can be dryly funny

[Bootsy Collins]

I recently said to my phone, "OK, Google, is it supposed to rain today? When you answer, keep in mind that you got this question wrong yesterday."

To my surprise, it replied with a list of links to alternative digital assistants I could install.

Re:Google Assistant can be dryly funny

[Ichijo]

Each morning I want to know whether the day will be warmer or colder than the day before. It seems like such a simple question but the best either Google or Alexa can do is give me today's temperature, expecting me to complete the research on my own.

"Directions to Whole Foods"

[greenwow]

is a question Siri can't answer. Tried that yesterday eight times and had three coworkers try that each at least two times. One of the responses gave us directions to Austin, TX I assume because that is the home of Whole Foods. If AI isn't good enough to give us directions to a store a couple of blocks away, it's just useless and not even worth discussing.

Re:"Directions to Whole Foods"

[cascadingstylesheet]

is a question Siri can't answer. Tried that yesterday eight times and had three coworkers try that each at least two times. One of the responses gave us directions to Austin, TX I assume because that is the home of Whole Foods. If AI isn't good enough to give us directions to a store a couple of blocks away, it's just useless and not even worth discussing.

Wow, really?

"Drive to Whole Foods" (or to some market I actually use) with Google voice search on Android would result in immediate directions, or "you have a few to choose from", etc.

And that's just the plain old voice search, without the "assistant" window dressing.

Define smarter

[SuperKendall]

Yes Google's assistant can process natural language far better.

But is it "smarter"? I'd define smarter as "able to do more" and it's clear that not only does Alexa win there, it has an insurmountable lead both in terms of marketshare and with the furious clip that Amazon has people developing "Skills" for Alexa.

Does anyone doubt eventually Alexa will handle natural language too? But then it will ALSO have a wide distribution of users, and vastly more skills... not to mention being wired into something people want, need and use (Amazon purchasing).

Re:Define smarter

[thegarbz]

and vastly more skills... not to mention being wired into something people want

Yes I doubt these two parts. The rest of it seems okay but frankly I would be very amazed and actually quite let down if Google with it's phenomenal amount of data mining and large amounts of data research doesn't end up with the most useful and functional assistant.

As for being something people want, I would say that is trumped by something people have. Many people (myself included) don't want a digital assistant. None the less I have one in my phone. It would be quite phenomenal if a digital assistant people need to buy separately trumps the install base of a digital assistant that multiple people already have in their house in their phones.

Re:Confusing bar graphs...

[Ecuador]

They are stupid. They have comparison bar graphs showing % correct of those attempted and bar graphs with % attempted. Which are both relatively useless metrics on their own, the most useful one for a comparison would be % correct over asked and they don't provide that!
I can't believe anyone with knowledge of statistics, data analysis or a person of sound logic in general had anything to do with this report.
All their graphs are like that, the year-over-year comparisons compare either the % attempted or the % correct of those, never the % correct overall, even their % wrong comparison is just on those attempted! Crazy-frustrating report! How confident can you be that their methodology is sound after that?

And smartest of all

[theshowmecanuck]

Are people who don't invite these invasions of privacy into their homes. Alexa, turn up the heat. Get off your fat ass and do it yourself. Alexa look this up. Read a fucking book. FFS.