Slashdot Mirror
Vista Speech Recognition Goes Awry
An anonymous reader writes "It seems even MSNBC is willing to take a jab on those rare occasions when Microsoft products don't work. During a demo of Vista's speech recognition technology, Vista couldn't differentiate between mom and aunt, and all attempts to rectify the problem just made it worse. Wait until you see what it spat out, I think we have a new 'All your base.' Don't you just love Microsoft's live demonstrations?"
Reminds me of the Roald Dahl short story about the ant-eater who ate someone's aunt because their accent rendered the two words the same.
I can't remember what the story was called.
the layman's guide to computer science
It's just a one-time thing.
I mean, it's not like they have a reputation for releasing half-assed code that's been hyped up through marketing to the point that it will never perform as advertised.
And it's not like this is a company that is having image problems due to its monopolistic nature.
Or headed by an infamous ragaholic with a history of intolerance towards free standards.
Nope, I'm sure that this is just an accident by a company that spends its off hours petting little baby chickens and bunnies.
Reminds me of the time when I worked at a computer store and we played with the voice recognition card in a PowerMac floor model. Somebody programmed it so that if someone said "Computer, bite me", it would respond with "Can't bite what's not there". Over time the accuracy of the recognition fell. One day as a salesman was talking to a customer about the computer it misinterpreted something he said and said "Can't bite what's not there". Needless to say that system was wiped and we weren't allowed to play with it anymore.
it could lead to surprising porn....
Monstar L
Yes, once again Microsoft S/W Engineers learn that the more public the demo or the more important the audience, the more likely some will go wrong. It's one of Murphy's laws. Been there. Did that. Barely survived.
Experience is the human quality that enables you to recognize a mistake immediately when you make it again.
Dacap
English -- gotta love it! / The engineers refuse to refuse the rocket until the refuse is removed from the launch pad.
This isn't the first presentation went wrong, isn't it? ;)
Win98 gone wild: http://www.youtube.com/watch?v=Hrbx9_AY720
Media Center Edition gone wild http://www.youtube.com/watch?v=j7EEbokKLHI
We can add this one to the list too
Final text:
Not quite as embarrassing as the Windows 98 BSOD, but more entertaining than the Ballmer developer's video.
http://www.ntk.net/media/developers.mpg
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
Steve Ballmers accidently send an e-mail while diligently testing the software. The e-mail says:
"Sir put down the chair, then we'll talk"
"No Steve wait up, don't do that"
"BOOM CRASH BOOM CRASH BOOM CRAASH WAAAH NOOO STOOOOP"
"DUDE, THE COMP HAS A BSOD! WAAH!"
I have used Naturally Speaking, it can take a bit of time to train it, but if only you use it then you can eventually get it to the point where you can talk at a normal speed (although it has to be clear) and it will get to approaching 90% accuracy, sometimes I had it higher. The point was that it couldn't be used as an alternative to typing for extended periods though because you had to check everything it wrote.
One thing it did do which was good though is tried to understand sections of speech, rather than just each word, which did improve accuracy. Words often follow patters and there are few words that make sense after a word, so it was often right with "over there".
SR tech will eventually be as good as on star trek as long as people work on it. I would give it 20 years if it is seen as something which could make a lot of money, 40 if you have to wait for interested people to do it for free on their own time
*''I can't believe it's not a hyperlink.''
Well, I've taken a look at that (a while back): Dragon seems to be the leader, they get (with a month of traning) te best accurracy.
However, sound recognition engineers are slowly realizing that the problem of recognising words is not just the algorithm's fault. Even people arn't able to understand all words from a taped conversation in a cafeteria.
Dragon is currently the best, getting further will probably require more input, like a webcam to read your lips. This is just another Microsoft product where they read the wikipedia page on it, produced a flashy interface and packaged it with their OS. If you want sound recognition, don't go with Microsoft, they don't have the expertise.
PS: Don't forget, that getting a good or even special microfoon can make all the difference.
Microsoft routinely puts out their excellence over everyone else including OSS. Hear them talk about Office w.r.t. OpenOffice. They talk down about it, mock it, dismiss it, etc...
It's called modesty. If MSFT had any [and some humility] they wouldn't get laughed at so hard for this. I mean look at Linux. Find a bug in the Kernel, fix it, post notices that its. You don't see anyone saying "Oh hahaha, Linus is at it again!" That's because you also don't see Linus on CNN mocking the rest of the world.
Microsoft deserves all the negative press and humilitation they get because they are shameless, deceitful, greedy monopolistic bastards.
Tom
Someday, I'll have a real sig.
It's hard to wreck a nice beach. :)
Nah, that's not it. I don't hate "Microsoft"; that's just a name on a door somewhere. I don't hate 'the corporation'; corporations are not individuals, no matter what the law would have us believe.
The reason I find this eminently amusing is that Microsoft is a company built on marketing. At no particular point has Microsoft had "The Superior Technical Solution"; they have always had luck and better marketing. Since DOS 3.3 there have frequently been products that were more stable, faster, easier to use - you name it. And Microsoft's captains have beaten them in two ways: Marketing and Money.
So of course when a company who has built their foundation on marketing flubs it, it's more amusing than when a company who has built their foundation on performance of one kind or another flubs it. It's inescapable that the Bg Dog gets more scrutiny than the Contender, anyway. And Microsoft apologists should understand that.
Thinking outside my Head
Maybe they're twin sisters... ;)
Who cares if they ever get up to star trek level. The technology still sucks. It's much quicker and less annoying to the people around you to just type on your keyboard. Sure it has some uses such as those who don't have full use of their hands. We shouldn't abandon all research on the subject, becuase it does have it's uses, but I don't think it's something worth pushing on the general population, especially before the technology is actually ready. People already don't like their computers, pushing buggy technology like this out will just increase the problem.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
The computer in Star Trek (at least in the Next Generation) was WAY too smart. For it to do what it supposedly did in the show, it would have to be sitting there, monitoring the conversation all the time, and be totally able to understand the context of what was being said to know what to do. Not only when people directly asked the computer a question, but also when people wanted to converse with someone.
;)
For example, how does the computer know that Picard wants to call Riker and isn't just talking about him? Oh and keep in mind the computer never misinterpreted something. In other examples, people would carry on intelligent conversations with the computer - all those holodeck scenes, Troi ordering chocolate, etc.
Star Trek-style of SR I think would be the holy grail and is probably always going to be out of reach. Barring some amazing breakthrough in AI algorithms, the computer power required just for the situations above would be incredible - and that's computer time that probably could be put to better use elsewhere, even if it was found to be possible.
I think the computer in the original Star Trek was more realistic - but even there the voice-recognition was far beyond what we're capable of today, as Microsoft has demonstrated so well. Plus all the blinkenlights that seemed to have no useful purpose were cool.
"Dear mom comma"
Dear aunt,
"Fix aunt"
Dear aunt, let's set
"Delete that"
Dear aunt, let's set
"Delete that"
Dear aunt, let's set
"Delete that"
Dear aunt, let's set so
"I think it's picking up a little bit of echo here...delete - select all"
Dear aunt, let's set so double the killer delete select all
*Manually selects all and deletes*
"Okay, I'm glad you're enjoying this"
*Laughter*
Speech recognition is still just a gimmick anyway. We still have a LONG way to go before it gets to the point that Joe Average User imagines it should be. Joe average user wants his computer to respond like the one in Star Trek. I still want to set up my Asterisk server with speech recognition, though, so that people can either dial or say the extension they want. It'd also be neat to pick up the phone, say "Call Mom" to the dial tone and have it call my aunt for me.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
why not just use two mics, one to record the ambient noise (positioned away from the voice mic) the other to record the voice (headset) then as you have two signals just subtract the ambient noise signal from the heaset signal , voila clean headset mic audio
works for music too, you could control your music player by voice even when its playing loud (at a party) by removing the music signal from the mic signal
-AJS
Hmmm, no. Maybe it's the way they deal with failures. Remember Bill gates trying hard to demonstrate the Media Center? Some time after that Steve Jobs gave his regular Macworld keynote when his Mac didn't respond anymore. He moved a monitor switch to continue the presentation on another Mac and said: "Well, that's why we have backup systems here."
Reason 1: You don't have to train this software. That's when you have to read aloud a canned piece of prose that it displays on the screen -- a standard ritual that has begun the speech-recognition adventure for thousands of people.
I can remember, in the early days, having to read 45 minutes' worth of these scripts for the software's benefit. [...] NatSpeak 9 requires no training at all."
As for ambient noise, there's often useful contextual information there too. Ambient noise can give information about where the speech is occurring and about what is happening at that location. In some rare cases the ambient noise might even be responsive to the speech itself. The audience laughing in the example was a clue that a) an audience is watching, and b) the system made a mistake. A human would have recognized that and used it to advantage. For a speech recognition system to work as well as a human, it will not only have to get better at separating speech from ambient noise, but it will need to be able to recognize the ambient noise.
OS/2 Warp had speech recognition in 1994 with OS/2 Warp. Better yet, the OS/2 version of netscape at the time was speech enabled (browse simply by speaking the link). Even cooler was that the netscape developers actually listened to the OS/2 community with that version (I remember them implementing something that I had asked for...very cool). Keep in mind that the average system of that time was a pentium 133 with 100MB of ram. And here we are at 2006, With GHz processors and GBytes of RAM dirt cheap, and M$ is just now starting to experiment with this? By now this technology should be damned near perfectly integrated across the board! Thanks for abusing your monopoly power to destroy all of the competition and REAL innovation, Microsoft!
That's so last century. NPR did a bit on the new Dragon Dictate 9. The NPR reporter got 100% accuracy out of the box, no training.
Dictation Software Improves Usability, Accuracy
I recall a friend of mine demoing the MacOS speech recognition engine a decade or so ago. The dialogue went something like this: User: Open SimpleText. Computer: Are you sure you wish to shut down. User: No. Computer: Shutting down... As you can imagine, he didn't leave it switched on all that often...
I am TheRaven on Soylent News
Yes bugs happen, yes vista is still in beta but rather then just admit "vista is still a buggy piece of crap software that can't even be used properly by its own engineers" they tell us to sit and wait because we can trust them to fix it.
To MS credit, it is a strategy that works.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
all those holodeck scenes, Troi ordering chocolate, etc.
Hmmm - holodeck, Troi, chocolate.....the combination of those three items is something that gives one pause to ponder.
Um, I'll be back in a little bit.
Please stand clear of the doors, por favor mantenganse alejado de las puertas
A friend of mine called me at work (since he knew that to access MSNBC's videos requires Internet Explorer, Windows Media 9 or better, and Flash, and I have neither IE nor WMP at home) and told me about this.
I went to msnbc.com - and there it was, third on the list of videos on the main page.
I called this to the attention of two of my coworkers, and we viewed the video - total elapsed time, maybe twenty minutes.
Then I went to call it to the attention of a third coworker - and the video was no longer on the front page of MSNBC. OK, so maybe they've moved it off the front page, but it should still be on the Technology subsection, right?
Wrong.
Nor was it under Videos, nor anywhere else I could find it easily.
Perhaps this was just a normal rotation of a video. Perhaps not. But no matter what the real cause, there is the appearance that it was removed from the page because it was too embarrassing. Not good for Microsoft.
However, I will give MSNBC this - they didn't give Microsoft a free ride on this, they ribbed them pretty hard.
However, I knew that this would be appearing on other sources as a video that could be viewed outside of Windows. Actually, I am rather surprised that it took this long.
Now, as to the demonstration itself - it looks to me (a person who does signal processing and analysis for a living) like the presenter had the mike gain too high - every time he spoke he maxed out the bar graph on the display. *IF* he had the gain too high, and the audio was clipping significantly, that could make "mom" have enough of a pop to maybe sound like AUNT - especially if the software is using context to try to reduce the search-space for the words. Of course, that's why I would have a monitoring routine in the system, and if any of the samples are at 100% full scale, or if many of the samples are over 90% full scale, or the signal power is too high, I'd have my software adjust the mike gain down *and* flag an alert to the user. I'd also try to look for the mike element itself being overloaded.
www.eFax.com are spammers
Footnote: Microsoft was a monopolistic, backwards company that started the PC revolution.
;-)
They don't deserve credit for starting the "PC revolution". The credit properly belongs to the hundreds of little startups and hobbyists, the whole CP/M crowd and others like Amiga. Microsoft was a subcontractor to a giant monopoly (IBM) that stepped in after the little guys demoed there was a market, and took over that market. They succeeded mostly because of a marketing budget greater than the budgets of all the little companies combined.
And there's a good argument that, by marketing PC/DOS rather than CP/M, they set back the PC revolution by 5 to 10 years, the time it took for PC/DOS to match the capabilities of CP/M when IBM started their PC marketing campaign.
Sorry; that's the way "the Market" works in the computer field. Small, independent developers make something new and start selling it; the big companies then step in and take over the market through traditional monopoly strategies.
It's likely that we're now going to hear people crediting Microsoft for starting the "voice recognition" revolution by inventing the new idea that computers can understand speech. Marketing can redefine history like that.
(Whereas we computer geeks know that Al Gore invented speech recognition.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
Who knows how the algorithm they implemented works. Chances are the computer scientists behind it are not total asshats and they assumed the sales guy would follow the same procedure they did [e.g. to train it].
Point is, if the sales guy had tried the system out beforehand he would have noticed it not working.
That is, suppose the code is total shit [I know, big stretch for MSFT]. Then isn't it likely it would have failed during the preparation stage? If you are saying "mom" and it always comes back "aunt" you may want to cancel the presentation.
That's why I think he didn't do any prep work for the presentation.
Tom
Someday, I'll have a real sig.
According to Rob Chambers, a developer on the Vista speech recognition team, the failures during the demo were caused by audio gain issues.
From his blog:
Read the entire blog post for a more complete explanation of what happened... one that's just slightly more plausible than most of the explanations proffer by your fellow Slashdotters.
I expect in 300+ years when Star Trek is set, our AI will beat the piss out of Star Trek AI. Hell, the computer has been around for only a little over 50 years. A little over 100 years ago we had just first discovered electricity and flight.
Well, maybe. But we invented microscopes around 300 years ago, and discovered microorganisms immediately thereafter. The understanding that some bacteria were involved in diseases followed quickly. But it was nearly 300 years before we successfully eradicated a disease (smallpox). Today, we're still battling new diseases, and we don't have anything like a general solution to all diseases. We have a few antibiotics that effect more than one disease, but we haven't made much progress in solving the problem of the development of resistance to our antibiotics. Hell, we can't even convince the general public that it's the evolutionary process at work here, and we've understood that for around 150 years.
I wouldn't predict any general solution to a complex problem like voice recognition in a mere 300 years. Maybe we will. But our history of general solutions to other complex biological problems is not encouraging. Neither is the history of our first 50 years of AI, despite the constant hype and Hollywood movies claiming that AI is just around the corner.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
I use Dragon NaturallySpeaking every day (carpal tunnel syndrome), and version 9 has around 99% accuracy, with around 98% out-of-the-box with no training. This means 10 or so errors out of a 1000 word dictation.
g ue.html?ex=1154318400&en=6fd795114b3f72ea&ei=5070
y Id=5577523
I didn't believe it either, until I actually tried it. Dragon is the first worthwhile speech recognition solution I've seen that's practical for general use (Though I'd love if they'd release a "programmers" version to compliment the Medial/Legal versions). I get about 99% accuracy (a decent microphone is *very* important!)
Dragon 9 also doesn't "technically" need training, but accuracy further improves if you do bother to train it a bit. The NYT reviewer was able to get 99.6% accuracy after a short training session.
Here's a few reviews of version 9:
http://www.nytimes.com/2006/07/20/technology/20po
http://www.npr.org/templates/story/story.php?stor
The fleet's computers have "known" Picard since he entered the service. They should be pretty well trained .
The communicator badges in TNG could be transmitting supplementary biometric data and non-verbal commands, which by now have become almost automatic: "Watson. I need you!"
The badge also indicates the location of the person. So if Picard says "Will" (or "number one", which is simply an alias that Picard made for "Riker, William T.") and the computer sees that Will isn't in the same room as Picard (or isn't within normal hearing distance), it simply connects the two via a communication channel.
No 90% is no place near good enough for dictation but it sure might be good enough for some applications.
Think "computer lights", if it gets it wrong you just try again. All those media PC would be good candidates as well. If I say "change to channel six" and thing swiches to sixty 1/10th of the time well I could repeat myself that often in that application anyway; and still be pretty satisfied.
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
Is this "voice recognition" as in "understand English vocabulary and grammar rules and differentiate between speech and commands" or as in "match this sound up with one of 10 prerecorded ones to autodial a number"?
90% accuracy is nowhere near enough for voice recognition in a dictation context
Depends on your own context...I deployed (admittedly, an older version) of Dragon NaturallySpeaking in an office full of mobility-impaired employees. They found it much easier to spend 10% of their writing time fixing errors than 100% of it trying to, for example, type with the onscreen keyboard. If you can't use a keyboard, even crappy voice recognition is a godsend.
Stasis is death. Embrace change.
Since when did Mum sound anything like Aunt?
When I was last involved in adding speech control to an app, I attended a developer workshop at Apple, and found out much to my surprise that my mic wasn't any good. It sounded fine when I used it for voice recording, but for recognition the gain curve was all wrong. When I tried one of the mics that the speech team from Apple provided, the hit rate went from under 20% to well over 90%.
When Kim Silverman demos Apple's speech recognition, he uses a high-quality noise cancelling mic. It makes all the difference.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
My mom is in a coma.
My aunt is in a,
If you are saying "mom" and it always comes back "aunt" you may want to cancel the presentation.
Or, in the very least, don't say 'mom'! I've had plenty of times where a salesperson tests something the day before a demo (usually after a week of knowing he had a demo, but that's a different rant) and finding something. Our usual response with that short of notice is 'well, don't show that' since we didn't have enough time to fix it. At best, we could send a version that had the error suppressed but not truly fixed.
R: That voice. Where have I heard that voice before? B: In about 365 other episodes. But I don't know who it is either.
Every time I say the word "Linux" it gets typed out as "Windows". Go figure.
Best. Comment. Ever. Enjoy!
Who knows how the algorithm they implemented works.
Probably nobody at Microsoft. . .
The PR guy was demoted. Microsoft put him in a box with a keyboard and called him "Vista Speech Recognition." See, such technology is in reach!
I have freaks! I did something right...
Actually the last MS demo flame out of this magnitude came when the beta of Windows 98 was being demoed. They wanted to show a scanner "just working" with Windows 98 and USB. W98 hit the blue screen of death when the USB scanner was plugged in.
There, I found it. The file is an old QuickTime movie. I'm going to put this up on YouTube. There, that's done. Have at it.
Knowledge is power. Knowledge shared is power multiplied.