Slashdot Mirror
Microsoft Speech Recognition Now As Accurate As Professional Transcribers (techcrunch.com)
An anonymous reader quotes TechCrunch:
Microsoft announced today that its conversational speech recognition system has reached a 5.1% error rate, its lowest so far. This surpasses the 5.9% error rate reached last year by a group of researchers from Microsoft Artificial Intelligence and Research and puts its accuracy on par with professional human transcribers who have advantages like the ability to listen to text several times. Both studies transcribed recordings from the Switchboard corpus, a collection of about 2,400 telephone conversations that have been used by researchers to test speech recognition systems since the early 1990s. The new study was performed by a group of researchers at Microsoft AI and Research with the goal of achieving the same level of accuracy as a group of human transcribers who were able to listen to what they were transcribing several times, access its conversational context and work with other transcribers.
On a daily basis in my work environment Microsoft technology is used to a) record voicemail and b) generate text from the speech. Never, ever, have I received any converted voicemail that wasn't completely unintelligible gibberish. Seriously. This is utter nonsense.
When a human transcriptionist makes a mistake you can usually work out what they meant. When Speech-to-text (STT) makes a mistake it is often gibberish. So objectively it is "better" at transcribing, but subjectively much worse.
holyfield is these all of this was made worse by the fact that i had these birds skilled estimate uh... supplying itself what's your special prom to prevent fraud reform
thoughtfulness julia roberts police comments entry drug connections predicting that nighttime beating
Harald
Some months ago, I did some tests with speech recognition software and my conclusion was that it is still too unreliable. My intention was to develop an application allowing me to write moderately complex code by voice (creating files and folders, including proper indentation, recognising functions, variables and other basic elements, etc. Basically, allowing me to write/edit the main parts of a random algorithm in certain language without touching the keyboard). I did test Microsoft in-built functionality (+ used one of Microsoft's .NET programming languages) and it wasn't even close to what "5.9% error rate" seems to indicate (almost perfect?).
In defence of the software, I have to say that my English accent isn't precisely excellent (some people say that it is "too thick" and other people just say "what?". LOL) and honestly I make a very little effort to pronounce properly. But this is also the problem with speech recognition: it is mostly focused on a specific language/accent/intonation. I was doing my tests in an English Windows version and this was the language for the default speech recognition (and adding a different one wasn't precisely straightforward).
I do perfectly understand the complexity associated with developing a reliable enough piece of software delivering what I was expecting; but this is precisely the reason why I looked for existing solutions rather than developing everything myself (what I do pretty often). In any case, my impression is that you can still not expect good enough reliability of (Microsoft's) speech recognition software, much less when mixing languages/accents up (particularly problematic situation: including Spanish words when talking in English). I might give a new shot at all this next year though.
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
"As Accurate As Professional Transcribers..."
They left out "from Uzbekistan transcribing Navajo - underwater".
Never trust anything Clippy say.
If it can recognize "It's difficult to wreck a nice beach", I'll be thoroughly 'whelmed'.
In a sound proof studio built for sound recording spoken by someone with speech training?
Or in an environment with 30 people talking in the background, an air condition running, doors and drawers slamming, people laughing, feet
and chairs shuffling across the floor, some photocopiers that got their last service before Bush left office whining for hours and a person speaking into the phone while at the same time talking to coworkers and you're expected to know which words belong to you and which ones are directed at someone else?
Aka "open plan office".
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
It still showed up at the South Park "Save Films from their Directors" club for the wrong reason when it heard, "Free Hat".
(For those that aren't South Park followers...)
Cartman writes "Free Hat" on the advertising poster in the belief that freebies are necessary to attract people. However, the crowd mistakenly thinks the rally is to free Hat McCullough, a convicted baby killer they believe was innocent.
Now thinking that "Free Hat" would be a great name of one of those Windows App Store pirate streaming apps ...
It must have been something you assimilated. . . .
If you believe Microsoft without independent verification from an otherwise uninterested third-party who has no investment in the outcome, then you're a fool.
One in 20 words is wrong?
How can a human transcriptionist be that bad?
Help! I'm a slashdot refugee.