We just released Deep Speech 0.4.1!
Known Issues tab of
DeepSpeech 0.4.0 release you mentioned
Incorrect model was uploaded to release which will be fixed in 0.4.1…
is this (
0.4.1) release has the correct model. will it produce good results than
and will it solve the issues discussed here
This is the link to 0.4.1 and as stated in the 0.4.0 release the correct model is uploaded in the 0.4.1 release.
As to 0.4.1. vs 0.3.0 results, see the release notes for both for a comparison.
Hi @kdavis, I just did a WER test for the Windows client, here’s the result:
Estable RAM usage of 1,7GB.
The test took about 3h on a virtual Intel Xeon Platinum 8168 @ 2.7GHz vcores 16
WER 8,87% with LM enabled.
You can see the tool that I wrote here
I noticed considerable amount of errors related to ’ for example with “i’m” and “i am”, this should happen? Yes the WER increases but at the end is the same meaning.
At the moment I can’t build for CUDA, hopefully soon I got access to a CUDA device
Cool! Nice having a second pair of eyes on the WER.
We had a slightly lower value 8.26%, but basically it seems about the same.
As to the problems with apostrophes, yes we’ve noted the same. I’d guess it’s a hard problem to solve as when spoken quickly it can sometimes be unclear if a person said “i am” or “i’m”.
If you have any ideas on how one could solve it, we’re “all ears.”
Well no at the moment
What about this one ”perform’d”? There are a few with 'd
I’m still collecting Spanish from Librivox so, I’m not experienced with the creation of the LM, if I think I got something that can improve the apostrophe issue of course I’ll share.
I just grep’ed the SLR11 text and there are 175 lines that contain “perform’d”. So that’s the source of the “perform’d” problem.
Here’s a list with the 'd issue
I better share the result, I’m not native so I may be missing couple more.
I run the WER test again and noticed that few of them also are appearing in the LibriSpeech clean test corpus
Here’s the wer result https://pastebin.com/1Wrp3pVH
I don’t know if
thee's are correct.
When I validate I pay special attention to small things like whether the person said “I’m” or “I am” but I have no idea if other validators do. I think if you’re clicking through quickly you may miss stuff like that. So there may perhaps be incorrectly transcribed clips in the dataset contributing to this.
I haven looked for all the strings you mention. But I’ve found examples of all the ones I’ve searched for in SLR11. For example…
beyond the green within its western close a little vine hung leafy arbor rose where the pale lustre of the moony flood dimm’d the vermillion’d woodbine’s scarlet bud and glancing through the foliage fluttering round in tiny circles gemm’d the freckled ground
amidst them next a light of so clear amplitude emerg’d that winter’s month were but a single day were such a crystal in the cancer’s sign
a cleric lately poison’d his own mother and being brought before the courts of the church they but degraded him
I seems like this is a common construct in older forms of English and SLR11 contains many texts that are in public domain, as they are old enough to pass in to public domain, and thus reflect this old construct.
It seems like we could get a pretty good boost by simply using newer texts in place of SLR11 . However, then we have the legal question of how to obtain modern texts that are still open.
If we correct the existing text? I think is not too hard since they are easy to spot. The question is, is there any legal issue editing the existing text?
Editing shouldn’t be problematic
Well I said it will be easy to spot, but not easy to correct them hahaha, is worse than I thought.
I can take it, but will need the help of native speakers, for example “worse’n” I changed it to “worse than”.
Changing to “better than” here makes no sense.
A GIRL LIKE THAT OUGHT TO DO SOMETHIN BETTER’N THAN STAY HERE IN SOUTH HARNISS AND KEEP STORE
I found Spanish and French sentences, should I remove them or is there any reason to mix languages? Sometimes is mixed, for example “he said hola amigos”.
I have an idea, which I’ve not had time to test yet.
I’ve made a number of language models with different parameters. In particular in some of them I’ve limited the vocabulary to the N most frequent words, with various N’s, of SLR11.
As the various “million’d”, “emerg’d”… are not common they’ll likely not be in the N most frequent words and the language model will not exhibit this strange behavior.
But I need to test this idea with some WER runs.
If you share them I can run the WER test
Here’s the librispeech-lm-norm.txt.gz cleaned, I’ve removed a lot of Spanish and French. One more to test
Unfortunately it’s 16 TB of language models; sharing is a bit hard.