What does “same issue” mean in this context?
That my trained model gives me only one word for all files. Either I checked it on train files or test files.
When I trained on a very small corpus, almost 6 files, it learned only one word, though that word doesn’t belong to any set of data. and it gives the same word if I tried to decode either on train file or test file.
and When I tried on almost 100 hours, it doesn’t give any single word in the results.
Epochs: 20, with early stop = TRUE
Learning rate: 0.0001
Why is it so ?
What happens if you run run-ldc93s1.sh as follows:
(.virtualenv) kdavis-19htdh:DeepSpeech kdavis$ ./bin/run-ldc93s1.sh
WER: 1.000000, CER: 48.000000, loss: 27.773754
- src: “she had your dark suit in greasy wash water all year”
- res: “edted”
This is the result of ldc93s1.
I’m not sure how but your install is very broken. This is basically a “smoke test” that every one of our PR’s has to pass.
If I were you, I’d check everything out from scratch and follow the README instructions again.
Are we suppose to train the model while activating the virtual environment ?
It’s recommended that you use a virtual environment.
I followed the github guideline throughout and working on 0.4.1 master.
I also got the exported model with best validation loss.
Installation:
Linux: 16.04 LTS
CUDA 9.0
CUDNN= 7.1.3
Python 3.6.3
DeepSpeech 0.4.1 master
requirements.txt installed. but Tensorflow-gpu ==1.12.0
Installed LFS from the link given on github
Bazel 0.5.1
Downloaded and checked the pre-trained model from Common voice utterances and results are almost 99%
install CTC
Data prepared in CSV format.
Build the native client
have language model prepared from KenLM, generate trie file.
There is no error I got in training and got the .pb model in the end and also results gave me WER though it doesn’t have anything in decoded output.
Please let me know where I am doing wrong,
Thank you for all the help.
There are many steps here and a problem can creep in anywhere.
To help in debugging can you supply the final training log?
the data is in utf-8 and I believe, DeepSpeech support it. Isn’t ?
If I trained on 5-6 files, I got one word decode, but if I trained on a big corpus of 100 hours, nothing decoded.
Thank you so much.
I asked for the log, what’s printed out as training runs.
I will share my next iteration result, which has short utterances.
I believe, if one doesn’t have a lot of data (like thousands of hours), it wouldn’t work on long utterances. Isn’t ?
Is it possible to show the entire log? Including the command that’s running?