Error when training model


(karthikeyan k) #83

the output is…

(deep4.0) userk@demosys:~/DeepSpeechPro/native_client4.0$ deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt --lm models/lm.binary --trie models/trie --audio /mnt/c/users/karthikeyan/Downloads/chunk15.wav
Loading model from file models/output_graph.pbmm
TensorFlow: v1.12.0-10-ge232881
DeepSpeech: v0.4.0-0-g48ad711
2019-01-09 14:01:51.683099: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Loaded model in 0.27s.
Loading language model from files models/lm.binary models/trie
Loaded language model in 0.419s.
Running inference.
i can do and picture of minutes for eating for the saintly in bloom no deterrent
Inference took 81.658s for 6.060s audio file.

Please have a look…


(kdavis) #84

@karthikeyank I was asking @lissyx as I wanted to see if the process is indeterministic which it should not be.


(karthikeyan k) #85

ohh… am sorry… I missed to read you mentioned lissyx…


(kdavis) #86

No problem. I just want a “second opinion” on the nondeterministic result, if indeed it is nondeterministic.


(Lissyx) #87
alex@portable-alex:~/tmp/deepspeech$ ./tc-linux/deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt -t --audio chunk15.wav 
TensorFlow: v1.12.0-10-ge232881
DeepSpeech: v0.4.0-0-g48ad711
2019-01-09 10:01:00.860650: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
i con dag ond pe tucople of minetes fel e gading for tree sod an tlition plovram nodidpersubrok te fint you
cpu_time_overall=4.45048
alex@portable-alex:~/tmp/deepspeech$ ./tc-linux/deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt -t --audio chunk15.wav 
TensorFlow: v1.12.0-10-ge232881
DeepSpeech: v0.4.0-0-g48ad711
2019-01-09 10:01:09.841426: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
i con dag ond pe tucople of minetes fel e gading for tree sod an tlition plovram nodidpersubrok te fint you
cpu_time_overall=4.23568
alex@portable-alex:~/tmp/deepspeech$ ./tc-linux/deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt -t --audio chunk15.wav --lm models/lm.binary --trie models/trie 
TensorFlow: v1.12.0-10-ge232881
DeepSpeech: v0.4.0-0-g48ad711
2019-01-09 10:01:26.518440: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
i can do and picture of minutes for eating for the saintly in bloom no deterrent
cpu_time_overall=4.08630
alex@portable-alex:~/tmp/deepspeech$ ./tc-linux/deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt -t --audio chunk15.wav --lm models/lm.binary --trie models/trie 
TensorFlow: v1.12.0-10-ge232881
DeepSpeech: v0.4.0-0-g48ad711
2019-01-09 10:01:30.556242: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
i can do and picture of minutes for eating for the saintly in bloom no deterrent
cpu_time_overall=3.87949
alex@portable-alex:~/tmp/deepspeech$ ./tc-linux/deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt -t --audio chunk15.wav --lm models/lm.binary --trie models/trie 
TensorFlow: v1.12.0-10-ge232881
DeepSpeech: v0.4.0-0-g48ad711
2019-01-09 10:01:34.473983: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
i can do and picture of minutes for eating for the saintly in bloom no deterrent
cpu_time_overall=3.74882

(Lissyx) #88
alex@portable-alex:~/tmp/deepspeech$ md5sum models/*
7423967be98caae08a348a3c674ce20d  models/alphabet.txt
5f762eecdc4c4cc2068dc1a84ec57873  models/lm.binary
a27b6d0e0b3dfd1ee6174443694ced05  models/output_graph.pb
a3cafcb87fcf09d38ce58cbc41e5c681  models/output_graph.pbmm
fe25baa1de505592b79fdb95432d7ad0  models/output_graph.rounded.pb
1b2f760aca3e9db0d334c33751ac94ce  models/output_graph.rounded.pbmm
182f72835a19800a3564b5da75ffc526  models/trie

(kdavis) #89

Funny. Or not so funny. I’m the odd man out. My transcription doesn’t match.

Mine was

i can dig and people of minutes for a god in perfecting latin poem notepaper

but you both got

i can do and picture of minutes for eating for the saintly in bloom no deterrent

Out of curiosity what platforms are you on?


(Lissyx) #90

AMD64 Linux/Ubuntu 18.10. Do you get the same result without LM loaded ?


(karthikeyan k) #91

I’m on Windows Subsystem for Linux – Ubuntu 16.04


(kdavis) #92

I’m on OSX 10.13.6. Let me try without LM


Deep Speech v0.4.1 Released
(karthikeyan k) #93

Hi @lissyx @kdavis… Is It possible to perform inference on DeepSpeech 0.4.1 model with a batch of 100 audio files at a time in a loop…


(Lissyx) #94

If you write the code yes …


(karthikeyan k) #95

I already tried this…
but the models are loaded on each iteration… Is there any way that I can avoid loading models each on iterations…


(Lissyx) #96

That’s because you wrote the code in a way that loads the model at each iteration …


(karthikeyan k) #97

Am Sorry… Am unaware how to do that… can you please suggest me any ways to achieve it…


(Lissyx) #98

I’m pretty sure we already explained you what you need to do …