Unable to load pre-trained model


(Rameshkjes) #1

I am facing problem in setting up the Deep speech, When I try to load the pretrained model by executing:

deepspeech models/output_graph.pbmm models/alphabet.txt models/lm.binary models/trie my_audio_file.wav

Terminal Output:

Loading model from file models/output_graph.pbmm
Killed

Could anyone please help me with this?


(Lissyx) #2

How did you installed DeepSpeech? Nothing more on the terminal ? What system are you running ? We have not yet released anything with pbmm support, how did you produced your file ?


(Rameshkjes) #3

I downloaded from here: https://github.com/mozilla/DeepSpeech/releases

  1. audio-0.1.1.tar.gz
  2. deepspeech-0.1.1-models.tar.gz
  3. Source code(tar.gz)

First I created virtual environment in python2.7, then I followed steps from here: https://github.com/mozilla/DeepSpeech

pip install deepspeech

Then I executed:

deepspeech models/output_graph.pbmm models/alphabet.txt models/lm.binary models/trie my_audio_file.wav

I know there is no output_graph.ppmm, but i also tried with output_graph.pb

Operating System : ubunut 16.04 (But its setup as virtual )
Intel® Core™ i7-3630QM CPU @ 2.40GHz


(Lissyx) #4

What’s the exact output with output_graph.pb ? How much RAM do you have allocated ?
Also, please ensure that AVX instructions are avaiable: share the output of cat /proc/cpuinfo


(Rameshkjes) #5

I have Allocated 2 Gb Ram

cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel® Core™ i7-3630QM CPU @ 2.40GHz
stepping : 9
microcode : 0x1b
cpu MHz : 2392.350
cache size : 6144 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc cpuid aperfmperf pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cpuid_fault epb pti retpoline fsgsbase tsc_adjust smep dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2
bogomips : 4784.70
clflush size : 64
cache_alignment : 64
address sizes : 42 bits physical, 48 bits virtual
power management:


(Lissyx) #6

Well, then 2GB is obviously not enough memory, it’s being killed by the kernel’s OOM killer. Likely you can see that in dmesg.


(Rameshkjes) #7

Thanks, I increased the ram size to 8GB. But Now I am getting this error:

Loading model from file models/output_graph.pb
Error: Alphabet size does not match loaded model: alphabet has size 3349619, but model has 28 classes in its output. Make sure you’re passing an alphabet file with the same size as the one used for training.
Loaded model in 39.144s.
Loading language model from files models/trie my_audio_file.wav
Loading the LM will be faster if you build a binary file.
Reading models/trie
----5—10—15—20—25—30—35—40—45—50—55—60—65—70—75—80—85—90—95–100
terminate called after throwing an instance of 'lm::FormatLoadException’
what(): native_client/kenlm/lm/read_arpa.cc:65 in void lm::ReadARPACounts(util::FilePiece&, std::vector&) threw FormatLoadException.
first non-empty line was “1414678853” not \data. Byte: 11
Aborted (core dumped)


(Lissyx) #8

Which is expected if you have not read the documentation properly and mixed the order of arguments.


(Rameshkjes) #9

Oh Sorry, I fixed this using

deepspeech models/output_graph.pb audio/2830-3980-0043.wav models/alphabet.txt models/lm.binary models/trie


(Lissyx) #10

And does it works now? :slight_smile:


(Rameshkjes) #11

Yes. Thanks very much


(Rjzevallos Salazar) #12

I had the same problem, but now when i run it,

terminal output:

Loading model from file output_graph.pb
2018-04-04 11:57:19.772386: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Loaded model in 0.271s.
Loading language model from files 5-gram.binary quz_trie
Loaded language model in 0.699s.
Running inference.
2018-04-04 11:57:23.228952: E tensorflow/core/framework/op_segment.cc:53] Create kernel failed: Invalid argument: NodeDef mentions attr ‘identical_element_shapes’ not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name=“bidirectional_rnn/bw/bw/dynamic_rnn/input_0”, _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
2018-04-04 11:57:23.229242: E tensorflow/core/common_runtime/executor.cc:643] Executor failed to create kernel. Invalid argument: NodeDef mentions attr ‘identical_element_shapes’ not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name=“bidirectional_rnn/bw/bw/dynamic_rnn/input_0”, _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name=“bidirectional_rnn/bw/bw/dynamic_rnn/input_0”, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Error running session: Invalid argument: NodeDef mentions attr ‘identical_element_shapes’ not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name=“bidirectional_rnn/bw/bw/dynamic_rnn/input_0”, _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name=“bidirectional_rnn/bw/bw/dynamic_rnn/input_0”, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
None
Inference took 2.494s for 1.117s audio file.

help me plz…


(kdavis) #13

What did you run, details of command line arguments?

How was deepspeech installed?


(Lissyx) #14

So, I helped @rjzevallos.salazar in private, he has not commented, but the issue was what was expected and documented above: he trained a specific model on tensorflow >= 1.5 and was trying to perform inference on binaries built from r1.4


(bdmcgraw) #15

Hi–
New user here getting my feet wet. The main page on github (https://github.com/mozilla/DeepSpeech) has the order listed as :

"Note: the following command assumes you downloaded the pre-trained model.

./deepspeech models/output_graph.pbmm models/alphabet.txt models/lm.binary models/trie audio_input.wav"

which is probably why some are getting this wrong.


(Lissyx) #16

That’s the documentation for the current master branch. The v0.1.1 README is good: https://github.com/mozilla/DeepSpeech/blob/v0.1.1/README.md

I don’t know how we can improve that, feel free if you have any suggestion.