Illegal instruction whatever I do

DeepSpeech
#1

I am trying to train a custom model for Czech. I followed the instruction in README and in the FAQ, prepared my data, used the pre-built binaries for x64 linux and was getting “Illegal instruction” after a few seconds without further info.

So I thought I’d go from the beginning and I installed the deepspeech package with pip in a fresh virtualenv, I downloaded the pre-built models and ran it as
deepspeech models/output_graph.pb models/alphabet.txt models/lm.binary models/trie data/smoke_test/LDC93S1.wav
Again, I got Illegal instruction.

I was trying on these machines (uname -a)
Linux harmony 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Linux orion8 4.2.8-1-pve #1 SMP Sat Mar 19 10:44:29 CET 2016 x86_64 x86_64 x86_64 GNU/Linux

#2

can you share cat /proc/cpuinfo ?

#3

run deepspeech -h and check the requested position of the wav file

#4

the university machine (orion8):
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU 5130 @ 2.00GHz
stepping : 6
microcode : 0xc6
cpu MHz : 1994.978
cache size : 4096 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dtherm tpr_shadow
bugs :
bogomips : 3989.95
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU 5130 @ 2.00GHz
stepping : 6
microcode : 0xc6
cpu MHz : 1994.978
cache size : 4096 KB
physical id : 3
siblings : 2
core id : 0
cpu cores : 2
apicid : 6
initial apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dtherm tpr_shadow
bugs :
bogomips : 3990.05
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU 5130 @ 2.00GHz
stepping : 6
microcode : 0xc6
cpu MHz : 1994.978
cache size : 4096 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dtherm tpr_shadow
bugs :
bogomips : 3989.95
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU 5130 @ 2.00GHz
stepping : 6
microcode : 0xc6
cpu MHz : 1994.978
cache size : 4096 KB
physical id : 3
siblings : 2
core id : 1
cpu cores : 2
apicid : 7
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dtherm tpr_shadow
bugs :
bogomips : 3990.05
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

my laptop (harmony):
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Core™2 Duo CPU T9400 @ 2.53GHz
stepping : 6
microcode : 0x60f
cpu MHz : 1600.000
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm retpoline kaiser tpr_shadow vnmi flexpriority dtherm
bugs : cpu_meltdown spectre_v1 spectre_v2
bogomips : 5053.99
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Core™2 Duo CPU T9400 @ 2.53GHz
stepping : 6
microcode : 0x60f
cpu MHz : 800.000
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm retpoline kaiser tpr_shadow vnmi flexpriority dtherm
bugs : cpu_meltdown spectre_v1 spectre_v2
bogomips : 5053.99
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

#5

$ . bin/activate
$ deepspeech -h
Illegal instruction

on both machines

#6

No AVX, as confirmed by https://ark.intel.com/products/35562/Intel-Core2-Duo-Processor-T9400-6M-Cache-2_53-GHz-1066-MHz-FSB?q=T9400

You’ll have to rebuild with --copt=-march=native to target your CPU, but with not even SSE4.2 I don’t know how much performances you could expect :slight_smile:

1 Like