Using DeepSpeech without AVX?

Hello,

I’m trying to use DeepSpeech on a small Ubuntu 18.04 Server. I don’t have a dedicated GPU so I went with the CPU version. Soon I found that the bundled tensorflow needs a processor that supports AVX, which my CPU does not support. So I got the “Illegal instruction (core dumped)” exception.
After some digging I found out that I can build tensorflow with optimized settings for my CPU.
Following this thread I managed to build tensorflow and deepspeech. Even the make deepspeech in the native_client works.
But sadly when I run
deepspeech --model ~/DeepSpeech/models/output_graph.pb --lm ~/DeepSpeech/models/lm.binary --trie ~/DeepSpeech/models/trie --audio ~/DeepSpeech/models/2830-3980-0043.wav
I still get the Illegal instruction core dumped error.

And running:
ARGS="~/DeepSpeech/models/output_graph.pb ~/DeepSpeech/models/2830-3980-0043.wav ~/DeepSpeech/models/alphabet.txt ~/DeepSpeech/models/lm.binary ~/DeepSpeech/models/trie" make run

generates:
~/DeepSpeech/native_client$ ARGS="~/DeepSpeech/models/output_graph.pb ~/DeepSpeech/models/2830-3980-0043.wav ~/DeepSpeech/models/alphabet.txt ~/DeepSpeech/models/lm.binary ~/DeepSpeech/models/trie" make run
LD_LIBRARY_PATH=//tensorflow/bazel-bin/native_client: ./deepspeech ~/DeepSpeech/models/output_graph.pb ~/DeepSpeech/models/2830-3980-0043.wav ~/DeepSpeech/models/alphabet.txt ~/DeepSpeech/models/lm.binary ~/DeepSpeech/models/trie
Usage: ./deepspeech --model MODEL [–lm LM --trie TRIE] --audio AUDIO [-t] [-e]

Running DeepSpeech inference.

--model MODEL		Path to the model (protocol buffer binary file)
--lm LM			Path to the language model binary file
--trie TRIE		Path to the language model trie file created with native_client/generate_trie
--audio AUDIO		Path to the audio file to run (WAV format)
--beam_width BEAM_WIDTH	Value for decoder beam width (int)
--lm_alpha LM_ALPHA	Value for language model alpha param (float)
--lm_beta LM_BETA	Value for language model beta param (float)
-t			Run in benchmark mode, output mfcc & inference time
--extended		Output string from extended metadata
--json			Extended output, shows word timings as JSON
--stream size		Run in stream mode, output intermediate results
--help			Show help
--version		Print version and exits
TensorFlow: v1.14.0-21-ge77504ac6b
DeepSpeech: v0.6.1-0-g3df20fee
Makefile:29: recipe for target 'run' failed
make: *** [run] Error 1

Has anyone any idea how I can narrow this error down?

Are you sure this deepspeech binary is the one your built ? What flags did you end up using ?

Just a side note, I’m not sure how much performances you can expect from a CPU that does not support AVX.

I used
-march=native -mssse3 -mcx16 -msse4.1 -msse4.2 -mpopcnt

First I build tensorflow with:
bazel build //tensorflow/tools/pip_package:build_pip_package

then I hope I build deepspeech with:
bazel build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --config=monolithic -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --copt=-fvisibility=hidden //native_client:libdeepspeech.so

Have you ever read our documentation ? Please do so, you don’t need that kind of step.

--copt=-march=native should be enough, but that will also depend on your GCC version.

If you used that command line, then you have not compiled with march=native

bazel build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --config=monolithic -c opt --copt=-march=native --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --copt=-fvisibility=hidden //native_client:libdeepspeech.so

That makes sense. Thank you very much.
I will update on the result.

So first of all the build was successful. But unfortunately running : make deepspeech results in:

ARGS="~/DeepSpeech/models/output_graph.pb ~/DeepSpeech/models/2830-3980-0043.wav ~/DeepSpeech/models/alphabet.txt ~/DeepSpeech/models/lm.binary ~/DeepSpeech/models/trie" make run
c++   -std=c++11 -o deepspeech 'pkg-config --cflags sox' client.cc  -Wl,--no-as-needed -Wl,-rpath,\$ORIGIN -L/<user_home>/tensorflow/bazel-bin/native_client  -ldeepspeech  'pkg-config --libs sox'
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
/tmp/ccDN401d.o: In function 'GetAudioBuffer(char const*, int)':
client.cc:(.text+0x640): undefined reference to 'sox_open_read'
client.cc:(.text+0x725): undefined reference to 'sox_open_memstream_write'
client.cc:(.text+0x7bf): undefined reference to 'sox_create_effects_chain'
client.cc:(.text+0x805): undefined reference to 'sox_find_effect'
client.cc:(.text+0x80d): undefined reference to 'sox_create_effect'
client.cc:(.text+0x837): undefined reference to 'sox_effect_options'
client.cc:(.text+0x882): undefined reference to 'sox_add_effect'
client.cc:(.text+0x8c0): undefined reference to 'sox_find_effect'
client.cc:(.text+0x8c8): undefined reference to 'sox_create_effect'
client.cc:(.text+0x8e8): undefined reference to 'sox_effect_options'
client.cc:(.text+0x933): undefined reference to 'sox_add_effect'
client.cc:(.text+0x971): undefined reference to 'sox_find_effect'
client.cc:(.text+0x979): undefined reference to 'sox_create_effect'
client.cc:(.text+0x999): undefined reference to 'sox_effect_options'
client.cc:(.text+0x9e4): undefined reference to 'sox_add_effect'
client.cc:(.text+0xa22): undefined reference to 'sox_find_effect'
client.cc:(.text+0xa2a): undefined reference to 'sox_create_effect'
client.cc:(.text+0xa54): undefined reference to 'sox_effect_options'
client.cc:(.text+0xa9f): undefined reference to 'sox_add_effect'
client.cc:(.text+0xaea): undefined reference to 'sox_flow_effects'
client.cc:(.text+0xaf9): undefined reference to 'sox_delete_effects_chain'
client.cc:(.text+0xb08): undefined reference to 'sox_close'
client.cc:(.text+0xb17): undefined reference to 'sox_close'
/tmp/ccDN401d.o: In function 'main':
client.cc:(.text+0x13a4): undefined reference to 'sox_init'
client.cc:(.text+0x167c): undefined reference to 'sox_quit'
collect2: error: ld returned 1 exit status
Makefile:22: recipe for target 'deepspeech' failed
make: *** [deepspeech] Error 1

I reinstalled pkg-config with brew install pkg-config with no effect. Also TFDIR is set to the tensorflow dir.

Did I miss something again?

Again, the docs ?

pkg-config needs to be in your PATH, and brew’s sox needs to be installed.
As documented, TFDIR might need to be overridden to your tensorflow’s path if you have changed its name.