Where can I find cross compiling for aarch64?

[sekarpdkt]

@lissyx

Before attempting to cross compile, I want to ensure I am able to natively compile it in my machine locally so that everything works. I was able to successfully compile (without avx support ) and able to import python modules (earlier used to get illegal instruction as pip install deepspeech will install a version requires avx support).

However, I am getting garbage as output.

python3 client.py --model …/data/models/output_graph.pbmm --alphabet …/data/models/alphabet.txt --lm …/data/models/lm.binary --trie …/data/models/trie --audio ./LDC93S1.wav

Loading model from file …/data/models/output_graph.pbmm
TensorFlow: v1.12.0-rc2-11-gbea86c1e88
DeepSpeech: v0.4.0-alpha.0-69-g50d62b8
Loaded model in 0.0198s.
Loading language model from files …/data/models/lm.binary …/data/models/trie
Loaded language model in 0.649s.
Running inference.
te ee i vy sn m u wuw rh mh ki py ay k jih th r zo l ghn axt bags r ogy ng n lu rr b’o yoh up ghm ihy dm mc pez o l rr spu piw ze bahr ayi uor qe
Inference took 7.323s for 2.925s audio file.

I downloaded tire and lm.binary from current master. Even I tried to download it from v0.4.0-aloha-0, but similar result.

As mentioned above, it could be due to mismatch of version.

deepspeech binary also runs, but, produces similar output

./deepspeech --model …/data/models/output_graph.pbmm --alphabet …/data/models/alphabet.txt --lm …/data/models/lm1.binary --trie …/data/models/trie1 --audio ./LDC93S1.wav
TensorFlow: v1.12.0-rc2-11-gbea86c1e88
DeepSpeech: v0.4.0-alpha.0-69-g50d62b8
te ee i vy sn m u wuw rh mh ki py ay k jih th r zo l ghn axt bags r ogy ng n lu rr b’o yoh up ghm ihy dm mc pez o l rr spu piw ze bahr ayi uor qe

[reuben]

Looks like you’re using a model that’s not compatible with master. You need to re-export the v0.3 checkpoint with the code on master, or download from here: https://github.com/reuben/DeepSpeech/releases/tag/v0.2.0-prod-ctcdecode

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

Before attempting to cross compile, I want to ensure I am able to natively compile it in my machine locally so that everything works. I was able to successfully compile (without avx support ) and able to import python modules (earlier used to get illegal instruction as pip install deepspeech will install a version requires avx support).

Then it’s likely what @reuben suggested and you should be able to safely move to cross-compile, by adding the --config flag at build time.

[sekarpdkt]

@lissyx @reuben I checked out v0.3.0 and it worked in my PC.

[sekarpdkt]

When I try to bazel build

bazel build --config=monolithic -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --copt=-mtune=generic --copt=-march=armv8-a --copt=-fvisibility=hidden //native_client:libdeepspeech.so //native_client:generate_trie --verbose_failures --action_env=LD_LIBRARY_PATH=${LD_LIBRARY_PATH} --local_resources 6128,2,1.0

it failed with following error

cc1plus: error: bad value (‘armv8-a’) for ‘-march=’ switch
cc1plus: note: valid arguments to ‘-march=’ switch are: nocona core2 nehalem corei7 westmere sandybridge corei7-avx ivybridge core-avx-i haswell core-avx2 broadwell skylake skylake-avx512 bonnell atom silvermont slm knl x86-64 eden-x2 nano nano-1000 nano-2000 nano-3000 nano-x2 eden-x4 nano-x4 k8 k8-sse3 opteron opteron-sse3 athlon64 athlon64-sse3 athlon-fx amdfam10 barcelona bdver1 bdver2 bdver3 bdver4 znver1 btver1 btver2
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 9254.824s, Critical Path: 341.76s
INFO: 3583 processes: 3583 local.
FAILED: Build did NOT complete successfully

I already installed

root@d07b6a987d17:/# apt-get install -y gcc-aarch64-linux-gnu g+±aarch64-linux-gnu && rm -rf /var/lib/lists/*
Reading package lists… Done
Building dependency tree
Reading state information… Done
g+±aarch64-linux-gnu is already the newest version (4:7.3.0-3ubuntu2.1).
gcc-aarch64-linux-gnu is already the newest version (4:7.3.0-3ubuntu2.1).
0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.

Am I supposed to do something similar to this instead of trying to use above aarch64 gcc?

Cross-compile latest Tensorflow (1.5+) for the Nvidia Jetson TK1

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

I already installed

Not needed, please uninstall.

cc1plus: error: bad value (‘armv8-a’) for ‘-march=’ switch

You are not passing the --config=rpi3-armv8 as I documented earlier.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

I’ve already shared that link @sekarpdkt but please read it again, it defines the --config flags you need: https://github.com/mozilla/tensorflow/blob/r1.12/tools/bazel.rc#L70-L74

Command line should be in the end:

$ bazel build --config=monolithic --config=rpi3-armv8 --config=rpi3-armv8_opt -c opt --copt=-fvisibility=hidden 

To which of course you need to append the build targets, like //native_client:libdeepspeech.so

[sekarpdkt]

Thanks. It is now downloading aarch64 tool chain and building. I will give feedback in couple of hours.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

I will give feedback in couple of hours.

It should not take that long, except if you have some slow network of course. But the build itself should be fast enough, maybe 30 mins ?

[sekarpdkt]

@lissyx

1. Bazel build is successful. Compiled libdeepspeech.so is working in S912 arm box with downloaded deepspeech. I am sure that it uses compiled .so file. If I rename .so file in my S912 box, it gives lib not found error. So it works and it uses compiled .so.

2. make deepspeech as well as make bindings fails with error

root@fa0b0022e017:/DeepSpeech/native_client# make deepspeech
c++ -std=c++11 -o deepspeech pkg-config --cflags sox client.cc -Wl,–no-as-needed -Wl,-rpath,$ORIGIN -L/tensorflow/bazel-bin/native_client -ldeepspeech pkg-config --libs sox
/usr/bin/ld: skipping incompatible /tensorflow/bazel-bin/native_client/libdeepspeech.so when searching for -ldeepspeech
/usr/bin/ld: cannot find -ldeepspeech
collect2: error: ld returned 1 exit status
Makefile:22: recipe for target ‘deepspeech’ failed
make: *** [deepspeech] Error 1
root@fa0b0022e017:/DeepSpeech/native_client#

root@45e60e28c7f0:/DeepSpeech/native_client/python# make bindings
pip install --quiet numpy wheel==0.31.0 setuptools==39.1.0
AS=as CC=gcc CXX=c++ LD=ld CFLAGS=" " LDFLAGS="-Wl,–no-as-needed ‘-Wl,-rpath,$ORIGIN/lib/’ -Wl,-rpath,$ORIGIN" MODEL_LDFLAGS="-L/tensorflow/bazel-bin/native_client " MODEL_LIBS="-ldeepspeech " python ./setup.py build_ext
/usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: ‘long_description_content_type’
warnings.warn(msg)
running build_ext
building ‘deepspeech._impl’ extension
swigging impl.i to impl_wrap.cpp
swig -python -c++ -keyword -o impl_wrap.cpp impl.i
creating temp_build
gcc -DNDEBUG -g -fwrapv -O2 -Wall -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/lib/python3/dist-packages/numpy/core/include -I…/ -I./ -I/usr/include/python3.6m -c impl_wrap.cpp -o temp_build/impl_wrap.o
creating temp_build/deepspeech
c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,–no-as-needed -Wl,-rpath,$ORIGIN/lib/ -Wl,-rpath,$ORIGIN -Wdate-time -D_FORTIFY_SOURCE=2 temp_build/impl_wrap.o -L/tensorflow/bazel-bin/native_client -ldeepspeech -o temp_build/deepspeech/_impl.cpython-36m-x86_64-linux-gnu.so
/usr/bin/ld: skipping incompatible /tensorflow/bazel-bin/native_client/libdeepspeech.so when searching for -ldeepspeech
/usr/bin/ld: cannot find -ldeepspeech
collect2: error: ld returned 1 exit status
error: command ‘c++’ failed with exit status 1
Makefile:10: recipe for target ‘bindings-build’ failed
make: *** [bindings-build] Error 1

Am I supposed to do these in S912 directly?

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

Am I supposed to do these in S912 directly?

No, but you are not supposed to run that as root as well. Check the native_client/definitions.mk there’s a section if about rpi3-armv8. So to cross-compile you should augment your make call with TARGET=rpi3-armv8 and of course with TFDIR if you did not put tensorflow tree at the documented place matching the default value.

Then it should leverage the cross-compilation toolchain you have setup in Bazel and cross-build whatever you want.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

. Compiled libdeepspeech.so is working in S912 arm box with downloaded deepspeech. I am sure that it uses compiled .so file. If I rename .so file in my S912 box, it gives lib not found error. So it works and it uses compiled .so.

Just let me rephrase to make sure I understand: you used our builds for Aarch64 of deepspeech binary with your build of libdeepspeech.so and it works as expected, right? That’s already good news, and if you properly follow the instructions above, then you should be able to cross-compile what you want.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

Oh, forgot to mention you might have to setup a sysroot with multistrap, using the configurations we have at native_client/multistrap_armbian64_stretch.conf. Since you are using a different distribution, though, you might have to perform adaptations. You can always set the sysroot matching your target system using the make variable RASPBIAN (ok, the naming could have been better).

[sekarpdkt]

Yes. Default downloaded Aarch64 binary works with my libdeepspeech.so. Rest of your suggeestion, I will try.

[sekarpdkt]

As I am trying to do it inside a docker, not looked into running as ‘root’. Wrt multistrap sysroot building, got into gpg key error. I am not sure, you will be able to help here. Just need a confirmation that my command it right. Ignore that change in repository URL. I just changed to try

root@fa0b0022e017:/DeepSpeech/native_client# multistrap -a aarch64 -d /DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/ --no-auth -f multistrap_armbian64_stretch.conf
multistrap 2.2.9 using multistrap_armbian64_stretch.conf
multistrap 2.2.9 using multistrap_armbian64_stretch.conf
Using foreign architecture: aarch64
multistrap building aarch64 multistrap on ‘amd64’
Getting package lists: APT_CONFIG=/tmp/multistrap.5gUxn1 apt-get -o Apt::Architecture=aarch64 -o Dir::Etc::TrustedParts=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/etc/apt/trusted.gpg.d -o Dir::Etc::Trusted=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/etc/apt/trusted.gpg -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true -o Apt::Install-Recommends=false -o Dir=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/ -o Dir::Etc=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/etc/apt/ -o Dir::Etc::Parts=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/etc/apt/apt.conf.d/ -o Dir::Etc::PreferencesParts=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/etc/apt/preferences.d/ -o APT::Default-Release=’*’ -o Dir::State=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/var/lib/apt/ -o Dir::State::Status=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/var/lib/dpkg/status -o Dir::Cache=/DeepSpeech/multistrap-raspbian64-stretch/usr/lib/aarch64-linux-gnu/var/cache/apt/ update
Ign:1 http://ftp.uk.debian.org/debian stretch InRelease
Get:2 http://ftp.uk.debian.org/debian stretch Release [118 kB]
Get:3 http://ftp.uk.debian.org/debian stretch Release.gpg [2434 B]
Ign:3 http://ftp.uk.debian.org/debian stretch Release.gpg
Reading package lists… Done
W: GPG error: http://ftp.uk.debian.org/debian stretch Release: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY 8B48AD6246925553 NO_PUBKEY 7638D0442B90D010 NO_PUBKEY EF0F382A1A7B6500
E: The repository ‘http://ftp.uk.debian.org/debian stretch Release’ is not signed.
N: Updating from such a repository can’t be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
apt update failed. Exit value: 100
root@fa0b0022e017:/DeepSpeech/native_client#

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

got into gpg key error

ah that might come from newer releases of multistrap. Often, I just copy the keys from my host system, /etc/apt/trusted.gpg.d/ into the multistrap directory’s /DeepSpeech/multistrap-raspbian64-stretch/etc/apt/trusted.gpg.d/

Why do you force -a aarch64, it should not be needed? Also the -d is the sysroot target, so should just be /DeepSpeech/multistrap-raspbian64-stretch/

[sekarpdkt]

rpi3-armv8

cp /etc/apt/trusted.gpg.d/* /DeepSpeech/multistrap-raspbian64-stretch/etc/apt/trusted.gpg.d/

followed by other steps did the trick. deepspeech is compiled and it works in target. This is really very good news. However, final step of building wheel has some issue. Wheel got built successfully and even got installed in my S912 box. But client.py execution gives some error like

sekar@aml:~/deepspeech/bin/v0.3.0$ python3 client.py --model ./models/output_graph.pbmm --alphabet ./models/alphabet.txt --lm ./models/lm.binary --trie ./models/trie --audio ./LDC93S1.wav
Traceback (most recent call last):
File “/home/sekar/.local/lib/python3.6/site-packages/deepspeech/impl.py”, line 14, in swig_import_helper
return importlib.import_module(mname)
File “/usr/lib/python3.6/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 994, in _gcd_import
File “”, line 971, in _find_and_load
File “”, line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named ‘deepspeech._impl’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “client.py”, line 12, in
from deepspeech import Model, printVersions
File “/home/sekar/.local/lib/python3.6/site-packages/deepspeech/init.py”, line 4, in
from deepspeech.impl import AudioToInputVector as audioToInputVector
File “/home/sekar/.local/lib/python3.6/site-packages/deepspeech/impl.py”, line 17, in
_impl = swig_import_helper()
File “/home/sekar/.local/lib/python3.6/site-packages/deepspeech/impl.py”, line 16, in swig_import_helper
return importlib.import_module(’_impl’)
File “/usr/lib/python3.6/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named ‘_impl’
sekar@aml:~/deepspeech/bin/v0.3.0$

I am going to start right from rebuilding docker to enure,

1. Target is bionic, instead of stretch by creating a multistrap conf for target bionic with libpython3.6 instead of 3.5, Let me try this.

However as seen earlier all issues were fixed by simple configs, if you have any idea will be great.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

python3 client.py --model ./models/output_graph.pbmm --alphabet ./models/alphabet.txt --lm ./models/lm.binary --trie ./models/trie --audio ./LDC93S1.wav

Run that with LD_DEBUG=libs:

$ LD_DEBUG=libs python3 client.py --model ./models/output_graph.pbmm --alphabet ./models/alphabet.txt --lm ./models/lm.binary --trie ./models/trie --audio ./LDC93S1.wav

This might give us hints of what is missing.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

Also, can you zipinfo -l against the wheel you built to check what is packaged inside?

[sekarpdkt]

zipinfo -l is having something interesting deepspeech/_impl.cpython-36m-x86_64-linux-gnu.so

sekar@aml:~/deepspeech/bin/v0.3.0$ zipinfo -l deepspeech-0.3.0-cp36-cp36m-linux_aarch64.whl
Archive: deepspeech-0.3.0-cp36-cp36m-linux_aarch64.whl
Zip file size: 103790 bytes, number of entries: 9
-rw-r–r-- 2.0 unx 1670 b- 509 defN 18-Nov-15 17:16 deepspeech/init.py
-rwxr-xr-x 2.0 unx 249432 b- 90349 defN 18-Nov-26 01:42 deepspeech/_impl.cpython-36m-x86_64-linux-gnu.so
-rw-r–r-- 2.0 unx 4355 b- 1735 defN 18-Nov-15 17:16 deepspeech/client.py
-rw-r–r-- 2.0 unx 4509 b- 1351 defN 18-Nov-26 01:42 deepspeech/impl.py
-rw-r–r-- 2.0 unx 55 b- 48 defN 18-Nov-26 01:42 deepspeech-0.3.0.dist-info/entry_points.txt
-rw-r–r-- 2.0 unx 11 b- 13 defN 18-Nov-26 01:42 deepspeech-0.3.0.dist-info/top_level.txt
-rw-r–r-- 2.0 unx 105 b- 101 defN 18-Nov-26 01:42 deepspeech-0.3.0.dist-info/WHEEL
-rw-r–r-- 2.0 unx 23463 b- 7923 defN 18-Nov-26 01:42 deepspeech-0.3.0.dist-info/METADATA
-rw-r–r-- 2.0 unx 757 b- 473 defN 18-Nov-26 01:42 deepspeech-0.3.0.dist-info/RECORD
9 files, 284357 bytes uncompressed, 102502 bytes compressed: 64.0%
sekar@aml:~/deepspeech/bin/v0.3.0$