Remember: cross-compilation will re-use TensorFlow build system where we setup and bundle cross-compilation toolchain, and we have a make target to populate a minimal raspbian buster tree for cross-compilation purposes.
I did run this inside the container
I’ve seen the this step, it did succeed while building the container. There wasn’t any problem at that time.
After the container was built, I did try to run the step again, thats where I got my error. I know that this builds for amd64, but I wanted to check that this is working, before setting a target architecture like rpi3. My next steps then would be to extract the .whl file out of the container to be able to install it somewhere else.
The container is mostly here to help some people who needs to use docker for the inference, but if you intend to cross-build or hack stuff, it’s not really a supported setup, so you should prepare yourself to hack and investigate.
Sorry, I might be dumb, but I can’t figure out exactly what you are doing. Please share precise step-by-steps commands you are following …
# 1. Build container
make Dockerfile.build
docker build -f Dockerfile.build -t dsbuild .
# 2. Start container
docker run --rm -it dsbuild
# 3. Build ctc-decoder inside container again
cd native_client/ctcdecode && make NUM_PROCESSES=$(nproc) bindings
I have no idea why it fails this way … And sorry, but I have no time to investigate / repo.
Hmm, I’m not quite convinced that this would be easier 
There is no problem renaming the layers of my network, but my input and output shapes of the intermediate layers are completely different because I did change network architecture. So I’m assuming that I have to change more than a few lines in the above files.
Sorry, I really have no time to dig into the specifics, but TFLite shapes should not be very complicated to adapt, and likely much less headaches than making ds_ctcdecoder cross-build, or having to build it on your hardware directly. Which eventually, never solves the fact that you end up exercising the tensorflow runtime, and not the TFLite one.
I did make some progress, my network now requires tflite-runtime only, but the ds-ctcdecoder is not yet working. I was able to cross-build it and install a copy of it on my raspbian container image, but when I try to import it I get following error:
ImportError: cannot import name '_swigwrapper' from 'ds_ctcdecoder'
I did follow the steps from https://github.com/mozilla/DeepSpeech/issues/3379, built swig on the raspi and replaced the original ds-swig directory with it.
In the build process it seems the wrapper is added, but was built with the wrong architecture:
creating temp_build/temp_build/ds_ctcdecoder-0.10.0a3.dist-info/WHEEL
creating 'dist/ds_ctcdecoder-0.10.0a3-cp37-cp37m-linux_armv7l.whl' and adding 'temp_build/temp_build' to it
adding 'ds_ctcdecoder/__init__.py'
adding 'ds_ctcdecoder/_swigwrapper.cpython-37m-x86_64-linux-gnu.so'
adding 'ds_ctcdecoder/swigwrapper.py'
I did start the build with make TARGET=rpi3 NUM_PROCESSES=$(nproc) bindings and I think the error is related to this build step:
/DeepSpeech/tensorflow/bazel-tensorflow/external/LinaroArmGcc72/bin/arm-linux-gnueabihf-c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,--no-as-needed -march=armv7-a -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -D_GLIBCXX_USE_CXX11_ABI=0 --sysroot /DeepSpeech/multistrap-raspbian-buster -march=armv7-a -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -D_GLIBCXX_USE_CXX11_ABI=0 --sysroot /DeepSpeech/multistrap-raspbian-buster -Wdate-time -D_FORTIFY_SOURCE=2 temp_build/temp_build/swigwrapper_wrap.o -o temp_build/temp_build/ds_ctcdecoder/_swigwrapper.cpython-37m-x86_64-linux-gnu.so first_party.a third_party.a
Do you know if there is some flag I did miss? Or something other I can change?
You are doing something wrong, see the x86-64 here. We have some hack in place in definitions.mk to fix that
It’s related to this: https://github.com/mozilla/DeepSpeech/blob/master/native_client/definitions.mk#L105
But you will likely have to read more of the makefiles and hack around, sorry.
What exactly do I need to change in the line you linked? And shouldn’t it be that line: https://github.com/mozilla/DeepSpeech/blob/master/native_client/definitions.mk#L86?
The wrong .so file is created somewhere in here: https://github.com/mozilla/DeepSpeech/blob/dd4122a04a084a7a60f27f3c592b2a1153dc20b4/native_client/ctcdecode/setup.py#L70
This is called from here: https://github.com/mozilla/DeepSpeech/blob/8f9d6ad02467bdf79e9e150a41fb1ddbd8eabfaa/native_client/ctcdecode/Makefile#L48
The logged cmd is
DISTUTILS_USE_SDK=1 PATH=/DeepSpeech/native_client/ds-swig/bin:/DeepSpeech/tensorflow/bazel-tensorflow/external/LinaroArmGcc72/bin/arm-linux-gnueabihf-:$PATH SWIG_LIB="/DeepSpeech/native_client/ds-swig/share/swig/4.0.2/" AS=/DeepSpeech/tensorflow/bazel-tensorflow/external/LinaroArmGcc72/bin/arm-linux-gnueabihf-as CC=/DeepSpeech/tensorflow/bazel-tensorflow/external/LinaroArmGcc72/bin/arm-linux-gnueabihf-gcc CXX=/DeepSpeech/tensorflow/bazel-tensorflow/external/LinaroArmGcc72/bin/arm-linux-gnueabihf-c++ LD=/DeepSpeech/tensorflow/bazel-tensorflow/external/LinaroArmGcc72/bin/arm-linux-gnueabihf-ld LIBEXE=/DeepSpeech/tensorflow/bazel-tensorflow/external/LinaroArmGcc72/bin/arm-linux-gnueabihf- CFLAGS="-march=armv7-a -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -D_GLIBCXX_USE_CXX11_ABI=0 --sysroot /DeepSpeech/multistrap-raspbian-buster -march=armv7-a -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -D_GLIBCXX_USE_CXX11_ABI=0 --sysroot /DeepSpeech/multistrap-raspbian-buster " LDFLAGS="-Wl,--no-as-needed" PYTHONPATH=/DeepSpeech/multistrap-raspbian-buster/usr/lib/python3.7/:/DeepSpeech/multistrap-raspbian-buster/usr/lib/python3/dist-packages/ NUMPY_INCLUDE=/DeepSpeech/multistrap-raspbian-buster/usr/include/python3.7m/ python ./setup.py build_ext --num_processes 16 --plat-name linux_armv7l
For me the paths and architecture seem to be correct.
Nothing
There is no sysconfig info passed, see the calling code on the python binding: https://github.com/mozilla/DeepSpeech/blob/8f9d6ad02467bdf79e9e150a41fb1ddbd8eabfaa/native_client/python/Makefile#L13
As you can see, no $(PYTHON_SYSCONFIGDATA) on the ctcdecode part (which is not surprising, since we don’t support this).
I guess adding that should be enough, but I can’t guarantee anything since I have no time to actually check.
Thanks. If I add the flag to the Makefile line, it tries to build ds_ctcdecoder/_swigwrapper.cpython-37m-arm-linux-gnueabihf.so, but now it doesn’t find swig anymore.
swig -python -c++ -extranative -o swigwrapper_wrap.cpp swigwrapper.i
unable to execute 'swig': No such file or directory
A few log lines before calling the changed line, swig was working:
swig -version
SWIG Version 4.0.2
Compiled with g++ [armv7l-unknown-linux-gnueabihf]
Do you have an idea for that too?
Found the solution. Had to link the files from ds-swig to /usr/bin/ that I can use swig system wide.
The created ds_ctcdecoder.whl is now working in my raspbian container:)
Will do some cleanup and then share all the build steps…
That’s likely an oversight, it should not be required if you have it under ds-swig or there’s something else messing with your PATH
Nice. Overall, I guess it was not “that” complicated, just a bit of stuff spreads here and there?
Hmm, as someone who had no idea what he was doing, and one solved error resulted in another error, I’m sorry, but I have to disagree with you here 
. Thank you very much for your help:)
I was glad that I did the building from inside a container, because that way I could just restart the container everytime I messed up the system 
The build steps and output files can be found here: https://gitlab.com/Jaco-Assistant/deepspeech-polyglot/-/tree/dspol/extras/misc
Use at your own risk, this is not officially supported from me or the DeepSpeech team.
While this is nice of you and it probably serves as notes for you, I’d really like we warn people this is not something the project supports. It’s not unusual people come here to complain about “this” and we discover later that “this” is not made by us (and is wrong) and cover unsupported things.