Deepspeech python : Illegal instruction (core dumped) with Jetson Nano 2 GB

DeepSpeech
#1

Hello to all,

I have a problem with deepspeech python API. I tryed to used this exemple :

I made it work on my computer which is on Ubuntu 20, everything is working well.

I need to do the same now on a Jetson Nano 2GB (Ubuntu), but i’m stuck with an import deepspeech error which make the mic_vad_streaming failed.

I followed this :

I got this with gdb :

(env) prd@prd-desktop:~/PRD$ gdb --args python src/mic_vad_streaming.py
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright © 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type “show copying”
and “show warranty” for details.
This GDB was configured as “aarch64-linux-gnu”.
Type “show configuration” for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type “help”.
Type “apropos word” to search for commands related to “word”…
Reading symbols from python…(no debugging symbols found)…done.
(gdb) run
Starting program: /home/prd/PRD/env/bin/python src/mic_vad_streaming.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library “/lib/aarch64-linux-gnu/libthread_db.so.1”.
2021-01-25 18:51:53.922438: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2

Program received signal SIGILL, Illegal instruction.
0x0000007fb0172f54 in gotoblas_dynamic_init () from /home/prd/PRD/env/lib/python3.6/site-packages/numpy/core/…/…/numpy.libs/libopenblasp-r0-32ff4d91.3.13.so
(gdb)

If you need :

(env) prd@prd-desktop:~/PRD$ cat /proc/cpuinfo
processor : 0
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 1
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 2
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 3
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

(env) prd@prd-desktop:~/PRD$ uname -a
Linux prd-desktop 4.9.140-tegra #1 SMP PREEMPT Tue Oct 27 21:02:37 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux

(env) prd@prd-desktop:~/PRD$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION=“Ubuntu 18.04.5 LTS”

If you could help me !

#2

We don’t support this platform with CUDA. Please repro with our Linux/Aarch64 TFLite only.

Might be NumPY ABI mismatch. Not our builds, we can’t help.

#3

Maybe @dkreutz can provide support?

#4

Hi @natdop,
I am the maintainer of the DeepSpeech-for-Jetson-Nano repo (my GitHub nickname is ‘domcross’).
What I understand so far is you try to run the DeepSpeech-examples on a Jetson Nano 2GB and get some strange error.
I have just noticed that DS-examples require DeepSpeech 0.9.3 while my latest build for Jetson/Xavier is 0.9.2. I will try to build 0.9.3 now, but that will take a few hours/day. In the meantime you resolve the openblas-error by sudo apt-get install libopenblas-base libopenblas-dev

#5

Hi, thank you for your answers !

What I understand so far is you try to run the DeepSpeech-examples on a Jetson Nano 2GB and get some strange error.

Indeed, the exemple failed when it tryed to import DeepSpeech. So the first lines :cold_sweat:

I have just noticed that DS-examples require DeepSpeech 0.9.3 while my latest build for Jetson/Xavier is 0.9.2.

Well, it’s good that you notify it I hadn’t seen that :sweat_smile:

In the meantime you resolve the openblas-error by sudo apt-get install libopenblas-base libopenblas-dev

I’ll try this to see if it solves the import problem.

I will try to build 0.9.3 now, but that will take a few hours/day.

That would be awsome ! I’ve heard from my research here GPU Inference on Jetson Nano that it’s not an easy thing to do :laughing: and I’m a complete novice in the field.

#6

If anyone is willing to contribute cross-compilation and support, we would welcome PR, just saying’.

#7

The python wheel for DS v0.9.3 for Jetson/Xavier is ready now, see here or go directly to my github-repo release page here

1 Like
#8

also, how much do you trust your setup of numpy? Is it possible you are using a binary not properly built for your specific arch, hence the SIGILL ?

#9

also, how much do you trust your setup of numpy? Is it possible you are using a binary not properly built for your specific arch, hence the SIGILL ?

I really don’t know, but I think it’s the probleme… But @dkreutz can run it so I guess I should to

In the meantime you resolve the openblas-error by sudo apt-get install libopenblas-base libopenblas-dev

I did this but I still got the same illegal :frowning:
0000007fb016ff54 in gotoblas_dynamic_init ()

The python wheel for DS v0.9.3 for Jetson/Xavier is ready now

I used this one now, thank you !

#10

Ok, I found it :

It works well with numpy 1.19.4 ^^

1 Like
#11

Ok, so thank you the mic_vad exemple is working fine to !
Maybe @dkreutz you should prevent that numpy 1.19.5 is not working and we should install the previous version :), and thank you again for the build !

Well, That would take me so much time to understand how to do it, but maybe :slight_smile:

#12

Thanks for confirmation.

I have added a note on the Numpy 1.19.5 issue in my release notes.

1 Like