Deepspeech : undefined reference to `lzma_auto_decoder'

DeepSpeech
#1

Hello , i have this error that i’m not sure what is producing it , i spent 4 days trying to install every dependency i can think of and i have no idea what to do , can someone help me please , i’m new to deepspeech :frowning:

make deepspeech
c++   -std=c++11 -o deepspeech `pkg-config --cflags sox` -fopenmp client.cc  -Wl,--no-as-needed -Wl,-rpath,\$ORIGIN -L/home/anes/tensorflow/bazel-bin/native_client  -ldeepspeech  -Wl,-Bstatic `pkg-config --static --libs sox` -lgsm `pkg-config --static --libs libpng | cut -d' ' -f1` -lz -lmagic -lltdl -Wl,-Bdynamic -ldl
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libmagic.a(compress.o): in function `uncompressbuf':
(.text+0x88a): undefined reference to `lzma_auto_decoder'
/usr/bin/ld: (.text+0x8ed): undefined reference to `lzma_code'
/usr/bin/ld: (.text+0x90e): undefined reference to `lzma_end'
/usr/bin/ld: (.text+0xa85): undefined reference to `BZ2_bzDecompressInit'
/usr/bin/ld: (.text+0xad9): undefined reference to `BZ2_bzDecompress'
/usr/bin/ld: (.text+0xafb): undefined reference to `BZ2_bzDecompressEnd'
collect2: error: ld returned 1 exit status
make: *** [Makefile:22: deepspeech] Error 1
#2

You lack lzma. Don’t ask my why, I don’t know.

#3

Ran : sudo apt-get install -y lzma
Still the same output error…

#4

Did you add proper linkage flags? What distro is this? Why do you need to rebuild yourself? Which dependency requires lzma, sox?

#5

Hey sorry for the late reply , i’m not sure i’m following what you mean when you say " Linkage flags " , my distro is 20.04 LTS , ah i’m following the french model training tutorial , and i’m not entirely sure what depends on lzma :confused:

#6

What french model training tutorial ? You can use our prebuilt binaries on Ubuntu 20.04,
no need to rebuild yourself, especially if you don’t know how to build …

Le dim. 7 juin 2020 à 01:03, Anes Hmida via Mozilla Discourse notifications@discourse.mozilla.org a écrit :

#7

Also, again, can you explicit your steps? I use Ubuntu 20.04 and I dont have this issue, and I dont have any lzma linkage either, so I’d like to understand …

Le dim. 7 juin 2020 à 01:50, Alexandre Lissy alissy@mozilla.com a écrit :

#9

Dont downgrade, just install python3.6 package and use a virtualenv based on that. If you are looking to produce a french model, can you please join efforts ?

https://github.com/Common-Voice/commonvoice-fr/blob/master/DeepSpeech/Dockerfile.train

This is still 0.6-based because I lack of time to update it but that should be done soon. On the releases page of that project you can also find trained french model.

Le dim. 7 juin 2020 à 03:22, Anes Hmida via Mozilla Discourse notifications@discourse.mozilla.org a écrit :

#10

Hello again , i will try to contribute once i get this up and running !
Btw thank you for the tip , i got everything setup correctly following the latest documentation , and now when i try to train my files using my own .sh :

#!/bin/sh
set -xe
if [ ! -f DeepSpeech.py ]; then
    echo "Please make sure you run this from DeepSpeech's top level directory."
    exit 1
fi;

python3 -u DeepSpeech.py \
  --train_files /home/anes/DeepSpeech/STT/Train/train.csv \
  --dev_files /home/anes/DeepSpeech/STT/Dev/dev.csv \
  --test_files /home/anes/DeepSpeech/STT/Test/test.csv \
  --train_batch_size 80 \
  --dev_batch_size 80 \
  --test_batch_size 40 \
  --n_hidden 375 \
  --epochs 33 \
  --early_stop True \
  --dropout_rate 0.22 \
  --learning_rate 0.00095 \
  --report_count 100 \
  --scorer_path /home/anes/DeepSpeech/STT/deepspeech-0.7.3-models.scorer \
  --export_dir /home/anes/DeepSpeech/STT/Result/ \
  --checkpoint_dir /home/anes/DeepSpeech/STT/Result/checkout/ \
  --alphabet_config_path /home/anes/DeepSpeech/STT/alphabet.txt \
  "$@"

, i get this strange error :

raise RuntimeError('No transcript data (missing CSV column)')
RuntimeError: No transcript data (missing CSV column)

Even though all of my CSV’s are perfect and everything is in place , i’m i doing something wrong ?

#11

Hard to tell without more informations, but I’d suspect your CSV is still improperly formatted. Were there generated using our importers or from other source?

Le dim. 7 juin 2020 à 13:40, Anes Hmida via Mozilla Discourse notifications@discourse.mozilla.org a écrit :

#12

upstream allows linking against lzma or bz2 since 5.38: https://github.com/file/file/commit/b259a07ea95827f565faa20f0316e5b2704064f7, which seems to be done on debian / ubuntu

when building current master, it should now check libmagic.so on linux/amd64 platforms and detect if linkage is required.