Missing libdeepspeech_utils.so

DeepSpeech
#1

Hi,

I have configured and installed two Ubuntu 16.04 environments for training with DeepSpeech in the last weeks.

Both environments with the same steps as before:

  1. Git clone DeepSpeech
  2. Install requirements
  3. Instal python bindings
    $ pip install deepspeech
  4. Install libraries (and make them available in the LD_LIBRARY_PATH)
    $ ./utils/taskcluster.py --target .
  5. Training run with DeepSpeech.py

The first one was working fine as usual.
But the second one, failed running the training.

The errors were :

/DeepSpeech/util/audio.py:17: DeepSpeechDeprecationWarning: DeepSpeech Python bindings could not be imported, resorting to slower code to compute audio features. Refer to README.md for instructions on how to install (or build) the DeepSpeech Python bindings.
  category=DeepSpeechDeprecationWarning)

Traceback (most recent call last):
  File "./DeepSpeech.numcep.py", line 1952, in <module>
    tf.app.run()
  File "/home/ubuntu/miniconda3/envs/deep-speech/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "./DeepSpeech.numcep.py", line 1909, in main
    train()
  File "./DeepSpeech.numcep.py", line 1558, in train 
    results_tuple, gradients, mean_edit_distance, loss = get_tower_results(model_feeder, optimizer)
  File "./DeepSpeech.numcep.py", line 669, in get_tower_results
    calculate_mean_edit_distance_and_loss(model_feeder, i, dropout_rates)
  File "./DeepSpeech.numcep.py", line 562, in calculate_mean_edit_distance_and_loss
    decoded, _ = decode_with_lm(logits, batch_seq_len, merge_repeated=False, beam_width=FLAGS.beam_width)
  File "./DeepSpeech.numcep.py", line 517, in decode_with_lm
    top_paths=top_paths, merge_repeated=merge_repeated))
TypeError: ctc_beam_search_decoder_with_lm() got an unexpected keyword argument 'word_count_weight'

I noticed that the libdeepspeech_utils.so is not anymore in the native_client.tar.xz package, but the python bindings seems to keep wanting it. Even more the ctc_beam_search_decoder_with_lm function seems to be different as expected.

Indeed using the native_client_tar.xz package from the first environment solved the problem.

I have also noticed that in the native_client compilation README.md the //native_client:deepspeech_utils target has dissappear as well.

So, do I have to download something else installing other environments from now on? Or is just a temporary version mismatch between python bindings, DeepSpeech.py and taskcluster?.

Best regards,
Mar

#2

Looks like you are using something specific: DeepSpeech.numcep.py is not our code. And you are passing some unsupported parameter. It’s not related to libdeepspeech_utils.so. Check that comment: https://github.com/mozilla/DeepSpeech/pull/1497#issuecomment-412522301

#3

Sorry, DeepSpeech.numcep.py is just the same DeepSpeech.py cloned from github a few days ago but receiving n_mfcc as a parameter :slight_smile: because I am training with 8KHz audio.

So about the WORD_COUNT_WEIGHT I understand it has to be removed, and also the libdeepspeech_utils.so.
Then I will get the new DeepSpeech.py and utils/audio.py from github, and upgrade the bindings.

Thanks

P.S. Every time I upgrade the DeepSpeech software I have to re-add the n_mfcc optional parameter not only to DeepSpeech.py but also to native_client/client.cc (and args.h), aside to beam_width and sample_rate as optional parameters too. Do you agree about me contributing with those small changes to your software?, because maybe anyone else can need them…

#4

I cannot promise anything, but open a PR and we can discuss that :slight_smile:

#5

Ok, I will put a PR :slight_smile:

I got the newest code, install bindings and executed taskcluster successfully:

$ python3 util/taskcluster.py --arch gpu --target .

But I keep getting the warning at training time:

DeepSpeech/util/audio.py:17: DeepSpeechDeprecationWarning: DeepSpeech Python bindings could not be imported, resorting to slower code to compute audio features. Refer to README.md for instructions on how to install (or build) the DeepSpeech Python bindings.
category=DeepSpeechDeprecationWarning)

The training works, but anyway it is good to know what’s going on.

About my environment:

$ uname -a
Linux jose-casa 4.15.0-30-generic #32-Ubuntu SMP Thu Jul 26 17:42:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ echo $LD_LIBRARY_PATH
/tools/DeepSpeech:/usr/local/cuda/extras/CUPTI/lib64

$ ls -al /tools/DeepSpeech/*.so

-r-xr-xr-x 1 jose jose 478553 ago 12 04:35 /tools/DeepSpeech/libctc_decoder_with_kenlm.so

-r-xr-xr-x 1 jose jose 417188316 ago 12 04:17 /tools/DeepSpeech/libdeepspeech.so

$ python3 --version

Python 3.6.6

$ pip3 list | grep deepspeech

deepspeech-gpu 0.1.1

And a command line experiment:

$ python3 -c “from deepspeech import audioToInputVector”

Traceback (most recent call last):

File “<string>”, line 1, in <module>

ImportError: cannot import name ‘audioToInputVector’

What am I missing?

Tanks in advance,
Mar

#6

So you have deepspeech-gpu==0.1.1 installed, your import should be done from deepspeech.utils and not just deepspeech.

If you want to do from deepspeech import audioToInputVector then you need deepspeech-gpu>=0.2.0a9

#7

Yes, in the fresh git clone file util/audio.py file this line is not ok:

from deepspeech import audioToInputVector

I’ve changed to this, and it works:

from deepspeech.utils import audioToInputVector

Thanks

#8

So now it’s all good @mar_martinez ?

#9

Yes, thanks a lot for your help.