Is deepspeech (nativeclient) thread-safe?

DeepSpeech
#1

If I’m going to convert multiple WAV files simultaneously, there’re different ways to do it: multi-process and multi-thread.

The questions is:

  1. Is DS native client thread-safe? E.g. Can I initialize (with LM) once and then create multiple threads to decode multiple streams? Is there anything I should lock among threads?

  2. If it’s not thread-safe, can I initialize once and then fork multiple processes to decode multiple streams?

If both answers are negative, every new forked process has to perform the redudant initialization before decoding a new stream. Thanks!

#2

It should be thread-safe as long as you don’t cross the streams (aka StreamingState*). The only catch is that in the GPU-enabled clients, TensorFlow will allocate the entire GPU RAM by default, so the second instance will run into OOMs.

For batch decoding of multiple WAV files, take a look at evaluate.py, which is written for that use case. Maybe it’s a better fit for you.

#3

Thanks, Reuben!

It’s for something like a web server, which might receive quite a few requests at the same time. So I’m going to maintain a thread/process pool to decode multiple streams simultaneously.

Just took a look at evaluate.py, which decodes a list of files in batch mode. It seems not easy to modify it to support the above use case.

#4

You might want to take a look at https://github.com/mozilla/DeepSpeech/issues/838