Android tflite model inference issues

DeepSpeech
#1

@lissyx I had exported the tflite model from https://github.com/mozilla/DeepSpeech/releases/download/v0.4.1/deepspeech-0.4.1-checkpoint.tar.gz and used it on android 8.1 version with deepspeech version 0.4.1. I get the below error when I perform an inference.

Error: Alphabet size does not match loaded model: alphabet has size 28, but model has 2047 classes in its output. Make sure you’re passing an alphabet file with the same size as the one used for training.
Could not create model.

Have you seen this issue? should I pass a different alphabet file for working with the tflite export?

#2

Care to explain a bit more what you did? It seems to have nothing to do with Android nor TFLite. The 2047 classes seems weird …

#3

@sranjeet.visteon How did you proceeded with the export ? How do you run on device ?

#4

@lissyx All I did was reexported the tflite model using the checkpoint @https://github.com/mozilla/DeepSpeech/releases/download/v0.4.1/deepspeech-0.4.1-checkpoint.tar.gz with the options “–notrain --notest --export_tflite --export_dir /model/export/destination” as part of DeepSpeech.py. This generated the .tflite file.

I ran the https://github.com/mozilla/DeepSpeech/releases/download/v0.4.1/native_client.armv7.cpu.android.tar.xz binaries on a RPI3 running android version 8.1 using this tflite model and alphabet file.

Below is the command that used and output was the error that I mentioned previously.

./deepspeech --model models/output_graph.tflite --alphabet models/alphabet.txt --audio models/ted1_mod.wav -t

#5

That’s adventurous …

#6

Your models/alphabet.txt is the one from the release, with 28 chars ? 2047 classes makes no sense at all, and it’s completely inconsistent with what we have on CI …

#7

There’s something going on for sure … I can still run old TFLite exported model, but exporting 0.4.1 results in a segfault for me

#8

Can you share the old TFLite export model? I can give it a quick try to see if this works, so that can confirm that my setup is working properly.

#9

No, I can’t do that, sorry.

#10

That’s definitively what @reuben feared, the ordering of the inputs / outputs nodes is not guaranteed and there’s no mention of it in the doc (or I missed it). Pay attenton to the “Input node name”. In your case, you seems to have an exported model with input_node being properly fed, but then it explains why the output layer is not the good one, and instead of getting the 28-dimensions logits, we get some of the intermediate state.

139|walleye:/ $ LD_LIBRARY_PATH=/data/local/tmp/arm64/ /data/local/tmp/arm64/deepspeech --model /sdcard/deepspeech/ldc93s1.tflite   --alphabet /sdcard/deepspeech/alphabet.txt --audio /data/local/tmp/LDC93S1.wav -t
DeepSpeech: v0.4.1-5-g8dff77e
Input node name: input_node
audio_format=1
num_channels=1
sample_rate=16000
bits_per_sample=16
res.buffer_size=93594
she had your dark suit in greasy wash water all year
cpu_time_overall=2.05880

walleye:/ $ LD_LIBRARY_PATH=/data/local/tmp/arm64/ /data/local/tmp/arm64/deepspeech --model /sdcard/deepspeech/model_0.4.1.tflite  --alphabet /sdcard/deepspeech/alphabet.txt --audio /data/local/tmp/LDC93S1.wav -t
DeepSpeech: v0.4.1-5-g8dff77e
Input node name: previous_state_c
audio_format=1
num_channels=1
sample_rate=16000
bits_per_sample=16
res.buffer_size=93594
Segmentation fault 
139|walleye:/ $
#11

@lissyx is there a way to specify the “Input node name” parameter as part of the re-export command for tflite?

#12

No, it’s just that the TF Lite code we have needs to be fixed.

#13

@sranjeet.visteon Should be dealt with in https://github.com/mozilla/DeepSpeech/pull/1836, you can already apply locally and rebuild if you wish

#14

@sranjeet.visteon In the mean time, this should be Android ARMv7 including the fix: https://queue.taskcluster.net/v1/task/eL288SmeTV6tfU-mRnjzaw/runs/0/artifacts/public/native_client.tar.xz

#15

@lissyx thanks for sharing. I tried this new version, but still get a “segmentation fault”. Is it working ok for you with the re-exported tflite model generated from checkpoint released as part of 0.4.1?

./deepspeech --model models/output_graph.tflite --alphabet models/alphabet.txt --audio models/ted1_mod.wav -t
DeepSpeech: v0.4.1-7-g8226dbc
Segmentation fault (core dumped)

#16

Yes, it’s working for me:

walleye:/ $ LD_LIBRARY_PATH=/data/local/tmp/arm64/ /data/local/tmp/arm64/deepspeech --model /sdcard/deepspeech/model_0.4.1.tflite  --alphabet /sdcard/deepspeech/alphabet.txt --audio /data/local/tmp/LDC93S1.wav -t
DeepSpeech: v0.4.1-7-g8226dbc
audio_format=1
num_channels=1
sample_rate=16000
bits_per_sample=16
res.buffer_size=93594
she haded duck suit and grecy washwor all year
cpu_time_overall=6.43663
#17

Your way of running the native client on Android is weird, you should have to specify LD_LIBRARY_PATH etc.

#18

Yes, I did that. I did not mention it here. Could this be an issue only with ARMv7 version and will a rebuilding of the binaries help address it?

#19

As much as I can test, ARMv7 are good as well. It’s hard to get more insight, your platform is not really what we target …

#20

@sranjeet.visteon Could you try replacing libdeepspeech.so by this one? libdeepspeech.armv7.zip (1.3 MB)