Tflite Model giving rubbish output during inference

Weird. What device / arch is this? I’ll recheck but CI is green so I doubt we have a bug on our side.

Checked https://bintray.com/alissy/org.mozilla.deepspeech/download_file?file_path=org%2Fmozilla%2Fdeepspeech%2Flibdeepspeech%2F0.6.1%2Flibdeepspeech-0.6.1.aar and:

$ zipinfo -l  libdeepspeech-0.6.1.aar
Archive:  libdeepspeech-0.6.1.aar
Zip file size: 4936641 bytes, number of entries: 20
-rw-r--r--  2.0 unx      327 b-      196 defN 80-Feb-01 00:00 AndroidManifest.xml
-rw-r--r--  2.0 unx       31 b-       33 defN 80-Feb-01 00:00 R.txt
-rw-r--r--  2.0 unx    13386 b-    11204 defN 80-Feb-01 00:00 classes.jar
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 res/
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 res/values/
-rw-r--r--  2.0 unx      114 b-       95 defN 80-Feb-01 00:00 res/values/values.xml
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 jni/
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 jni/arm64-v8a/
-rw-r--r--  2.0 unx   219216 b-    68801 defN 80-Feb-01 00:00 jni/arm64-v8a/libdeepspeech-jni.so
-rw-r--r--  2.0 unx  4540144 b-  1623613 defN 80-Feb-01 00:00 jni/arm64-v8a/libdeepspeech.so
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 jni/armeabi-v7a/
-rw-r--r--  2.0 unx   104076 b-    45696 defN 80-Feb-01 00:00 jni/armeabi-v7a/libdeepspeech-jni.so
-rw-r--r--  2.0 unx  2834512 b-  1295614 defN 80-Feb-01 00:00 jni/armeabi-v7a/libdeepspeech.so
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 jni/x86_64/
-rw-r--r--  2.0 unx   223576 b-    74040 defN 80-Feb-01 00:00 jni/x86_64/libdeepspeech-jni.so
-rw-r--r--  2.0 unx  5072784 b-  1815103 defN 80-Feb-01 00:00 jni/x86_64/libdeepspeech.so
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 x86_64/
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 values/
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 arm64-v8a/
drwxr-xr-x  2.0 unx        0 b-        2 defN 80-Feb-01 00:00 armeabi-v7a/

So libdeepspeech-jni.so is here.

Thank you @lissyx
Issue of crashing resolved now. But now i am still facing inference issue. i dont get any inference on my audio file like the output is blank when i do inference in android using tflite while if i use your given audio it does gives output. Is there any change in wav file properties for tflite? I am using wavfile 16 bit mono 16khz . And my audio file is not blank as i get correct inference of the same file with output.pb file .

Using

TensorFlow: v1.14.0-21-ge77504a
DeepSpeech: v0.6.1-0-g3df20fe

I tried tflite file on desktop platform i.e linux by installing deepspeech-tflite library and it is giving correct inference on the same wav file where it fails to give any output in the android. I think so the problem is in android AAR file. As my tflite is working well on desktop and also on any of my wav file where in android it gives blank as an inference. Can you please check what is going wrong while running on android. I need to test the model on android. Thanking you in advance.

Obviously you don’t test with the same code, do you ?

Can you please ensure your code on Android is good ? You obviously don’t exercize the same way. We have tests checking that in CI.

Didnt get you. Sorry please elaborate

I am not using any other code from my side. I am using your sample code for android and aar file which you have shared.

And you are using your own file. How are you testing on desktop ? Please ?

@sanjay.pandey We publish examples files for a reason, does it works with those files ?

@sanjay.pandey Just re-verified from scratch on my Google Pixel 2 device, both our app and https://github.com/mozilla/androidspeech/ works 100%.

After installing deepspeech-tflite on v0.6.1 I ran following command

deepspeech --model /home/sanjjayyp/export_model_0.6.1/output_graph.tflite --lm data/lm/lm.binary --trie data/lm/trie --audio /home/sanjjayyp/sapna_apna/cheese_mysore_masala_dosa_idli_vada_sambhar.wav

and it is giving inference as
cheese mysore masala dosa idli vada sambhar
which is correct.

Also i am trying with evaluate_tflite.py as well. Currently it is running for 16k audio yet to get the result.

As I already told you several times, this Android code is not really intendend for real use: https://github.com/mozilla/DeepSpeech/blob/master/native_client/java/app/src/main/java/org/mozilla/deepspeech/DeepSpeechActivity.java#L68-L102

We read the WAVE content very simply. Any malformed file will fail. Badly.

It works with those 3 file but not with any of my wav file .

One thing i observed that

Length of Byte array for our audio file is always : 26

Length of Byte array for your audio file is always : 87520

See ? Just what I told you. When did you observed that ? Your file is just bogus.

But why the same file working on desktop with tflite if it is malformed what is the reason for that and how i can overcome? i have around 7 lacs audio file i have randomly tried inference with different file of mine on android but result is always same i.e blank

It means your WAVE file has a different header format, the Android code only supports the “SoX format”. Are you using ffmpeg anywhere to generate/process this file? If so, you’ll need to use this option: https://stackoverflow.com/a/39671913/346048

Or you can make a PR to our Android code to add support for other formats, for example by using a SoX or ffmpeg library.

As @reuben said, they likely are all broken the same way.

As linked, because we read directly the file on Android for simplicity (less deps, not supposed to be any user-facing app) while on desktop we use more generic tools.

Understood what was going wrong. I used sox to convert audio into 16 bit mono 16 khz and now it is working fine and i am getting output on android. So actually python file was already using SoX for all the audio i trained on and hence i was not receiving any error on desktop or in python.

Thanks a lot @reuben @lissyx for the help. but the inference is too slow . For 5 seconds audio it took 10 seconds. Shouldnt tflite be fast?