Creating tflite file for Android

[Tuomas]

How do you convert custom trained .pb file to .tflite file?

import tensorflow as tf

Convert the model.

converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(
graph_def_file=’/home/tuomas/Desktop/output_graph.pb’,
# both .pb and .pbtxt files are accepted.
input_arrays=[‘input_node’, ‘input_samples’, ‘input_lengths’, ‘previous_state_c’, ‘previous_state_h’],
output_arrays=[‘mfccs’, ‘logits’, ‘new_state_h’, ‘new_state_c’]
)
converter.target_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()

Save the model.

with open(‘model.tflite’, ‘wb’) as f:
f.write(tflite_model)

Above python converter produces following error: Converting unsupported operation: BlockLSTM

If deepspeech uses LSTM and you are not able to convert .pb files with LSTM to .tflite, can you deploy custom model on Android at all?

Pretrained model with android_mic_streaming is working fine, but unable to convert my own model.

[Tuomas]

Okey. I used --export_tflite flag in training to get the tflite file but following error occurs after training and model is not created:

tensorflow.lite.python.convert.ConverterError: See console for info.
/bin/sh: 1: toco_from_protos: not found

[othiele (Olaf Thiele)]

Did you follow the steps provided in the docs? Where exactly does it happen?

[Tuomas]

Yep, when doing the training it says:

File “DeepSpeech.py”, line 12, in
ds_train.run_script()
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/train.py”, line 976, in run_script
absl.app.run(main)
File “/home/tuomas/.local/lib/python3.6/site-packages/absl/app.py”, line 303, in run
_run_main(main, args)
File “/home/tuomas/.local/lib/python3.6/site-packages/absl/app.py”, line 251, in _run_main
sys.exit(main(argv))
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/train.py”, line 956, in main
export()
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/train.py”, line 841, in export
tflite_model = converter.convert()
File “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/lite/python/lite.py”, line 983, in convert
**converter_kwargs)
File “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/lite/python/convert.py”, line 449, in toco_convert_impl
enable_mlir_converter=enable_mlir_converter)
File “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/lite/python/convert.py”, line 200, in toco_convert_protos
raise ConverterError(“See console for info.\n%s\n%s\n” % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: See console for info.
/bin/sh: 1: toco_from_protos: not found

When trying to convert using checkpoints:

File “DeepSpeech.py”, line 12, in
ds_train.run_script()
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/train.py”, line 976, in run_script
absl.app.run(main)
File “/home/tuomas/.local/lib/python3.6/site-packages/absl/app.py”, line 303, in run
_run_main(main, args)
File “/home/tuomas/.local/lib/python3.6/site-packages/absl/app.py”, line 251, in _run_main
sys.exit(main(argv))
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/train.py”, line 956, in main
export()
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/train.py”, line 809, in export
load_graph_for_evaluation(session)
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/util/checkpoints.py”, line 151, in load_graph_for_evaluation
_load_or_init_impl(session, methods, allow_drop_layers=False)
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/util/checkpoints.py”, line 98, in _load_or_init_impl
return _load_checkpoint(session, ckpt_path, allow_drop_layers, allow_lr_init=allow_lr_init)
File “/home/tuomas/Desktop/DeepSpeech-0.9.1/deepspeech_training/util/checkpoints.py”, line 71, in _load_checkpoint
v.load(ckpt.get_tensor(v.op.name), session=session)
File “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/python/util/deprecation.py”, line 324, in new_func
return func(*args, **kwargs)
File “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py”, line 1033, in load
session.run(self.initializer, {self.initializer.inputs[1]: value})
File “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py”, line 956, in run
run_metadata_ptr)
File “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/python/client/session.py”, line 1156, in _run
(np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1504,) for Tensor ‘cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/bias/Initializer/zeros:0’, which has shape ‘(8192,)’

[Tuomas]

Added following exports to .bashrc:

export PATH="/home/tuomas/.local/bin/toco_from_protos:$PATH"

export PATH="/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/lite/toco/python/:$PATH"

export PATH="/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/lite/toco/python/pycache/:$PATH"

but no luck.

[othiele (Olaf Thiele)]

Use venv and give full commands:

What and how to report if you need support

We are happy for every new user of DeepSpeech. As it is under active development, stuff changes quickly and you might have problems running it. We are here to help and might point to this thread if needed Please do the following before posting: Search this forum or Google for your problem Read the documentation for your version here Check Github issues for more technical problems If you still struggle, feel free to open a new thread in this forum (don’t answer to this post). …

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

@Tuomas

/bin/sh: 1: toco_from_protos: not found

This is a problem with your TensorFlow install, sorry.

From your stack, it looks like you have not followed the documentation and setup a virtualenv.

Please ensure you follow the docs.

[Tuomas]

Now I did folow docs step by step and same error occurs.

From conver.py

Find the toco_from_protos binary using the resource loader if using from

bazel, otherwise we are in a pip where console_scripts already has

the toco_from_protos tool.

if lite_constants.EXPERIMENTAL_USE_TOCO_API_DIRECTLY:
_toco_from_proto_bin = “”
else:
_toco_from_proto_bin = _resource_loader.get_path_to_datafile(
“…/toco/python/toco_from_protos”)

if _toco_from_proto_bin and not _os.path.exists(_toco_from_proto_bin):
_toco_from_proto_bin = “toco_from_protos”

I installed tensorflow 1.15.4 normally and toco_from_proto.py is at that ("…/toco/python/toco_from_protos") destination. What am I missing?

Why is it still using “/home/tuomas/.local/lib/python3.6/site-packages/tensorflow_core/lite/python/convert.py” when I am using virtual env?

[othiele (Olaf Thiele)]

@Tuomas

Why is it still using when I am using virtual env?

Hard to tell whether you use it or not without the command line … read about virtual environments first.