How to convert a fine-tuning trained model for inference?

Archives DeepSpeech
1

Hi,

I just finish my first finetuning model training and I have realized that was generated two files in the folder output_models/

author_model_0.0.1.md
output_graph.pb

I was expecting to see a .scorer and .pbmm file to use in my ASR.

Regarding the documentation, I can run the following two commands:

$ python3 util/taskcluster.py --source tensorflow --artifact convert_graphdef_memmapped_format --branch r1.15 --target .

$ convert_graphdef_memmapped_format --in_graph=output_graph.pb --out_graph=output_graph.pbmm

Could you please confirm if is this the right way to submmit/publish that model for inference?

I’m using Ubuntu Linux.

Thank you

Last training buffer output:

I Exporting the model…
I Loading best validating checkpoint from fine_tuning_checkpoints/best_dev-1614170
I Loading variable from checkpoint: cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/bias
I Loading variable from checkpoint: cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/kernel
I Loading variable from checkpoint: layer_1/bias
I Loading variable from checkpoint: layer_1/weights
I Loading variable from checkpoint: layer_2/bias
I Loading variable from checkpoint: layer_2/weights
I Loading variable from checkpoint: layer_3/bias
I Loading variable from checkpoint: layer_3/weights
I Loading variable from checkpoint: layer_5/bias
I Loading variable from checkpoint: layer_5/weights
I Loading variable from checkpoint: layer_6/bias
I Loading variable from checkpoint: layer_6/weights
I Models exported at output_models/
I Model metadata file saved to output_models/author_model_0.0.1.md. Before submitting the exported model for publishing make sure all information in the metadata file is correct, and complete the URL fields.

2

Please use formatting for output, makes it a lot more readable.

That is fine, the pb is the freezed net, the pbmm is a different format which is better for inference.

Nope, you use the same scorer as for testing, this does not change in training.

You’ve done that correctly.

Would be great if you could share what you have finetuned with what material, how many epochs and some parameters.

3

Hi Othiele,

So, for inference, now I have just to convert using convert_graphdef_memmapped_format to output a .pbmm file, is that right?

I did a Librivox finetune training on the latest model 0.9.1.

At the first time I ran with 3 epoch, now I’m running again with 10 epochs.

My parameters are as follow:

python DeepSpeech.py --n_hidden 2048 \
--checkpoint_dir fine_tuning_checkpoints/ \
--epochs 10 \
--train_files /opt/data/librivox-train-clean-100.csv \
--dev_files /opt/data/librivox-dev-clean.csv \
--test_files /opt/data/librivox-test-clean.csv \
--learning_rate 0.0001 \
--export_dir output_models/ \
--train_cudnn \
--dropout_rate 0.4
4

You can run inference with the pb, but it is slower, pbmm is faster.

Try a train and dev batch size as high as you can without getting an error. This will speed things up.

5

Thank you.

Please how would be the parameters for that suggestion?

6

It’s time to look into the docs or search this forum or --help :slight_smile:

7

You are right. Lol

Thank you