Amount of loss that should be aimed at & test epoch without LM

When training a model, what should be the dev loss that we should aim at to get a descent WER (<15% lets say)?

And is there a way to use the TEST epoch but without passing a LM (since i have not trained this yet and first want to fine tune the acoustic model)?

Thank you :slight_smile:

It depends on sentence length, alphabet size, and probably other dataset characteristics, so thereโ€™s not much guidance we can give from scratch. You should try it and learn how the model behaves on your data. Training a model on a new dataset is not a turn key solution that you can just apply without experimentation.

Yep, just specify an empty value for the scorer flag when training: --scorer_path ''

On older versions this was the LM binary flag: --lm_binary_path ''

Thank you :slight_smile:

Could I also use the evaluate.py to calculate the total WER on my test set with the exported acoustic model?

No. evaluate.py needs the checkpoint.

Does it need a specific checkpoint or the whole checkpoint directory?

And will the model that gets exported at the end of the training be the model with the best val loss or the latest checkpoint?