While make inference was always 58 seconds

Hello,
I make use this notebook:
https://colab.research.google.com/drive/1tKHSI20kRlOL0PSA8mCVJQIrgRIswg0F?usp=sharing

I replace with my model. Last block, is make inference. Why is length 58 seconds if sentence very short?

Thanks you.

Decoder can not „find“ the end and stops eventually at the maximum decoder step - which is set to 1280000 in your case, a quite high value. You can try to add a „.“ at the end of your phrase. You also can try to set max_devoder_steps to a lower value.

2 Likes

Thanks you very much.