RAdam instead of adam

DeepSpeech
#1

Hello,
just a topic to discuss the use of Rectifier Adam (RAdam) instead of Adam in DeepSpeech.

If I trust this link using Radam might improve results.

Did this method will be implemented or is it not in the current reflexion ?

#2

As far as I can tell, it’s not on our radar.

#3

Is an RAdam implementation in TensorFlow?

#4

Looks like it’s not there yet, issue 422

#5

This looks testable: https://github.com/CyberZHG/keras-radam#tensorflow-without-keras

#6

It might be interesting to test it, don’t know if there will be improvement