Determining What Learning Rate to Use

Hi,

Let’s say I’m trying to use transfer learning to calibrate a model to a specific person’s voice. However, I only have a total of 20 people to train my model off of, so I want to train more generally for them and then calibrate my model to work specifically for this one person.

What would be a good learning rate for the general 20-person population and the specific person I’m calibrating to?

I see that the default rate is 0.001 for DeepSpeech.py.

1 Like

I’d first try 0.0001 for everything.

1 Like

What if I did 0.001 for the general population? Do you happen to know if I’d still get a fair loss on that?

I have 30,000 total files; 1 word each file; 455 different words

1 Like

I’d try 0.0001 for everything.

1 Like

Ok cool. I’m going to go for that for a general model anyway, so I might as well try that.

1 Like

Check validation loss vs. training loss. Closer they are the better. If so, you have found good learning rate, which is not overfitting or underfitting the model…

1 Like