Training results with 0.4.1 far worse than 0.3.0

(Rajpuneet Sandhu) #46

@reuben should I always extend the validation set with my own data instead of just using my validation data set? because otherwise it would over fit, wouldn’t it?

(Reuben Morais) #47

Your validation set should always be representative of the types of audio you want your model to be good at. Otherwise, yes, you risk overfitting.


fewer epochs, smaller learning rate, make sure your dev set is good

(Hafsa Farooq) #49

@rajpuneet.sandhu Can you post a complete guide for training DeepSpeech 0.4.1 ?
Plus guide about which file are required for it ?
How trie can be generated ?
Thank you!

(Rajpuneet Sandhu) #50

checkout ‘How I trained a French robot’. It has all the steps @noor_e_emaan11

(Rajpuneet Sandhu) #51

I trained with TEDlium (from Mozilla common voice website)and Voxforge (using the import script in DeepSpeech repo) datasets with the following:
python3 --n_hidden 2048 --checkpoint_dir ~/deepspeech-0.4.1-checkpoint --epoch -1 --train_files /home/rsandhu/ted-train.csv,/home/rsandhu/voxforge-train.csv --dev_files /home/rsandhu/ted-dev.csv,/home/rsandhu/voxforge-dev.csv,/home/rsandhu/common_voice_training_data/cv-valid-dev.csv,/mnt/librivox_data/librivox-dev-clean.csv,/mnt/librivox_data/librivox-dev-other.csv --test_files /home/rsandhu/ted-test.csv,/home/rsandhu/voxforge-test.csv --learning_rate 0.0001 --train_batch_size 24 --dev_batch_size 48 --test_batch_size 48 --display_step 0 --validation_step 1 --dropout_rate 0.2 --checkpoint_step 1 --lm_alpha 0.75 --lm_beta 1.85 --export_dir ~/new_model

I tested this generated model and the release 0.4.1 model and the results are in the attached (8.2 KB)

The datasets used for training are validated and clean but still I see the performance has degraded significantly. Any thoughts on this?
@lissyx @kdavis @reuben @josh_meyer

(Rajpuneet Sandhu) #52

@lissyx @kdavis @reuben @josh_meyer, did you get a chance to review the results?