I have problem with setting trainning and make me frustacing…
i think not need to give a specification, because the training is run.
last base order:
XLA_PYTHON_CLIENT_ALLOCATOR=platform TF_XLA_FLAGS=–tf_xla_cpu_global_jit python3 -u /home/bram/Documents/coding/speech/deepspeech/DeepSpeech.py --train_files “/home/bram/Documents/coding/speech/traindata/CVdata/clips/train.csv” --dev_files “/home/bram/Documents/coding/speech/traindata/CVdata/clips/dev.csv” --test_files “/home/bram/Documents/coding/speech/traindata/CVdata/clips/test.csv” --alphabet_config_path “/home/bram/Documents/speech/traindata/corpus/alphabet.txt” --lm_binary_path “/home/bram/Documents/speech/traindata/corpus/lm.binary” --lm_trie_path “/home/bram/Documents/speech/traindata/corpus/trie” --learning_rate 0.000025 --dropout_rate 0.2 --log_level 1 --epochs 12 --export_dir “/home/bram/Documents/speech/checkpoint” --checkpoint_dir “/home/bram/Documents/speech/checkpoint” --use_allow_growth true --train_batch_size 24
what already i have do:
set many varians of hyperparameters:
- learning_rate: 0.000025, 0.0001, 0.00001, 0.0000125, etc
- dropout_rate: 0.2, 0.15, 0.25, 0.4, 05
- n_hidden : 2048, 1536, 1024, 768.
- batch size: 4, 8, 12.
the problem:
- if i choose batch 12, at epoch 13 start overfitting, happens i batch 8, at epoch 9 start overfitting.
- validation result very variable depend hyper parameters, but it’s never reach below 60%, my best record is 64%, after that start overfitting. off course the result is very mess.
another problem, can save the checkpoint but can create model, bu i think i can handle it later, after the train got good result.
dataset:
Common Voice Bahasa Indonesia.