I’ve been retraining DeepSpeech pre-trained model using Common Voice for a few days with an Nvidia K80 GPU.
The process got shut midway, and I restarted the training process. First question is does the training start from last checkpoint on it’s own?
Secondly, how much time would a single epoch take?
Third, is there a way to convert a checkpoint to an exported model?
My configuration is this:
python3 DeepSpeech.py --n_hidden 2048 --initialize_from_frozen_model models/output_graph.pb --checkpoint_dir fine_tuning_checkpoints_cv --epoch 3 --train_files data/CV/cv-valid-train.csv,data/CV/cv-other-train.csv --dev_files data/CV/cv-valid-dev.csv --test_files data/CV/cv-valid-test.csv --learning_rate 0.0001 --export_dir fine_tuned_models_cv
All I see at the moment is the scene in the attached photo.
