How long does is take time to train the model with Librispeech

DeepSpeech
#1

I use three GPUs (GTX 1080 , Memory 8G) to train model with librispeech ( I use the script ./bin/run-librivox.sh ) and have for 4 days ,but it just displayed “I STARTING Optimization” , why so low? Then , I found my GPU Utilization is always 0% ，just a few moments is 100%. How to use the GPU?

#2

You might need to tweak the script to change batch size to improve GPU usage?

#3

now, my GPU’s memory usage is 7797Mib( total is 8113Mib), if I increase the batch size , is it OK ? or will over memory?

#4

my batch size is the script’s default : train_batch_size = 12, dev_batch_size=12 test_batch_size=12

#5

I’m not an oracle, you’ll only know if you try. It might also be that your CPUs is not fast enough to feed the GPUs ?

#6

ok,I will try it ,thank you.

#7

Lissyx, you’re an oracle…sometimes LOL :joy:

#8

Yeah,you are right.Thank you.