I use three GPUs (GTX 1080 , Memory 8G) to train model with librispeech ( I use the script ./bin/run-librivox.sh ) and have for 4 days ,but it just displayed “I STARTING Optimization” , why so low? Then , I found my GPU Utilization is always 0% ，just a few moments is 100%. How to use the GPU?
You might need to tweak the script to change batch size to improve GPU usage?
now, my GPU’s memory usage is 7797Mib( total is 8113Mib), if I increase the batch size , is it OK ? or will over memory?
my batch size is the script’s default : train_batch_size = 12, dev_batch_size=12 test_batch_size=12
I’m not an oracle, you’ll only know if you try. It might also be that your CPUs is not fast enough to feed the GPUs ?
ok,I will try it ,thank you.
Lissyx, you’re an oracle…sometimes LOL
Yeah,you are right.Thank you.