Doesn`t use GPU while training, but during recognition it uses one

DeepSpeech
#21

@lissyx is it enough to install tensorflow-gpu only in virtual environment or tensorflow-gpu need to be installed in the core system with pip list?

My situation is i have doubt while the training model is taking all power of GPU.
Looks like only 30% or sometimes 3% used GPU.

#22

Just in the virtualenv is enough

Likely batch size effect, try to increase it?

#23

I am using GPU RTX 4000, 32 ram, 500SSD.

Is that enough and any configuration issues on CUDA installed 10.2 and 7.6 may it cost slow training.

Could you please clarify.

#24

Clarify what ? Please check your batch size.

#25

you mean this batch --> train_batch_size 48 dev_batch_size 48 test_batch_size 48
If yes, May i know the maximum limit of batch size i have seen util/flag.py but i am not able to find the maximum size.

#26

Yes

Iā€™m sure we document that somewhere, but basically you need to adapt to your dataset and your hardware. Bigger batch size will require more GPU memory. Longer audio files requires more GPU memory. Bigger batch size improves GPU usage.

Too much GPU memory usage will crash.

1 Like