CPU based training time estimation

Can anyone tell me any experience with CPU based training? Xeon based CPU or any. How much time can a training epoch can take?

That depends on the exact CPU model, but mostly depends on your whole hardware as well as the amount of data you are training on. Usually, there’s a 10-100x difference in speed between CPU and GPU, so it’s really unlikely you can train something purely on CPU.

Hi lissyx Sir

Thank you so much.
It is
Intel® Xeon® CPU E5-2640 0 @ 2.50GHz based
two socket machine and the data usage is common voice German.

That does not document how much data you have, but I recall that Common Voice German is above 300 hours, so you will need GPUs.

OK. Yes, that is 340 hrs data more or less.
But, any minimal GPU which you can recommend which trains a model in 2-3 weeks. or I have to build 2-4 GPUs based machine or cluster?

You want to spend 2-3 weeks on each rounds ? FTR 250h of french I can train a model in ~4h on 2x RTX 2080 Ti

If I understand correctly, it means, this GEFORCE RTX™ 2080 TI GPU can train the 340 hrs data, like single GPU in 24-30 hrs?

Dear Sir,

How many epochs you have with this training model of 250h data in 4h.?

https://github.com/Common-Voice/commonvoice-fr/blob/master/DeepSpeech/Dockerfile.train.fr

Dear Sir,

Unfortunately, page (Dockerfile) is not accessible.
Error. Page not found.

Thank you @lissyx. Could you give an approximate estimate for a similar setting than the 4h one, with a 32-thread Intel Xeon with 184GB of RAM? Thank you.

No, I can just tell you that you are going to loose your time. One epoch on my Threadripper 1950X would take > 3 min when the GPU version on RTX2080Ti would take 2 seconds.

Thank you so much for your quick reply. I need to balance my investment on GPU hardware vs the time loss and your feedback is very valuable.

If you have any amount of data that is a little bit serious, you will need at least one good GPU. Training on CPUs is really way too slow.