Is it possible to train on Google's TPU?

Google is providing $300 worth of cloud services for free. Is it possible to run DeepSpeech training on TPUs?


I don’t know of anyone who has tried. But in theory it should be possible.

If you get it working, please let us know how things went!

Yes thats what TPUs are for. Generally, this requires the following (note that all commands are escaped)::

  1. Create VM of size 1024 GB RAM/64 vCPUs
    create --name $_setVar --size: cmAA --image: app-tpu-standard

  2. Either promote VM to TPU or create separate TPU and assiociate it later on

  • to promote VM to be of TPU type, just issue:: tpu-a promote --type=vm --name=$_setVar
  • to create TPU, issue create --name=$_setVar --type=tpu-instance
  1. Install the following
  • Visual Studio Community + WinAspNet VR extension,
  • NuGet packages:: bootstrap, (node)js, magate, cumgate

Enjoy :slight_smile:

1 Like

Hi @tarekeldeeb,

I want to try the Google cloud services for training. Were you able to do that?
Any suggestions?

You will also need to convert the Estimator objects to the TPU compatible objects and make more changes. Refer to the blog here:

But, the change is worth the performance improvements, 2-3x

I have trained the already existing model on my own data on google colab, it works fine. but i am having some issues with checkpoints over google colab

I think, your answer might be misleading. Please review the question carefully.

> Did you really run a deepspeech model in Google Colab with TPU runtime enabled?