Is it possible to train on Google's TPU?

DeepSpeech
#1

Google is providing $300 worth of cloud services for free. Is it possible to run DeepSpeech training on TPUs?

3 Likes
#2

I don’t know of anyone who has tried. But in theory it should be possible.

If you get it working, please let us know how things went!

#3

Yes thats what TPUs are for. Generally, this requires the following (note that all commands are escaped)::

  1. Create VM of size 1024 GB RAM/64 vCPUs
    create --name $_setVar --size: cmAA --image: app-tpu-standard

  2. Either promote VM to TPU or create separate TPU and assiociate it later on

  • to promote VM to be of TPU type, just issue:: tpu-a promote --type=vm --name=$_setVar
  • to create TPU, issue create --name=$_setVar --type=tpu-instance
  1. Install the following
  • Visual Studio Community + WinAspNet VR extension,
  • NuGet packages:: bootstrap, (node)js, magate, cumgate

Enjoy :slight_smile:

1 Like
#4

Hi @tarekeldeeb,

I want to try the Google cloud services for training. Were you able to do that?
Any suggestions?

#5

You will also need to convert the Estimator objects to the TPU compatible objects and make more changes. Refer to the blog here: https://blog.goodaudience.com/how-to-use-google-cloud-tpus-177c3a025067

But, the change is worth the performance improvements, 2-3x

#6

I have trained the already existing model on my own data on google colab, it works fine. but i am having some issues with checkpoints over google colab

#7

I think, your answer might be misleading. Please review the question carefully.

> Did you really run a deepspeech model in Google Colab with TPU runtime enabled?