How to train Deepspeech model on google colab?

Hi, guys I don’t own a gpu. So I’m trying to infering it on google colab. I get this error I think it is because of cuda
Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/deepspeech/impl.py", line 14, in swig_import_helper return importlib.import_module(mname) File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 658, in _load_unlocked File "<frozen importlib._bootstrap>", line 571, in module_from_spec File "<frozen importlib._bootstrap_external>", line 922, in create_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed ImportError: libcusolver.so.9.0: cannot open shared object file: No such file or directory During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/deepspeech", line 7, in <module> from deepspeech.client import main File "/usr/local/lib/python3.6/dist-packages/deepspeech/__init__.py", line 4, in <module> from deepspeech.impl import AudioToInputVector as audioToInputVector File "/usr/local/lib/python3.6/dist-packages/deepspeech/impl.py", line 17, in <module> _impl = swig_import_helper() File "/usr/local/lib/python3.6/dist-packages/deepspeech/impl.py", line 16, in swig_import_helper return importlib.import_module('_impl') File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ModuleNotFoundError: No module named '_impl'

Colab has following version of cuda and tensorflow:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Tue_Jun_12_23:07:04_CDT_2018
Cuda compilation tools, release 9.2, V9.2.148
mesh-tensorflow          0.0.4                

tensorflow               1.12.0               
tensorflow-hub           0.2.0                
tensorflow-metadata      0.9.0                
tensorflow-probability   0.5.0                

How to solve this issue ? Is it not possible to run it on 9.2 instead of 9.0. ?? Waiting for reply

TensorFlow upstream r1.12 is using CUDA 9.0, so we are building against it as well. You can rebuild against CUDA 9.2.

I use the following code snippets:

# Download and Install CUDA-9.0
!wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda_9.0.176_384.81_linux-run
!sh cuda_9.0.176_384.81_linux-run --silent --toolkit --toolkitpath=/usr/local/cuda-9.0 --override
# Set environment variables
import os
os.environ['LD_LIBRARY_PATH'] = "/usr/local/cuda-9.0/lib64:/usr/local/cuda-9.0/extras/CUPTI/lib64:" + os.environ['LD_LIBRARY_PATH']
os.environ['PATH'] = "/usr/local/cuda-9.0/bin:" + os.environ['PATH']

You need to rebuild everything, sadly.

Is rebuilding needed for the version 0.4? Cause I successfully trained on Colab with DeepSpeech v0.3 using these methods

If you’re training on v0.4, uninstall the deepspeech package and remove the old libctc_decoder_with_kenlm.so, then you should have no binary dependencies on CUDA anymore.

Assuming the TensorFlow that’s preinstalled in Colab is working fine.

Have you completed your training deepspeech on google colab. Will you please your experience and issues you might had faced. It will be a help and guidance for newcomers like me to train Deepspeech engine on google colab.

1 Like

Please don’t hijack 2 year old posts. Search the forum and start a new post with what you learned and what you want more details for. There were a lot of posts about colab in the last 2 years …