Walkthrough on how to fine-tune pre-trained English model?

Wesley_P · July 2, 2020, 4:18pm

I’ve read through DeepSpeech’s docs and I am still struggling with how to fine-tune the pre-trained models. I am mostly confused on how to create the necessary train.csv, dev.csv, and test.csv files. I do not have a proper computing environment to create them via importing CommonVoice and I cannot find an example of them online. If someone could give me an example of each or point me in the right direction that would be great!

lissyx · July 2, 2020, 5:03pm

You can see the fileformat by looking at the bin/ldc93s1.sh importer / exampel

Wesley_P · July 6, 2020, 8:35pm

Thank you for reaching out , but I am still a little bit confused. I’ve tried running the files associated with ldc93s1 since there is not an actual “bin/ldc93s1.sh” in the current DeepSpeech github repo and I still cannopt find an example of the csv files. If you could give me a series of commands to execute that would be extremely helpful!

lissyx · July 7, 2020, 8:10am

github.com/mozilla/DeepSpeech

bin/run-ldc93s1.sh

master

#!/bin/sh
set -xe
if [ ! -f DeepSpeech.py ]; then
    echo "Please make sure you run this from DeepSpeech's top level directory."
    exit 1
fi;

if [ ! -f "data/ldc93s1/ldc93s1.csv" ]; then
    echo "Downloading and preprocessing LDC93S1 example data, saving in ./data/ldc93s1."
    python -u bin/import_ldc93s1.py ./data/ldc93s1
fi;

if [ -d "${COMPUTE_KEEP_DIR}" ]; then
    checkpoint_dir=$COMPUTE_KEEP_DIR
else
    checkpoint_dir=$(python -c 'from xdg import BaseDirectory as xdg; print(xdg.save_data_path("deepspeech/ldc93s1"))')
fi

# Force only one visible device because we have a single-sample dataset
# and when trying to run on multiple devices (like GPUs), this will break

This file has been truncated. show original