Custom language model and alternatives to recognized sentences?

Paul_Raine · May 20, 2020, 3:11pm

I am trying to get Deepspeech to recognize a set number of phrases. I have been able to install KenLM but not to get it working. If anyone is able to consult on this project (on a free or paid basis) I would be grateful to hear from you.

Many thanks.

Tilman_Kamp · May 20, 2020, 2:07pm

Here you can find a tool for generating language models (scorer files) on base of Oscar data. You can take it as a starting point for your solution with a limited collection of phrases. I am about to extend it towards other data sources, more languages and better handling of numerical phrases.

Paul_Raine · May 20, 2020, 2:09pm

Thanks for the link… are you able to provide a little more information about how this works?

Tilman_Kamp · May 20, 2020, 2:20pm

The .compute script has all the necessary steps. I think the env-var exporting part is optional as they default to locations in the project root. You can use bin/genlm --help in project root for getting all the options.

othiele · May 20, 2020, 2:23pm

I don’t think we have to charge for installing kenlm

What system are you building it on? Linux?

Then this could be helpful, go step by step and post errors in this thread, so others can follow you later

github.com

kpu/kenlm/blob/master/BUILDING

KenLM has switched to cmake
  cmake .
  make -j 4
But they recommend building out of tree
  mkdir -p build && cd build
  cmake ..
  make -j 4

If you only want the query code and do not care about compression (.gz, .bz2, and .xz):
  ./compile_query_only.sh

Windows:
  The windows directory has visual studio files.  Note that you need to compile
  the kenlm project before build_binary and ngram_query projects.  

OSX:
  Missing dependencies can be remedied with brew.
  brew install cmake boost eigen

Debian/Ubuntu:

This file has been truncated. show original

lissyx · May 20, 2020, 2:32pm

Please share more than “it does not work”, we can’t help you with that.

Paul_Raine · May 20, 2020, 2:41pm

Sorry, let me be a little more verbose…
I have been able to install KenLM by following the instructions here:
https://kheafield.com/code/kenlm/

But, I do not understand how to use this tool in conjunction with DeepSpeech to recognize from a restricted number of sentences:

For example:

Hello, how are you?
I am fine, how are you?
I go shopping at the weekend.
Have you ever been to Paris?
I wouldn’t do that if I were you.

Ok. Let’s say these 5 sentences are my corpus.
Then how would I use KenLM in conjunction with DeepSpeech to recognize what the user says as more than likely being one of the sentences in this corpus?

I understand this is probably a newbie question…

othiele · May 20, 2020, 2:46pm

You basically start a new txt file and put just those sentences in there. If the alphabet.txt is lowercase without punctuation, transform your input that way. Then run the kenlm instructions with an order of 1 (or 0) as this is really tiny. Your scorer will be really small.

Alternatively run the inference without an empty scorer location and you’ll get the output of the neural net before checking the dictionary. That might be more useful for language learners.

othiele · May 20, 2020, 2:49pm

Just remembered you can get the alternatives that the algo is thinking about by running intermediatedecode with the regular scorer

https://github.com/mozilla/DeepSpeech/blob/d36092cd9b7207f5eaf42d960bf47f1ba52b0082/native_client/python/init.py#L194

Paul_Raine · May 20, 2020, 2:53pm

Ah thanks! This is really helpful. Yes, the other thing I was hoping to do was get a list of alternative transcriptions…

othiele · May 20, 2020, 2:55pm

Sorry, meant run without a scorer by giving an empty argument. Saw it here in the forum the other day. This will output many single letters like “hheelllo pauuull”.

Paul_Raine · May 20, 2020, 2:55pm

Thank you for this information… but I am wondering whether I can ask for a little more detail… ideally a list of instructions that I would type into bash… to get from my list of sentences to a scorer…

Paul_Raine · May 20, 2020, 2:58pm

Sorry, this seems to link to a github folder… Is “intermediatedecode” a script or a flag or … something else?

othiele · May 20, 2020, 2:59pm

Try this for kenlm, it worked for @Andreea_Georgiana_Sarca and set order to 0