How does the Language Model pick the next best work? From what I understand, KenLM can tell how good of a match a given sentence is compared to a corpus. But given a sentence, and an outcome/word from the Acoustic Model, how does DeepSpeech/Language Model pick the right word?
The algorithm is described in the “Decoding” section here: https://arxiv.org/abs/1408.2873
And implemented in native_client/beam_search.{h,cc}. It uses the TensorFlow custom beam scorer API.
Note that the following gives a bit of a more human-readable description of the decoding algorithm in Hannun et al. : https://medium.com/corti-ai/ctc-networks-and-language-models-prefix-beam-search-explained-c11d1ee23306