KenLM LM vs trie

[carlfm01]

How to Strict the output to the Language Model only?

The trie determines what is “in vocabulary” and the valid_word_count_weight determines how much importance should be given to the trie’s opinion on what is in and what is not in vocabulary. So, in particular, increasing the value of valid_word_count_weight should decrease the occurrence of out of vocabulary words as defined by the trie. The weights I mentioned lm_weight, word_count_weight, and valid_word_count_weight are external to the language model and are not part of the language model we…