Acoustic model outputs probabilities for each class of character, not at word-level. So once we decoded those and we have a string, I’m not so sure we have that information of “known” / “unknown” words. Maybe I misunderstood your point, but I guess that you should look deepeer into why your model confuses other words for one of the known words. I guess it’s too much overfitted.
Maybe have a look at what @elpimous_robot did ? He worked on something similar TUTORIAL : How I trained a specific french model to control my robot