Training DeepSpeech on Silence?

When recording silence (or near silence) DeepSpeech starts to produce gibberish instead of nothing. Would it be possible to add silence (or noise alone) to the dataset to prevent this?

Did you mean to post this in the Deepspeech section?