Deepspeech Python - Prevent the STDOUT of Versions

[lnxusr1 (lnxusr.com@gmail.com)]

I have searched everywhere and cannot find a way to prevent the STDOUT print out of deepspeech on the model creation.

Specific Python line of code:

model_file="/path/to/model.pbmm"
my_model_obj = deepspeech.Model(model_file)

Output on STDOUT (as best I can tell):

python3 my_example.py
TensorFlow: v1.15.0-24-gceb46aa
DeepSpeech: v0.7.3-0-g8858494

Please help… I’m sure it’s something stupid I’m missing (and been searching on and off for days) so I apologize for asking, but I’m really stuck. My goal is to log all output to a log file, and I simply cannot figure out what is actually outputting this and how to trap it.

[lnxusr1 (lnxusr.com@gmail.com)]

I’ll add that I can redirect stdout from the shell, but I’d prefer to trap this within python.

This method works… but is less than ideal.
python3 my_example.py 1>> myfile.log 2>> myfile.err

So to clarify my question… how, within Python, can I trap this message. The “sys.stdout” doesn’t seem to help. Is there a way to tell deepspeech to not print them? Or is there a documented way to trap the output?

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

Those are printed on stderr …

[lnxusr1 (lnxusr.com@gmail.com)]

So. Many. Questions.

Why would those be printed to stderr? They definitively are not errors.

Anyway, are the printed to stderr via a C Lib? I don’t see it in the python wrapper and any amount of effort I’ve put into using sys.stderr has failed to hide the messages or redirect them to the output file.

Simply put… this does not improve my situation.

import sys
sys.stderr = open("output.txt","w")
model_file="/path/to/model.pbmm"
my_model_obj = deepspeech.Model(model_file)

Still prints the output…
python3 my_example.py
TensorFlow: v1.15.0-24-gceb46aa
DeepSpeech: v0.7.3-0-g8858494

I’ve tried the Clib logic to redirect this output as well and the messages still print to the console and not the file. Even wrote my own stderr_out redirect in C with its own python wrapper and no luck. Very frustrating. Any words of wisdom?

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

@lnxusr1

Why would those be printed to stderr? They definitively are not errors.

Because we wanted to have them always displayed by default and not mess with stdout

@lnxusr1

Anyway, are the printed to stderr via a C Lib? I don’t see it in the python wrapper and any amount of effort I’ve put into using sys.stderr has failed to hide the messages or redirect them to the output file.

Yes, check in native_client/deepspeech.cc

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

@lnxusr1

import sys
sys.stderr = open("output.txt","w")
model_file="/path/to/model.pbmm"
my_model_obj = deepspeech.Model(model_file)

I don’t think this is expected to work

@lnxusr1

I’ve tried the Clib logic to redirect this output as well and the messages still print to the console and not the file. Even wrote my own stderr_out redirect in C with its own python wrapper and no luck. Very frustrating. Any words of wisdom?

What is your problem with the version being printed?

[lnxusr1 (lnxusr.com@gmail.com)]

I am using a console-based visualizer for the audio output responses. Since it prints the visualizations in the console anything printing to the console messes up the visualizer.

Bottom line is that I don’t want it printing anything to the console that I cannot trap/control inside of Python to be sure that what I put on the screen is what I expect to be there.

It’s mostly an annoyance because I only have to initialize the model once, but these really aren’t errors and so for the similar reasons as to why they weren’t on stdout I’d argue they shouldn’t be automatically included on stderr either.

For the visualizer you can check out github.com @ dpayne/cli-visualizer and it should be more clear why it’s annoying. (Initialize the model in the same console window after starting the visualizer)

[dabinat]

I made a PR for this a while back but it never made it into master:

-q quiet mode flag to suppress version numbers

@lissyx Is there value in resurrecting this? Although instead of a quiet mode, it may be better to put the versions behind a -v or —version flag like many other tools do.

[lnxusr1 (lnxusr.com@gmail.com)]

Thank you @dabinat. I actually ran across your post during my research on how to get around this. Obviously my vote is to go with the -v or --version flag as that’s way more common than writing it to the stderr on each initialization.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

@lnxusr1

It’s mostly an annoyance because I only have to initialize the model once, but these really aren’t errors and so for the similar reasons as to why they weren’t on stdout I’d argue they shouldn’t be automatically included on stderr either.

this was needed for smoothing the debugging process: people mostly never giving correct informations, we needed a way to easily get that and correctly

@lnxusr1

. I actually ran across your post during my research on how to get around this. Obviously my vote is to go with the -v or --version flag as that’s way more common than writing it to the stderr on each initialization.

Again, this is what I said earlier and that is explained in the PR.

sorry but I don’t have time to dig into how to redirect stderr from python, but that’s what you should be doing …

[lnxusr1 (lnxusr.com@gmail.com)]

@lissyx, Based on your comments I suggest amending the PR to implement a DEBUG LEVEL that can be set via config or parameter with the default level meeting your needs and at least one higher level that would only write out true errors. And still adding the -v option as a nice-to-have. Using a debug level would greatly improve your support ability as you could potentially add more information (even if not an error) based on the level without adversely affecting the output for everyone.

I know deepspeech hasn’t reached “1.0” status yet so call it “maturing the code” and set it as the lowest priority if that’s how you see it. Even if resolving this issue isn’t something that can be implemented in the short term it should be on the roadmap.

[lissyx ((slow to reply) [NOT PROVIDING SUPPORT])]

@lnxusr1

, Based on your comments I suggest amending the PR to implement a DEBUG LEVEL that can be set via config or parameter with the default level meeting your needs and at least one higher level that would only write out true errors. And still adding the -v option as a nice-to-have. Using a debug level would greatly improve your support ability as you could potentially add more information (even if not an error) based on the level without adversely affecting the output for everyone.

Patches are welcome.

@lnxusr1

I know deepspeech hasn’t reached “1.0” status yet so call it “maturing the code” and set it as the lowest priority if that’s how you see it. Even if resolving this issue isn’t something that can be implemented in the short term it should be on the roadmap.

The truth is, very very few people have had issues with that. Only @dabinat cared to send a PR, and it seems it was not enough of an issue for him to implement what you suggest.

I’m pretty sure you can just redierct stderr correctly, it’s just that your current code does not work: unfortunately, I Don’t have time to dig into that.

@lnxusr1

at least one higher level that would only write out true errors.

I hate software that hides errors: this is a pain as a user and when we provide support to people.

[lnxusr1 (lnxusr.com@gmail.com)]

I’ll look into writing a patch to add a debug level if that is something that could be considered in your intake.

For clarity, this is not the intention to hide errors. It is to hide debug messages.

[reuben]

import os
import sys
import deepspeech

class SilenceStream():
    def __init__(self, stream):
        self.fd_to_silence = stream.fileno()

    def __enter__(self):
        self.stored_dup = os.dup(self.fd_to_silence)
        self.devnull = open('/dev/null', 'w')
        os.dup2(self.devnull.fileno(), self.fd_to_silence)

    def __exit__(self):
        os.dup2(self.stored_dup, self.fd_to_silence)
        self.devnull = None
        self.stored_dup = None

print('Create model without context manager:')
m1 = deepspeech.Model('deepspeech-0.7.4-models.tflite')

print('Create model with context manager:')
with SilenceStream(sys.stderr):
    m2 = deepspeech.Model('deepspeech-0.7.4-models.tflite')

Output:

python silence_stream.py
Create model without context manager:
TensorFlow: v1.15.0-24-gceb46aae58
DeepSpeech: v0.7.4-0-gfcd9563f
Create model with context manager:

[lnxusr1 (lnxusr.com@gmail.com)]

That is perfect. THANK YOU!

[lnxusr1 (lnxusr.com@gmail.com)]

@reuben

import os
import sys
import deepspeech

class SilenceStream():
    def __init__(self, stream):
        self.fd_to_silence = stream.fileno()

    def __enter__(self):
        self.stored_dup = os.dup(self.fd_to_silence)
        self.devnull = open('/dev/null', 'w')
        os.dup2(self.devnull.fileno(), self.fd_to_silence)

    def __exit__(self, exc_type, exc_value, tb):
        os.dup2(self.stored_dup, self.fd_to_silence)
        self.devnull = None
        self.stored_dup = None

print('Create model without context manager:')
m1 = deepspeech.Model('deepspeech-0.7.4-models.tflite')

print('Create model with context manager:')
with SilenceStream(sys.stderr):
    m2 = deepspeech.Model('deepspeech-0.7.4-models.tflite')

Small fix to “exit” to add the required positional arguments as it was actually erroring on the termination of the SilenceStream() class.