How to obtain probabilities of each character

`confidence` ()[[source]](https://deepspeech.readthedocs.io/en/v0.6.1/_modules/native_client/python.html#Metadata.confidence)

Approximated confidence value for this transcription. This is roughly the sum of the acoustic model logit values for each timestep/character that contributed to the creation of this transcription.

I have multiple models trained on my custom dataset. I would like to ensemble all models, by considering the metadata of each model inferences.

Apart from getting confidence for whole transcription, is there any way to get output probabilities of each character.

@utunga has implemented this functionality, maybe you can convince them to make a PR and share it with the world :slight_smile:

If you havn’t been able to convince him, i doubt he will listen to me. :wink:

Is there a way to do it, can you guide me?

I can assure you, if i implement it, i will make a PR :slightly_smiling_face:

Hi!

For the record, we’d be more than happy to share the work we did on this. Honestly, it’s not very much though, just adding a few bits here and there and I think the only reason it hasn’t landed back into the Mozilla repo as a PR is that we’re still working on sorting out a clean PR against an up to date version. (Because DeepSpeech itself keeps moving ! ) Some other changes we made have made it back into the Mozilla master though which is nice.

As far as the timing changes go, the first time we made changes it was at the 0.5.1 version which you can see here. That worked OK, but then the whole way that scoring worked changed so we had to redo for 0.6 which is this version I think. The 0.5.1 version got a bit more in depth testing but either one should work.

I’ve recently been working on re-applying these changes for the 0.7.1 version of DeepSpeech which you can see here unfortunately the way the scoring model worked has changed quite a bit and it’s not really working at this point. So I wouldn’t recommend trying to use the 0.7 version of the changes - its still got problems. I created this this post yesterday to ask about those problems. But feel free to use the older vesrions if that would help. Bear in mind that models trained on versions where x is different (in the 0.x.y formatting) tend to not be compatible with each other.

Would welcome your input and help with that. Experimenting with it yourself would be great for example. That said very very far from prime time.

I think it would be fair to say we definitely intend to produce a clean PR and land these changes back to Mozilla repo but it may take some months before that happens. Also, as I mentioned they’re honestly not that much to write home about at this point.

Hopefully, we can collaborate on making this happen but either way, I hope you’ll be able to get letter confidences in future iterations of DeepSpeech.

2 Likes

@utunga Percentage probabilities is a feature I’m interested in. Let me know if you need any help turning it into a PR.