Precise meaning of thumbs up/thumbs down when validating

Common Voice
#1

I started validating items but there is no instruction given for what the conditions are for something being valid.

Does thumbs up mean…
-The person is genuinely trying to read the sentence?
-The audio is intelligible, i.e. I would know what they are saying without a transcript?
-The audio is read clearly with no mistakes?

When dealing with lower quality nonnative speakers I have no idea what to do during validation, and my concern is that different graders are validating differently based on their own personal interpretations of what “thumbs up” means, leading to inconsistencies in the dataset. Is this formally specified anywhere and if so is there a way we can make it more obvious to new validators? Thank you.

#2

In the Italian community we say this:

Basically, the audio must be correct (text corresponds to the audio and vic.) and you could understand it also without read the text, and the audio doesn’t have mistakes (noises, other voices, …).

Here you can find some guideline about validation: Discussion of new guidelines for recording validation

When you are not sure, I suggest you to Skip the clip :slight_smile:

(I’m not an employee of Mozilla, so these information could be not correct)

1 Like
#3

Thank you, I’ve taken the discussion to that thread! Hopefully we can find a way to make the guidelines consistent and prominent. Cheers.