I started validating items but there is no instruction given for what the conditions are for something being valid.
Does thumbs up mean…
-The person is genuinely trying to read the sentence?
-The audio is intelligible, i.e. I would know what they are saying without a transcript?
-The audio is read clearly with no mistakes?
When dealing with lower quality nonnative speakers I have no idea what to do during validation, and my concern is that different graders are validating differently based on their own personal interpretations of what “thumbs up” means, leading to inconsistencies in the dataset. Is this formally specified anywhere and if so is there a way we can make it more obvious to new validators? Thank you.