Sentence Collector - Cleanup before export vs. cleanup on upload

[mkohler (Michael Kohler)]

@HelloTheWorld Yes, I think you got most parts right. There are some missing steps though. I have created this Pull Request to update the README in the repository if you could have a look at it

@HelloTheWorld

• then process it with CLEAN-UP routine. And record both raw and cleaned-up text as ‘awaiting validation’ text.
• then the VALIDATION stuff goes on, on the cleaned-up text (great idea !) , removing everything flagged (numerals, ACRONYMS, etc.) EXCEPT IF they have been changed to TEXT numerals or full described acronyms, thus they will not trigger anymore VALIDATION barriers (…like I’m proposing in French cleanup files in pull 635)

This is contrary to my initial suggestion in the post, however I think this does indeed bring up a good point. With my suggestion the cleanup could not be used to correct anything that would not pass the automatic validation. That’s why I commented on your PR that parts of your cleanup proposals are not needed, as they would not even pass validation (and therefore never get to the cleanup step).

I have to say, I kinda like the approach of running the cleanup before the validation, so that for example numbers could be converted from “2” to “two” etc. Of course, that might not be the best example (or easy to do) for each language. Overall we wouldn’t lose anything by switching this around. @bozden @ftyers do you have any preferences on which way around this is done?

@ftyers

This is potentially nice, but I think that both should be saved (e.g. the updated sentence — which would be the one uploaded to Common Voice, and the original sentence — which could be used for identifying sentences based on their original form).

Thanks for bringing up this point. I was not aware of this. I definitely would agree that we should support these use cases. I just don’t know exactly if saving both variants is the way to go here and what implications this would have. Let me think about this for a bit.