Today I while I spent a moment validating sentences in the sentence-collector app, I noticed that my back was a bit sore and my eyes a bit tired. I tried using the OS accessibility feature to read aloud the sentences, so I could rest my eyes. Just listening to a sentence is enough to know if is grammatical no not. I made me think. The value I am adding in validating data is a human brain that recognizes a sentence that conforms to the rules of english grammar or one that doesn’t(arguably), and then respond to indicate which. But there is no need for me eyes to read a screen, or my hands to touch a trackpad to do this. Having to look at a screen, or touch a computer puts limits on where, when, and how much I can validate sentences. Some alternative ways interacting with the system could reduce resistance to making validations.
An example I imagined, was a webpage that could load in a phone(or watch), and read out the sentence with TTS, and then use a gyro to pickup a flick of the device left for no and right for yes. And of course there is voice based interfaces that can be used today for “yes” “no” responses.
It would be interesting to hear others thoughts on this.
This is an interesting reflection. In general the sentence collector should be used only when there are no other large sources of text available to mass validate, because manual review of each sentence is painful as you described.
Has your language already done the wikipedia process or if European, the Europarl one? This should provide a language with enough buffer for close to 1000 hours without repetitions.
Thanks for starting this discussion. Generally I like that idea, however I don’t see this working without any additional checks on the sentences, as you couldn’t hear typos necessarily.
I’d definitely be in favor of less RSI-prone methods of validation, for both sentences and recordings.
Common Voice is now building up a big database of “yes” and “no” recordings, so hopefully voice-based validation might be an option at some point in the future.
Oh, I guess I haven’t kept up with the conversation on this. It’s not very clear from just looking at the sentence collector the site that english sentences are fine for the time being. It does make a lot of sense to use sentence that are already known to be valid as a base.
Yes, I guess so. I thought about using aspell to skip sentences that had “out of dictionary” words. what I noticed:
- the default was en-US, so I had to add UK
- Most names of people and places are OOD, so I had to find a big text file full of all the city names of the world, and lists of common first and last names.
Which raised the following questions for me:
- should the dataset contain both US and UK spellings? Will this lead to ambiguity on letter/sound correspondences? What will be outputted when someone says “colourise/colorize”?
- What counts a correctly spelled name?
- What standard was I using to judge if a sentence was spelled correctly anyway?
On a side note:
When I trained a gpt-2 mini-model on the Wikipedia sentences in the CV repo, the sheer variety of names, led the model to ‘be very confused’ as to what a conventionally valid word was. The sentences it was generating had a large number of kind of word like but not valid words[1]. When I trained on the sentence collector data with names and things filtered out, I got a much more conventional words[2]. I don’t know if deep-speech tokenizes at a whole word level, or breaks words down like the gpt-2 huggingface tool does.
[1]
The station was the Sempliocation of the United States.
In the area, a number of the original tropical areas with the canjake.
The mainstains were built in both cooking and is called the fitness of the new lady, with
The school is the first of the two-day-found of the world.
The town is located in the city of Arlinford, Mand’s city.
There are the Roconian-He was a holidayehallia Graves.
[2]
Why, we never
Will you find it, Mr. Dean?
Will you give to do here?
Will you take more presents, my lady?
Will you tell me, my dear?
With a lovely soul.