Hi @mkohler @heyhillary , We’d like to ask for your advice on collecting additional sentences for Belarusian. After @Aliaksandr finalized the pull request and a local team of enthusiasts launched a volunteering campaign, the number of clips in Belarusian increases at a steady rate of >10K per day. L…

[image] mytmpaccount2015: Do you think we should focus on reviewing the sentences which are currently in the sentence collector, or making a cleaner sample? That really depends on the actual quality. Can’t judge that really. What’s the rough ratio of sentences that are useful? [image] myt…

Thank you @mkohler for these recommendations. At the moment, we have ~80K new sentences exported from public domain Belarusian fiction; this dataset has some overlap with the backlog of Belarusian sentence collector (as it basically covers a wider range of texts by the same authors), but the sentenc…

[image] mytmpaccount2015: E.g. if we validate a 4K sample by Monday, June 28th, and the error rate is reasonably low, could we expect that the PR is accepted before the upcoming release? I can’t talk in @phire ’s name, so I can’t guarantee that. The next Common Voice release happens on June 30…

[image] mkohler: What I can say however is the earlier the PR is submitted the bigger the chances are that there is enough time for it to be merged before that deadline. Thanks! We’re now in progress with the verification; from what I saw so far, most sentences look nice, so I would anticipat…

[image] mytmpaccount2015: One more question, to be on the safe side with CC0 licensing. We’ve harvested the sentences from texts written by authors who died 70+ years ago, so the texts themselves are in public domain. But the online library knihi.com , which digitized the texts and put them on th…

Meanwhile we’ve done some additional filtering, leaving 69K sentences in consideration, and validated a 4K sample thereof. Their quality is high, with around 2% of errors, which is better than Wikipedia. Here is the pull request . @heyhillary – could you please advise on the above, i.e. should we co…

Hiya, I will have to check in with Mozilla legal team, to confirm if you can use the library. Is it possible if you could share the web-page that details this ? Many thanks, Hillary

Sure, please find attached two files in a ZIP archive: attachments.zip (23.2 KB) public-domain-authors-70.tsv is the list of authors whose texts, published during their lifetime, we consider to be in public domain. Links to knihi.com index pages were added manually for those authors whose texts a…

Update: I’ve checked once again - we’re in fact using sentences from a subset of these texts, 673 out of 2097. Attaching the list of links once again. knihi.com-source-text-links-subset.zip (8.4 KB)

Thank you for sharing the links !

Sentence collection for Belarusian – request for advice

Common Voice

mkohler (Michael Kohler) June 26, 2021, 2:43pm 6

@heyhillary I’m redirecting this question to you

Topic		Replies	Views
Sentence collection for Belarusian Common Voice sentence-collection	8	1905	July 7, 2020
Sentence collector copyright issues Common Voice sentence-collection	54	6406	April 16, 2024
Polish sentences concerns Common Voice sentence-collection , issue , dataset	20	3357	May 4, 2020
Extending our sentence collection capabilities Common Voice sentence-collection , announcements	19	3763	September 11, 2019
📖 Readme: How to see my language on Common Voice Common Voice announcements	35	14450	May 10, 2022

Sentence collection for Belarusian – request for advice

Related topics