This is taken care of.
This is taken care of.
The Polish review tab is currently filled with segments from Lord of The Rings, which is very much not public domain. Didn’t even bother slicing it into sentences… Username is
narid, source is
from the book. (again!).
Thanks for reporting this. These have been removed.
Japanese language collector have the following problems:
Perhaps this is a problem with the corpus.
I went to the source page and checked the "Public Domain version" and it contains the above text. These sources are famous cartoons and games, and they are obviously not in the public domain. The "Public Domain version" file has a [Manga] flag, but some of the sentences are not. Honestly, I can't determine how much of the offending text is in the mix.
ru got some scripts parsed from opensubtitles.org. It’s clearly marked in source, so should be easy to parse.
Thanks for flagging, we’ll take a look on this end and get back to you here. cc @
@sinumade thanks again for flagging this, after review from our Mozilla Legal counterparts it’s been determined that this corpus is not fit for CC0 contribution and all usage should be removed from Common Voice.
@mkohler will work to remove this from the sentence collector, and any sentences that were merged to the primary platform for voice contribution will also be removed from the dataset. cc @phirework who’ll need to take that action.
Japanese language collection, again.
Are the reviewers, users Rrock9312 and Rrock2139 the same person?
This is insane, and a betrayal to the people.
Ping @mkohler for the mentioned copyright violations
Both Rrock users look suspicious, and there is also some asdf user? Could you check if any of them have any sentences marked invalid? If they have none, I’d say those are most definitely invalid reviews and should be deleted (or japanese submissions are without any mistakes, but I have a hard time believing in that)
If there remain less than 5k sentences, I’m afraid japanese collection will have to stop once again (but either way, 5k is awfully little to expect any great amount of useful collecting to be taking place)
If something has already been recorded, what to do then is a question for more qualified people than me, but it could unfortunately end in lot of bad ways
Second party checking the sources would be great, but now reviews at least show the sentence sources, which is still better than nothing
Mozilla staff can’t viably verify all of the sources for all of the languages, even if only because they don’t speak those. The best you can do is to defer to the local communities
@mkohler Hindi dataset,
https://tatoeba.org/eng as source, five random sentences with this source were checked, all were incompatible licence (CC-BY 2.0 FR)
Both the Japanese and Hindi issues have been resolved
Sorry, I can’t access to ‘kinto’ url, so can’t provide in requested format.
Ukrainian sentences contain a lot of text from a single book. And they have very bad quality.
Source: cetation of Володимир Білінський
The book is
“Білінський Володимир Броніславович. КРАЇНА МОКСЕЛЬ, або МОСКОВІЯ”
Very “holy-war” and politized one.
“Ростовсько-суздальська земля до приходу Рюриковичів зі своїми ватагами давно була заселена фінськими племенами”
“Початковий київський літопис доволі точно позначає місця проживання цих племен: він знає”
“Здавалося б, за церковними канонами, «царство Боже й царство кесареве мусять залишатися навіки розділеними»”
“Карамзін був палким великоросом на службі в імперії й Государя”
Link to google book (not sure if it’s legal copy)
And yet another one. It contains a lot of old/rare words. The meaning of some of them I even don’t know.
Book “Меч Арея” by Іван Білик. Link to online copy here
“І не питаєш, чого ради смо прийшли?”
“Пощо тобі князь? — жваво всміхнувся отрок у сорочці.”
Link to source
“Богдан пропустив його слова повз вуха, й усі троє посідали на коней.”
Also, there are a lot of pending sentences from this book. All marked as
“Source: cetation of Іван Білик”. It would be nice to clear them automatically.
These two books kill all pleasure from the project. Unfotunately, 80% of sentences now come from them.
Could you resolve it somehow, please?
@mkohler For the Swedish dataset, there are now a lot of sentences (more than 1000) in the queue with
https://tatoeba.org/ given as source and CC-BY 2.0 FR as the stated license.
я б теж підняв питання про прибирання цієї ЄРЕСІ з базового набору
Hi I added some sentences from Tatoeba to the Swedish sentences collector but I have since learned that the lichens is incompatible with Mozilla common voice. So they need to be removed.
Both Ukranian and Swedish has been taken care of and the migration should be deployed in a few minutes.
Just realized that there are quite a lot of subtitles from the movie The Big Lebowski in the Swedish dataset that likely come from Opensubtitles and has been there at least since August 2019.
Jag är The Dude.
Jag kan inte skicka en räkning till kinesen som pissade på mattan.
(Some of them are easily identified by the use of “The Dude” which apart from the copyright issue obviously isn’t ideal for capturing pronunciation in Swedish since most Swedish-speaking people would still use English pronunciation for that. Also there are some offensive sentences in there.)
The sentences have been removed from the Sentence Collector. Thanks for reporting!