Dear all, is there posibility to download dataset that’s not completed yet? Thank You

[image] aiteam: After that we will prepare it to add it to sentece collector I’ll reinforce my previous message, note that wikipedia extracted sentences are never imported into the sentence collector, please don’t do this. The process is to contribute to the rulesets and blacklist: …

ahh. sh.t - i didnt understand Your idea. Do You already have rulesets and blacklists for polish wikipedia scrapping database?

it seems that at beginning i didnt understand model for wiki scrapping and collector :wink: - i’ll try to get net source of polish datasets to review

No, we don’t, we need technical Polish-speaking contributors to help :slight_smile:

:slight_smile: - if u want me to prepare rules and blacklist just let me know

Hey - is there any hard limit (10 000) on sentence collector?. Coz I’m adding and also renewing new sentences, while counters on profile doesn’t change: Profile: aiteam 10000 sentences added 10000 sentences reviewed … across 1 language(s)

Hello @aiteam , a lot of contribution You have made, can You please add a better source description of sentences origin? “own choosed and edited” may not indicate that sentences are copyright free in my opinion. Some sentences seem broken and it would be a bit hard to read and process them. Like th…

Hey Kuba, i've build semi automatic solution to extract sentences (within nltk) from old movie subtitles. Process is not fully automated and i’m trying to review sentences before i’ll add it to collector (sometimes i can miss low quality senetce). I’ll try to modify process - it will take onl…

Kuba, what about counters on profile - is't safe to add new sentences? BTW I’ve reviewed my code - i had english pickle on tokenizer :wink: so new batches should be much more better now (now process is armed with polish pickle) BR

Polish dataset download

Common Voice

jakub.wrobel7 (Jakub Wrobel7) February 5, 2020, 9:03pm 20

Let’s do it together, see here https://discourse.mozilla.org/t/coordination-of-input-for-polish-language-wiki-scrapper/53380

Topic		Replies	Views
Polish sentences concerns Common Voice sentence-collection , issue , dataset	20	3315	May 4, 2020
Extending our sentence collection capabilities Common Voice sentence-collection , announcements	19	3720	September 11, 2019
Using the Europarl Dataset with sentences from speeches from the European Parliament Common Voice sentence-collection	61	6150	March 28, 2023
Polish dataset from Europarl - help needed Common Voice	14	1210	July 17, 2021
📖 Readme: How to see my language on Common Voice Common Voice announcements	40	14332	May 10, 2022

Polish dataset download

Related topics