# How can I send sentences to contribute?

**URL:** <https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525>\
**Category:** Common Voice\
**Tags:** sentence-collection\
**Created:** [September 5, 2018, 12:27pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525 "2018-09-05T12:27:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://avatars.discourse-cdn.com/v4/letter/c/dfb087/32.png) [@Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://discourse.mozilla.org/u/Codigo_Logo_Programacao_e_Inteligencia_Artificial)\
**Post date:** [September 5, 2018, 12:27pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/1 "2018-09-05T12:27:01Z")

</div>

Hi, I’m looking forward to the Portuguese language, I wanna contribute, how can I do?  
It seems like we need 5000 sentences, that wouldn’t be hard, there is a corpus with 130k sennteces for portuguese, developed by a speech research team, which we could use. For informal language we could write our own sentences. I’d like to see Spanish, I’m not a spanish speaker, but I think we need to focus on main languages, we don’t have an equivalent of Librivox, Tedx for those languages. Also do you guys have plans to add the Spoken wikipedia to the datasets page?

---

<div class="post-metadata">

**Author:** ![lissyx](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/lissyx/32/67389_2.png) [@lissyx](https://discourse.mozilla.org/u/lissyx)\
**Post date:** [September 5, 2018, 12:32pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/2 "2018-09-05T12:32:52Z")

</div>

I’m sure @reuben can give some hints on portuguese :). BTW, please make sure your 130k dataset can be licensed as CC-0 otherwise it cannot be used for Common Voice.

> [@Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](#):
>
> Also do you guys have plans to add the Spoken wikipedia to the datasets page?

We have some french contributor hacking tooling exactly on that purpose, being able to extract CC-0 compatible Wikipedia content whatever the language (right now, french, but he’s willing to expand it).

---

<div class="post-metadata">

**Author:** ![Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://avatars.discourse-cdn.com/v4/letter/c/dfb087/32.png) [@Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://discourse.mozilla.org/u/Codigo_Logo_Programacao_e_Inteligencia_Artificial)\
**Post date:** [September 5, 2018, 12:35pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/3 "2018-09-05T12:35:26Z")

</div>

Well, so we can use the wikipedia right? Ok, I’ll look forward, with regards to informal language, we could write some sentences to get 5000 and then get started.

---

<div class="post-metadata">

**Author:** ![lissyx](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/lissyx/32/67389_2.png) [@lissyx](https://discourse.mozilla.org/u/lissyx)\
**Post date:** [September 5, 2018, 12:36pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/4 "2018-09-05T12:36:12Z")

</div>

> [@Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](#):
>
> Well, so we can use the wikipedia right

Wikipedia should not be your only component of the dataset, but it can be a part of it. Please ensure you only extract `CC-0` content.

---

<div class="post-metadata">

**Author:** ![lissyx](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/lissyx/32/67389_2.png) [@lissyx](https://discourse.mozilla.org/u/lissyx)\
**Post date:** [September 5, 2018, 12:36pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/5 "2018-09-05T12:36:37Z")

</div>

Here is his code (tested only on french, so far): [https://github.com/jeanbaptisteb/commonvoice-fr/blob/master/Wikipedia\_CC0.py](https://github.com/jeanbaptisteb/commonvoice-fr/blob/master/Wikipedia_CC0.py)

---

<div class="post-metadata">

**Author:** ![J-b](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@J-b](https://discourse.mozilla.org/u/J-b)\
**Post date:** [September 5, 2018, 2:15pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/6 "2018-09-05T14:15:21Z")

</div>

As @lissyx said above, there’s a script to extract content from Wikipedia under the CC0 licence. I still need to fix a couple of bugs on it, and maybe to make it simpler to use.

I’ll try to check this week-end if it’s possible to extract content from the Wikipedia in Portuguese with this script. But feel free to check it directly if you’re comfortable with Python’s scripts and if you have the time to!

As for using the script, I currently strongly recommend to use the parameter “type=‘creation’”, and avoid using the parameter “type=‘all\_content’”. If you use the “type=‘all\_content’” parameter, the script may retrieve non-CC0 content, which is incompatible with the Common Voice licence. This is a bug I need to fix.

If you have any about question about the script, feel free to send me a direct message. I won’t have a lot of time to dedicate to the project in the coming weeks, but I’ll answer your questions as soon as I can.

---

<div class="post-metadata">

**Author:** ![nukeador](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/nukeador/32/20533_2.png) [@nukeador](https://discourse.mozilla.org/u/nukeador)\
**Post date:** [September 5, 2018, 3:18pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/7 "2018-09-05T15:18:31Z")

</div>

Please keep this efforts collecting sentences under public domain, as soon as we have our [sentence collection tool](https://discourse.mozilla.org/t/we-want-your-feedback-improving-the-sentence-collection/30358/) ready we should be able use it to submit, validate and review them so they can be incorporated in the database.

---

<div class="post-metadata">

**Author:** ![nukeador](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/nukeador/32/20533_2.png) [@nukeador](https://discourse.mozilla.org/u/nukeador)\
**Post date:** [September 5, 2018, 3:28pm UTC](https://discourse.mozilla.org/t/how-can-i-send-sentences-to-contribute/31525/8 "2018-09-05T15:28:01Z")

</div>

I’ve created this topic to centralize questions

> [@open\_book Readme: How to see my language on Common Voice](https://discourse.mozilla.org/t/readme-how-to-see-my-language-on-common-voice/31530):
>
> triangular_flag_on_post This information is also now available on the [About Pages](https://commonvoice.mozilla.org/about) on Common Voice Website. Please help us to localise this by joining [Pontoon](https://pontoon.mozilla.org/projects/common-voice/)open_book [Mozilla Voice Community Playbook](https://common-voice.github.io/community-playbook/): The source of truth for setting up and maintain self-sustainable communities. Hello everyone, I would like to open this topic to summarize some of the most asked question we are getting: How do I get my language in Common Voice. There are three steps to have your language ready: …
