Sentence collection tool development topic

[nukeador (Rubén Martín [❌ taking a break from Mozilla])]

Usability features

• Inform user why sentences were autorejected (length, digits, abbreviation, etc).
• Bug: Hitting “submit” on the add sentences form sometimes does nothing. For example when I tried to just copy and past the 438 sentences I wrote here https://pastebin.com/C0YpxtSJ it will not accept it and fails without any user feedback. Certain subsets of the paste are able to progress to the next screen. I’m not able right now to investigate further what is happening, but is is some bug. If I figure that issue out I can open an issue/PR on github.

These two issues are tracking that:

Issue: Add page. Explain when possible why each sentence has been rejected.

Issue: Make sure all sentences go through

[dabinat]

Here are my first impressions:

• The registration process is confusing - you have to first login with your desired username and password in order to register. I think there should be an actual registration page, even if it functions largely the same as the login page, because it’s more logical and mirrors how other sites work.

• The review process was also confusing to me. You tap Yes and it turns green. Does that mean it’s approved? But it’s still there when I return to the page. Oh, I have to physically click the Submit button, which was right by the page number box, so I thought it was for changing the page. Having it auto-submit, hide the sentence and show the next sentence would be a lot better.

• What is the purpose of the Skip button? I can choose to ignore sentences without consequence and I can navigate to any page I like, so what advantage does skipping provide?

• It seems pretty easy to add invalid sentences. I entered a period for the sentence and the source and it told me the sentence already exists (implying it would have allowed it if it didn’t). I then changed it to a comma in the sentence box and it went through and ended up as a blank sentence.

Other suggestions:

• I don’t think users should be allowed to review their own sentences.

• Should it reject if the first letter of a sentence doesn’t begin with a capital letter? That might help to catch copy and paste errors where there was an errant newline in the middle of the sentence.

[nukeador (Rubén Martín [❌ taking a break from Mozilla])]

The registration process is confusing - you have to first login with your desired username and password in order to register. I think there should be an actual registration page, even if it functions largely the same as the login page, because it’s more logical and mirrors how other sites work.

We decided to stick with the current login system to avoid adding additional work for this version, in the next deployment we have added some explanation about how it works in the same page.

The review process was also confusing to me. You tap Yes and it turns green. Does that mean it’s approved? But it’s still there when I return to the page. Oh, I have to physically click the Submit button, which was right by the page number box, so I thought it was for changing the page. Having it auto-submit, hide the sentence and show the next sentence would be a lot better.

Good idea, currently we are going to stick with this workflow (click yes on everything you want to approve and then finally submit) but we can see what other improvement we can do in the future, I’ve opened this issue to track it.

What is the purpose of the Skip button? I can choose to ignore sentences without consequence and I can navigate to any page I like, so what advantage does skipping provide?

The skip button is broken right now, we plan to remove it in the next deployment.

It seems pretty easy to add invalid sentences. I entered a period for the sentence and the source and it told me the sentence already exists (implying it would have allowed it if it didn’t). I then changed it to a comma in the sentence box and it went through and ended up as a blank sentence.

Can you describe this in more detail in a new issue? I don’t know if I fully understand what you got.

I don’t think users should be allowed to review their own sentences.

This is intentional since we know a lot of people will be adding a lot of sentences from different sources other than their own creation.

Should it reject if the first letter of a sentence doesn’t begin with a capital letter? That might help to catch copy and paste errors where there was an errant newline in the middle of the sentence.

Not sure, this might limit the ability for people to get sentences from long paragraphs and split them into smaller ones you can still read and make sense.

[mkohler (Michael Kohler)]

It seems pretty easy to add invalid sentences. I entered a period for the sentence and the source and it told me the sentence already exists (implying it would have allowed it if it didn’t). I then changed it to a comma in the sentence box and it went through and ended up as a blank sentence.

oh, I’ve filed https://github.com/Common-Voice/sentence-collector/issues/58 for this…

Should it reject if the first letter of a sentence doesn’t begin with a capital letter? That might help to catch copy and paste errors where there was an errant newline in the middle of the sentence.

This might work in English, but then we would need to define this per language as not all language probably use uppercase?

I will fix a few more bugs now and then deploy again.

[mkohler (Michael Kohler)]

I’ve deployed the latest version with many bugfixes, however the skip button will still be there. Will need to think about that removal a bit more. It should however not mark the sentence as approved/rejected anymore and do nothing. This is tracked in https://github.com/Common-Voice/sentence-collector/issues/44

[mkohler (Michael Kohler)]

I’ve deployed a lot more fixes for both bugs and UX topics. Would be great if all of you can keep testing to make sure I didn’t introduce new bugs

[nukeador (Rubén Martín [❌ taking a break from Mozilla])]

Thanks @txopi @DNGros @davidak @dabinat @jef.daniels and everyone who is helping with the QA phase, we have been able to fix a lot of issues and make the tool better thanks to your help.

Let’s do a final push in the next couple of days to make sure we are ready to move the tool to beta phase, let’s keep testing and reporting issues.

In the beta phase we will clean-up the database and offer to everyone in the common voice community who has been asking to get their sentences included to star using it as the main channel for sentence submission and review.

Sentences added and reviewed in the beta phase will start being incorporated in the main Common Voice site. We expect some languages to reach out the 5000 sentences and allow them to enable the voice collection

[mkohler (Michael Kohler)]

I have deployed a new version with several fixes. See the “CHANGELOG” column in https://github.com/Common-Voice/sentence-collector/projects/1 . Thanks everyone for your feedback and reporting bugs, this is getting better and better

[DNGros]

Ok, great. Sorry for not checking for existing issues before commenting.

[DNGros]

It would be great to hear from @josh_meyer or others involved with that team especially on the issue around acronyms as well as around how the 14 word limit was chosen. Specifically on acronyms I worry whether systems trained on Common Voice will be able to handle the acronyms that occur all over everyday speech, unless we allow them in the dataset.

Other areas that still seems to need more clarity is the issues around non-A-Z characters should be allowed. I was going through and reviewing some more sentences that are on the site now, and was not sure whether “Lucas played in the São Paulo soccer team” should be rejected or accepted.

[Codigo_Logo_Programacao_e_Inteligencia_Artificial (Pedro Lima)]

I sent Portuguese sentences to the English one by accident, not sure if I mismatch the combobox or the site didn’t recognize Portuguese and the sent those sentences to the English dataset, How do I remove those sentences?

[nukeador (Rubén Martín [❌ taking a break from Mozilla])]

I sent Portuguese sentences to the English one by accident, not sure if I mismatch the combobox or the site didn’t recognize Portuguese and the sent those sentences to the English dataset, How do I remove those sentences?

Currently there is no way to remove sentences, but don’t worry, we’ll clean the database before moving to the beta phase.

[mkohler (Michael Kohler)]

I’ve just deployed the latest changes to the website. You can find the changelog here: https://github.com/Common-Voice/sentence-collector/releases

[jumasheff]

Hey, do you have an endpoint, where I can post all the scraped data I have? The texts are scraped from a news website (ky.kloop.asia) a founder of which has generously shared all of its contents under CC0 (via facebook chat conversation, a screenshot can be shared upon request). It’s boring to copy-paste all the news articles from 2011 to 2018, you know
Or is it easier for you to accept all text data by email or other means? Perhaps a github repo with all the texts will work for you?

[nukeador (Rubén Martín [❌ taking a break from Mozilla])]

Hey, do you have an endpoint, where I can post all the scraped data I have? The texts are scraped from a news website (ky.kloop.asia) a founder of which has generously shared all of its contents under CC0 (via facebook chat conversation, a screenshot can be shared upon request). It’s boring to copy-paste all the news articles from 2011 to 2018, you know
Or is it easier for you to accept all text data by email or other means? Perhaps a github repo with all the texts will work for you?

We want to force all sentences to go through the sentence collector interface. Why? Because we want to ensure we run a few clean-up algorithms on them and they are properly reviewed by the rest of the community (quality is super important for the machine learning algorithm).

If the problem you are describing is being able to send a lot of sentences, we are improving the tool to allow you to just c&p paste a massive amount of them in one single step. Please, test the tool and let us know how can we improve the workflow (maybe in the future we can have an endpoint for adding sentences with your username)

[nukeador (Rubén Martín [❌ taking a break from Mozilla])]

Update (January 14th)

Again, thanks so much to everyone testing the tool and reporting issues, we have tremendously improved the quality of the tool in the past week and a half

Currently we don’t have any outstanding bugs, so if that remains the same we plan to clean-up the testing database and move to beta phase by the end of this week.

That means we still have a few days more to jump into the tool and make sure everything is working as expected.

We will also evaluate all feature requests and define priorities for the upcoming releases and we would love to get community involved in this, @mkohler and I will design a process to allow people over here to have the chance to influence what’s coming next.

Thanks!

[dabinat]

Does beta mean the sentences will be submitted to the real site or will they still be in a test database that gets wiped afterwards?

[nukeador (Rubén Martín [❌ taking a break from Mozilla])]

Sentences starting in the beta phase will be passed to the real site once they get enough positive reviews (currently 2/3, same as we are using for voice reviews).