# Sentence & Clip Reporting Strategy

**URL:** <https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530>\
**Category:** Common Voice\
**Tags:** feedback\
**Created:** [August 7, 2019, 4:09pm UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530 "2019-08-07T16:09:02Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![gregor](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/gregor/32/28922_2.png) [@gregor](https://discourse.mozilla.org/u/gregor)\
**Post date:** [August 7, 2019, 4:09pm UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/1 "2019-08-07T16:09:02Z")

</div>

Heya,

We’ve had the sentence & clip reporting feature up for about a month now. So far the reports just go into our database, with no automatic action being taken. Attached to this post you’ll find all the reports we’ve received so far (Content Warning for the reported clips with the “offensive-speech” tag):

> **[reported\_clips-2019-08-07.tsv](https://drive.google.com/file/d/1zjFHWjTkq2W5bSAg8urBOdv9qU6jc-yl/view?usp=sharing)**
>
> Google Drive file.

  

> **[reported\_sentences-2019-08-07.tsv](https://drive.google.com/file/d/1lSY6r5stGZCpc61EV6FJhGsPhvTQsiN-/view?usp=sharing)**
>
> Google Drive file.

Some ideas that were already floated with regards to what we should do based on the reports:

- Auto-downvote clips that people report (only for particular reasons given?)
- Disable recorded sentences and mark their respective clips as invalid

There might be more or different things we can do here:

- How do you think we should deal with sentences and clips with reports?
- Should we deal differently for each category? How?

Thank you!

---

<div class="post-metadata">

**Author:** ![dabinat](https://avatars.discourse-cdn.com/v4/letter/d/8dc957/32.png) [@dabinat](https://discourse.mozilla.org/u/dabinat)\
**Post date:** [August 8, 2019, 3:39am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/2 "2019-08-08T03:39:49Z")

</div>

I looked at a few of the clips and it seems like people are flagging them up for things like bad quality which you’re supposed to use the No button for. So clearly there is confusion over when to use No and when to use Report.

Another thing is that choosing the “Other” option seems to assign it to the clip when the complaint could be about the sentence. There’s no way to specify.

In terms of what actions should be taken:

1. Flagging up a clip should count as a No vote for that user.

2. We’ve had users record 50 or 100 clips in a row of racist abuse. There needs to be some way of getting those other clips out of rotation quickly once a certain number have been flagged. Maybe if 3 different users flag 3 different clips by that user in a certain space of time it puts that user’s remaining clips in quarantine? It could be limited to apply only to users with new accounts or with no approved recordings.

3. Five votes for the same reason on a sentence removes it from rotation. Eventually there should be some way of viewing this in Sentence Collector and submitting a corrected one if necessary.

---

<div class="post-metadata">

**Author:** ![belkacem77](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/belkacem77/32/22019_2.png) [@belkacem77](https://discourse.mozilla.org/u/belkacem77)\
**Post date:** [September 9, 2019, 8:58am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/3 "2019-09-09T08:58:07Z")

</div>

I’d like suggest to add admin/admins to manage the feedback. Sometimes people record well misspelled sentences. So, we can correct wrong sentences rather that losing recordings.

---

<div class="post-metadata">

**Author:** ![belkacem77](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/belkacem77/32/22019_2.png) [@belkacem77](https://discourse.mozilla.org/u/belkacem77)\
**Post date:** [September 9, 2019, 9:01am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/4 "2019-09-09T09:01:11Z")

</div>

@gregor, the tsv file is it utf8 encoded?

---

<div class="post-metadata">

**Author:** ![gregor](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/gregor/32/28922_2.png) [@gregor](https://discourse.mozilla.org/u/gregor)\
**Post date:** [September 9, 2019, 11:34am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/5 "2019-09-09T11:34:48Z")

</div>

that’s correcto! 🙂

---

<div class="post-metadata">

**Author:** ![nukeador](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/nukeador/32/20533_2.png) [@nukeador](https://discourse.mozilla.org/u/nukeador)\
**Post date:** [September 10, 2019, 11:25am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/6 "2019-09-10T11:25:33Z")

</div>

How would you scale your suggestion for languages where we can potentially have a lot of reports?

Is there a way to crowdsource what you are proposing so we don’t have to rely on just a few individuals (potential bottleneck)?

---

<div class="post-metadata">

**Author:** ![belkacem77](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/belkacem77/32/22019_2.png) [@belkacem77](https://discourse.mozilla.org/u/belkacem77)\
**Post date:** [September 14, 2019, 6:13am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/7 "2019-09-14T06:13:15Z")

</div>

@nukeador ,  
I would suggest an admin (admins) for that locales. That’s why I’m asking for. I don’t know how other locales are organised. For Kabyle, We have an important number of graduates from language departments that can help to correct. I’m planning a traning session for some (they are not techs) to help them with GitHub since there is no a suitable UI to correct.  
NB: Some reported errors, actually aren’t.

---

<div class="post-metadata">

**Author:** ![Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://avatars.discourse-cdn.com/v4/letter/c/dfb087/32.png) [@Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://discourse.mozilla.org/u/Codigo_Logo_Programacao_e_Inteligencia_Artificial)\
**Post date:** [September 14, 2019, 8:28am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/8 "2019-09-14T08:28:52Z")

</div>

@belkacem77 I think that’s a great Idea, I’m seeing some errors in the Portuguese corpus, I report the sentences, but nothing happens. If I had access to the repo, I could fix them and do a commit.

---

<div class="post-metadata">

**Author:** ![belkacem77](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/belkacem77/32/22019_2.png) [@belkacem77](https://discourse.mozilla.org/u/belkacem77)\
**Post date:** [September 14, 2019, 12:04pm UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/9 "2019-09-14T12:04:53Z")

</div>

@Codigo_Logo_Programacao_e_Inteligencia_Artificial, You can correct them on Github, but it’s not a suitable UI for people who are confortbale with such tools.

---

<div class="post-metadata">

**Author:** ![Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://avatars.discourse-cdn.com/v4/letter/c/dfb087/32.png) [@Codigo\_Logo\_Programacao\_e\_Inteligencia\_Artificial](https://discourse.mozilla.org/u/Codigo_Logo_Programacao_e_Inteligencia_Artificial)\
**Post date:** [September 15, 2019, 7:07am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/10 "2019-09-15T07:07:01Z")

</div>

@belkacem77 I’m glad to hear this, I’ll take some time this week to improve our repo, certainly will be a great step towards our goal.

---

<div class="post-metadata">

**Author:** ![nukeador](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/nukeador/32/20533_2.png) [@nukeador](https://discourse.mozilla.org/u/nukeador)\
**Post date:** [September 16, 2019, 10:52am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/11 "2019-09-16T10:52:02Z")

</div>

My current thinking is how we can implement something we can scale and that can be managed by the community crowd.

The same way we are doing to validate clips, so we have more eyes and hands to help vs a few individuals.

---

<div class="post-metadata">

**Author:** ![dabinat](https://avatars.discourse-cdn.com/v4/letter/d/8dc957/32.png) [@dabinat](https://discourse.mozilla.org/u/dabinat)\
**Post date:** [November 2, 2019, 4:05am UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/12 "2019-11-02T04:05:30Z")

</div>

Has a policy for this been determined yet? Because now that a lot of the harder foreign words have been filtered out, I’m finding the most common reason I flag a sentence is because of poor grammar. Some examples:

> In those days the councillors was called commissioners.

> With these schools they creates the Easton Valley Community School District.

> The storm caused severe flooding states such as New Jersey, New York and Pennsylvania.

> The is the only line.

I can’t really think of a good way of automatically filtering these out, so I think the reporting function is really the only way. Of course, this requires someone to actually see it (which is a bad user experience) and flag it, by which time it’s probably already been recorded, but I really can’t think of a way to do it accurately without human review.

---

<div class="post-metadata">

**Author:** ![nukeador](https://sea1.discourse-cdn.com/flex001/user_avatar/discourse.mozilla.org/nukeador/32/20533_2.png) [@nukeador](https://discourse.mozilla.org/u/nukeador)\
**Post date:** [November 4, 2019, 12:28pm UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/13 "2019-11-04T12:28:44Z")

</div>

Thanks for pinging back on this.

Since we have been moving our main dev around we haven’t been able to properly prioritize this feedback.

@mbranson is this something we can make sure we have in the backlog so it is properly triaged when we have time?

Thanks!

---

<div class="post-metadata">

**Author:** ![mbranson](https://avatars.discourse-cdn.com/v4/letter/m/c68b51/32.png) [@mbranson](https://discourse.mozilla.org/u/mbranson)\
**Post date:** [December 2, 2019, 6:57pm UTC](https://discourse.mozilla.org/t/sentence-clip-reporting-strategy/43530/14 "2019-12-02T18:57:27Z")

</div>

Just realized I didn’t respond here, apologies! Yes, this is captured in our backlog and we’re working to prioritize this among various other needs / requests as part of 2020 planning. In the immediate future the Common Voice team focus is on infrastructure improvements, database optimization and releasing the latest dataset. In the meantime please keep providing feedback and proposals here, our goal is to incorporate these learnings into the reporting feature improvements.