Recognition, Rewards and Contribution Pathways

heyhillary · March 2, 2022, 6:40pm

Hey Common Voice Community,

We value every person who contributes to Mozilla initiatives such as Common Voice. A reward and recognition system can help to acknowledge contributors and support community mobilisation efforts.

As you might be aware at Mozilla, programmes have contribution pathways and associated conversion points. This process helps to inform how people are recognised through non-monetary means e.g Swag.

Common Voice doesn’t have currently have conversion points associated with contribution pathways E.g Add-ons Community Process

Update: 2nd March - Common Voice Language Reps

Updated Reward, Enablement and Recongtion

Last week the Common Voice Language Reps gathered to discuss the Reward, Enablement and Recognition Scheme updated proposal [slides].

We did a unintended consequence scanning of the propsal some of the points raised where; How do we honour silent heroes, certification as recognition, the positive and negative behavioural impacts to community dynamics, the ecological impact.

Our discussion provided room for thought regarding the proposal.

At the end of the month (March 2022), I would like to poll the community via discourse to express approval for the suggested Reward, Enablement and Recognition.

Original Discussion: (click arrow to view)

**Ask: We would like to co-create with the community what this could look like.

To support the co-creation process, I have created a google slide deck pack, that provides more context and some ideas I have.

Please check out the slides and share your thoughts to the following questions.

Contribution Pathway [slides 4-13]

Do you agree or disagree with the following contribution pathways for each type of contributor?
What would you change?
How do you think rewards should be distributed over each of the four pathways?
- Four pathways [slides 6-14] : Casual, Active, Core, High-Level

Implementation of a rewards process [slides 14-16]

What are your thoughts on how we could implement this reward and recognition?

Your responses will help to inform the inaugural virtual meeting for Common Voice Reps that will take place in late September. Expression of interest to be a Common Voice Rep will be open soon.

I look forward to hearing your thoughts!

PS. If you would like to localise the thread please DM - thank you!

ftyers · August 12, 2021, 2:35pm

One approach would be to look at how permissions, rights and responsibilities work on Wikipedia/Wikimedia projects. See for example user access levels.

I think that in addition to basing it on how much people are contributing, it could also be based on what kind of rôles they are playing, etc. Not sure if that’s making a lot of sense, but a broad overview would be to also take into account how much people are contributing to self-management as well as the dataset.

kudanai · August 13, 2021, 5:41pm

Went through the reward metrics and noticed that contributions by way of validating submissions do not appear to be on the board.

Given it is a valuable and time consuming activity, and more importantly the route opted by users who still want to contribute but for various reasons are not inclined to submit their voice, it should probably be included as a reward point.

heyhillary · August 16, 2021, 8:43am

Hey,

Thanks so much for pointing that out. I defiantly agree will make the changes.

Githuma · August 16, 2021, 11:08am

Hi Hillary,

Thank you for this initiative. I have just had a look at slide 4-13 and here are some of mu thoughts:

I think the core contributor should pre validate voice clips and sentences which are then confirmed by the high level contributor.
Mentorship. The high level contributors should have regular mentorship programs with core contributors which the core contributors should pass down to active contributors and active conributors pass the same to casual contributors. For the high level conributors, they can get their mentorship from managers and occasionally Mozilla fellows. This can be done every three months. These sessions can also be used to communicate to contributors the state of the project so that they know how their contributions are being used.
More than swag, giving free coupons to different tech courses in machine learning and AI with a focus on Natural Language processing can help boost morale for the contributors. This give them an opportunity to gain a skills that they can make a living with.
Lastly, gadgets and free credits to cloud platforms or GPUs can be thrown into the mix of rewards.

On Slied 14-16:

I like the idea of automated certificates and stickers. Maybe just to add to automation, an automated chatbot that can help guide new contributors.

For now that is my feedback and will give more of my opinions as they come.

daniel.abzakh · August 27, 2021, 7:55pm

I love it!

I propose a rewarding system that counts recordings per invitation.
i.e as a contributor once I reach 225 recordings (max. limit) per language, I unlock an invitation feature for that language, where I am qualified to invite other people. Their recordings count against their accounts and my account, so the more I invite active contributors, the more counted records I get, same goes for validation.
This promotes a different kind of energy among contributors, cooperation over competition, a - you make records, you and I get benefits - rewarding system.

Githuma · August 19, 2021, 2:40pm

This is an interesting way of making sure that contributors can identify proper contributors bring them into the team. It also helps build a sustainable ranking system as the more people you get under you, the more you rise up the ranks.

Fjoerfoks · September 1, 2021, 11:50am

I would take out all items about submitting and validating voice clips, since this are thousands of people you actually don’t know. Why would you reward a High-level contributor for 50 submitted clips, if he/she was able to mobilize a few hundred contributors during an event, that doesn’t make sense.
Are rewardings kept separate from other contributions to Mozilla, or added to a major list?

bozden · September 2, 2021, 10:33am

My 2 cents as random thoughts…

I looked at the numbers as examples. They actually are very low IMHO.

I would not emphasize recordings too much as that would hurt diversity and make the dataset biased. (Actually if I were to design the system, I would disable that ability if the number of recordings reach N times the language average for example).

I would promote CC0 sentence additions as they are hard to come by. Pre-editing and validating takes a lot of time there.

I would also put more than moderate weight to validations (listening) to clear the queues.

It might also be possible to put reward on what is needed in a language dataset: If there are no sentences, that gets higher points, if there are no recordings, recording gets promoted etc.

I don’t know all but many languages lack female voices, some may also lack diversity in tongues, dialects etc. A positive discrimination in these areas would be nice.

But most importantly: A continuous commitment should be more rewarded. In my case (self criticism) I contributed hard for four months but I’ve been away for three due to another project. I don’t mean “stripping the ranks” but an adaptive algorithm could solve this by changing a weight calculated as a function of continuous effort.

Edit:
Discourse already has flags, some of them can also be used to contribute for this task.

What is the ecological impact of sending swag across the world ?
This is important. Although they are very nice to have, if anything is given digital that would be better - not that they don’t create carbon print, but they are already there running.

heyhillary · September 3, 2021, 2:30pm

These are really great points and it’s made me rethink if we do need explicit numbers for contributions on voice and text contributions. As such I’m thinking of removing these.

At the moment this process if focused on Common Voice, as such it’s technical separate from other Mozilla projects but would be shown on the wiki page listed in the post.

heyhillary · September 6, 2021, 1:08pm

Thanks so much for sharing your thoughts and I really like your suggestion of the positive discrimination and areas where the language needs more support.

heyhillary · September 6, 2021, 1:10pm

Hey George,

Thanks so much for your feedback. I wanted to ask two questions. Are there any course providers e.g plurasight in particular that you would be intrested in about ML and AI ? Also it would be ideal, if they provide materials that are localised or could be localisable.