Clip length increasing for Common Voice datasets

jesslynnrose · March 26, 2024, 2:49pm

Due to some impassioned and well informed ongoing community discussions and work with our internal experts, the process to begin including clips of up to 15 seconds (an increase of 50% from the previous clip length of 10 seconds) has begun for the Common Voice datasets.

We still have more work to complete to get these clips into future datasets, but you can find more information about our thinking and the process here.

bozden · April 26, 2024, 2:15pm

Hey @jesslynnrose, except for the rule files I mentioned here, is it possible for us to post longer sentences (assuming we adapted the rule files)?

I’m in the process of bulk-posting 15+ word sentences from books in the public domain (where the author died more than 70 years ago), where I already have them added with shorter sentences.

Can you please give some feedback on the status (I’m not sure “dmitrij” is here)?

lonzobrown · January 20, 2025, 8:57pm

I also want to know about that how it work…

bozden · January 21, 2025, 2:24pm

Yes, 15 sec is enabled. The constant is set here:

github.com

common-voice/common-voice/blob/7cd384f1f7916c9f509880cefefc58fb5e6d540a/web/src/components/pages/contribution/speak/speak.tsx#L48


import RecordingPill from './recording-pill'
import { SentenceRecording } from './sentence-recording'
import SpeakErrorContent from './speak-error-content'
import { USER_LANGUAGES } from './firstSubmissionCTA/firstPostSubmissionCTA'
import { castTrueString } from '../../../../utility'

import './speak.css'

const MIN_RECORDING_MS = 1000
const MIN_RECORDING_MS_BENCHMARK = 500
const MAX_RECORDING_MS = 15000
const MIN_VOLUME = 8 // Range: [0, 255].

enum RecordingError {
  TOO_SHORT = 'TOO_SHORT',
  TOO_LONG = 'TOO_LONG',
  TOO_QUIET = 'TOO_QUIET',
}

interface PropsFromState {
  api: API

But you need to create a validation file for your language (or modify any existing to increase) number of allowed words and/or sentence length if an existing validation file is using it. Here is an example which sets max-words to 20 for this purpose:

github.com

common-voice/common-voice/blob/main/server/src/core/sentences/validation/languages/sq.ts

import {
  ERR_NO_ABBREVIATIONS,
  ERR_NO_NUMBERS,
  ERR_NO_SYMBOLS,
  ERR_TOO_LONG,
  ValidatorRule,
} from '../../types'

const tokenizeWords = require('talisman/tokenizers/words')

// Minimum of words that qualify as a sentence.
const MIN_WORDS = 1

// Maximum of words allowed per sentence to keep recordings in a manageable duration.
const MAX_WORDS = 20

const INVALIDATIONS: ValidatorRule[] = [
  {
    type: 'fn',
    fn: (sentence: string) => {

This file has been truncated. show original

To correctly set a value here, you need to know char-speed in your dataset, if it is already released, I can help with that. What is your language code?

Topic		Replies	Views
Discussion: Relaxation of the 10 sec recording limitation Common Voice feedback	10	3878	March 16, 2024
Translation of sentences from other-language corpuses Common Voice sentence-collection	14	2159	November 25, 2022
Discussion: Best practices/steps for increased recording duration? Common Voice sentence-collection	2	1016	July 27, 2024
Proposal: Sentences lenght limit from 14 words to 100 characters Common Voice sentence-collection	11	6751	February 2, 2019
Single Sentence Record Limit feature release Common Voice announcements	18	3106	June 13, 2022

Clip length increasing for Common Voice datasets

Related topics