Common Voice sentence collector is very slow

I have noticed that is extremely slow. For example, uploading reviewed sentences takes roughly 5 times longer than actual reviewing.

Have you considered providing more than 5 sentences? In my opinion, 50 seems much more suitable.

Loading is also slow, but it’s another issue which is probably much harder to solve.

The backend is going to be upgraded to a completely different solution probably soon, which might help a bit with the performance.

As for the amount of sentences displayed at once, I’d actually argue that the 5 is better than if there were any more. On today’s average monitor, the five can be displayed without any scrolling, and it separates the review process into mentally more easily manageable chunks, whereas 50 sentences per page would just look scary. I feel like I probably wouldn’t have been able to review my something 7k-ish sentences were they 50 a page.

1 Like

Okay, maybe 50 is too much. My point is that time taken to review sentences should be larger than time to upload them.

Another option would be having 5 sentences per chunk, but upload every X chunks. Or maybe upload in background?

My experience is that slow uploading makes reviewing unpleasant.

Again, the uploading is something that should be vastly improved by changing the sentence collector backend soonish, @mkohler could probably give you more details or eta on that.

There definitely will be performance improvements with the new version (and other nice goodies!). I don’t have numbers right now, but would indeed be interesting.

I hope I can find time this weekend to finish this project from my side.

1 Like