Removing strings in the source file breaks the sync

Pontoon
#1

Hi,

When there are changes to the source file, e.g. a string is deleted, the sync breaks. Here is the error message I got

[ERROR:pontoon.base.errors] 2022-08-03 02:28:56,903 new row for relation "base_locale" violates check constraint "base_locale_approved_strings_check"
DETAIL:  Failing row contains (24, 0, -8, 0, 0, 0, 0, nl, nl, nl, nl-nl, , Dutch, (n != 1), 1,5, Latin, ltr, 31185000, , 47777, 48, 47, , ).
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.CheckViolation: new row for relation "base_locale" violates check constraint "base_locale_approved_strings_check"
DETAIL:  Failing row contains (24, 0, -8, 0, 0, 0, 0, nl, nl, nl, nl-nl, , Dutch, (n != 1), 1,5, Latin, ltr, 31185000, , 47777, 48, 47, , ).

What I understand is that the deleted string has been translated, upon removing it pontoon checks the translated string and it conflicts with the translated status of the string.

Is there a way to solve this? The best result would be that the source file is the ultimate source of the strings. After deleting the source string, pontoon also remove the string and its translations entirely in the database. Because many of the file formats of the source files are updated automatically by tools (e.g. xcode), it would be great to be able to solve it like this.

#2

Are you removing the string from the source file and let Pontoon sync, or are you removing it from the DB? What’s the file format?

#3

I’m removing the string from the source file. I’m hoping that the database strings can follow the source file strings.

The error showed was from the file format .xliff . And I’m also experiencing the same problem on .pot and .xml file formats.

#4

Hi Dave,

You seem to be using the feature as advertised - source strings should be removed in the code repository and on next sync reflected in Pontoon’s DB.

To help debug the issue, could you share the link to the repository that is hooked up with Pontoon? There might be a configuration issue.

#5

Hi Mathjazz,

Thanks for helping out. Here is my config file:

locales = [ "de", "es", "fr", "nl", "zh" ]

[[paths]]
    reference = "locale/en/*.xml"
    l10n = "locale/{locale}/*.xml"
[[paths]]
    reference = "locale/en/django.pot"
    l10n = "locale/{locale}/django.po"
[[paths]]
    reference = "locale/en/test.json"
    l10n = "locale/{locale}/test.json"
[[paths]]
    reference = "locale/en/test.xliff"
    l10n = "locale/{locale}/test.xliff"

I did some debug myself. I found that the problem is that the aggregated stats of the instances of Locale (total_strings, approved_strings, etc.) are all 0 at the beginning. So the sync task tries to update the base_locale table with total_strings=-1 and approved_strings=-1when there is 1 source string deleted, which violates the PostiveIntegerField constraint.

I also found the Teams page doesn’t show the stats in the UI, comparing to the Projects page.

Screenshot_20220811_185958
Screenshot_20220811_1859581454×803 61.6 KB
Screenshot_20220811_190116
Screenshot_20220811_1901161364×663 54.3 KB

#6

I found the issue is that the Locale object total_strings are not aggregated correctly when the project is private. When all the projects are private, the locales stats will all be 0 (total_strings, approved_strings, etc.). Then deletion of the source strings will cause adjust_stats() to write negative numbers to PositiveIntegerField s.

I would create a PR for this issue but it’s out of my depth. Please check if you have time. Thanks

#7

Hi Dave,

Thanks for digging into it!

Could you please file an issue with steps to reproduce?

#8

Hi mathjazz,

Thank you for following up. Here is the issue I created: