Page MenuHomePhabricator
Create Task
Maniphest T357407

Add a script to refresh translation progress stats
Closed, ResolvedPublic4 Estimated Story Points
Actions

Description

From time to time there are issues caused by outdated message group stats. It would be a good thing to have a script that can go over all the table and make sure everything is up to date.

Details

SubjectRepoBranchLines +/-
Rename RefreshTranslationProgressStats to add suffixmediawiki/extensions/Translatemaster+3 -3
RefreshTranslationProgressStats: provide vanity estimatemediawiki/extensions/Translatemaster+17 -7
Optimize RefreshTranslationProgressStatsmediawiki/extensions/Translatemaster+17 -13
Refactor RebuildMessageGroupStatsJob to implement GenericParameterJobmediawiki/extensions/Translatemaster+6 -6
Customize query in gerrit

Related Objects
Search...

Event Timeline

Nikerabbit created this task.Feb 13 2024, 1:23 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 13 2024, 1:23 PM
Nikerabbit set the point value for this task to 2.Feb 13 2024, 1:25 PM
Nikerabbit moved this task from Quarter Backlog to In Review on the Language-Team (Language-2024-January-March) board.
Nikerabbit moved this task from Backlog to Support (LM) on the Localization Infrastructure FY2023-24 board.
Nikerabbit moved this task from Backlog to Statistics on the MediaWiki-extensions-Translate board.
Nikerabbit added a subscriber: jhsoby.
gerritbot added a comment.Feb 13 2024, 1:25 PM

Change 1002972 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[mediawiki/extensions/Translate@master] Add script to refresh translation progress stats

https://gerrit.wikimedia.org/r/1002972

gerritbot added a project: Patch-For-Review.Feb 13 2024, 1:25 PM
Nikerabbit triaged this task as Low priority.Feb 13 2024, 1:49 PM
Nikerabbit added a subtask: T357416: Phan unable to make make its mind whether suppression is needed.
Nikerabbit closed subtask T357416: Phan unable to make make its mind whether suppression is needed as Invalid.Feb 15 2024, 2:58 PM
abi_ moved this task from In Review to Check after deployment on the Language-Team (Language-2024-January-March) board.Feb 20 2024, 1:41 PM
Nikerabbit removed a project: Patch-For-Review.Feb 20 2024, 1:42 PM
Nikerabbit moved this task from Check after deployment to In Progress on the Language-Team (Language-2024-January-March) board.Feb 22 2024, 8:08 AM

The job queue version doesn't work. The script spends a minute or so to construct all the jobs, using 2G of memory, then failing with a cryptic message:

php maintenance/run.php Translate:refresh-translation-progress-stats.php --use-job-queue
Wikimedia\Rdbms\DBUnexpectedError from line 399 of /srv/mediawiki/workdir/includes/libs/rdbms/database/TransactionManager.php: No atomic section is open (got Wikimedia\Rdbms\Database::onTransactionPreCommitOrIdle)
#0 /srv/mediawiki/workdir/includes/libs/rdbms/database/Database.php(2224): Wikimedia\Rdbms\TransactionManager->onCancelAtomicBeforeCriticalSection()
#1 /srv/mediawiki/workdir/includes/libs/rdbms/database/Database.php(1984): Wikimedia\Rdbms\Database->cancelAtomic()
#2 /srv/mediawiki/workdir/includes/libs/rdbms/database/DBConnRef.php(119): Wikimedia\Rdbms\Database->onTransactionPreCommitOrIdle()
#3 /srv/mediawiki/workdir/includes/libs/rdbms/database/DBConnRef.php(640): Wikimedia\Rdbms\DBConnRef->__call()
#4 /srv/mediawiki/workdir/includes/jobqueue/JobQueueDB.php(232): Wikimedia\Rdbms\DBConnRef->onTransactionPreCommitOrIdle()
#5 /srv/mediawiki/workdir/includes/jobqueue/JobQueue.php(375): JobQueueDB->doBatchPush()
#6 /srv/mediawiki/workdir/includes/jobqueue/JobQueue.php(347): JobQueue->batchPush()
#7 /srv/mediawiki/workdir/includes/jobqueue/JobQueueGroup.php(155): JobQueue->push()
#8 /srv/mediawiki/workdir/extensions/Translate/src/Diagnostics/RefreshTranslationProgressStats.php(77): JobQueueGroup->push()
#9 /srv/mediawiki/workdir/maintenance/includes/MaintenanceRunner.php(698): MediaWiki\Extension\Translate\Diagnostics\RefreshTranslationProgressStats->execute()
#10 /srv/mediawiki/workdir/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()
#11 {main}
gerritbot added a comment.Feb 22 2024, 8:13 AM

Change 1005683 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[mediawiki/extensions/Translate@master] Refactor RebuildMessageGroupStatsJob to implement GenericParameterJob

https://gerrit.wikimedia.org/r/1005683

gerritbot added a project: Patch-For-Review.Feb 22 2024, 8:13 AM
gerritbot added a comment.Feb 22 2024, 8:59 AM

Change 1005709 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[mediawiki/extensions/Translate@master] Optimize RefrestTranslationProgressStats

https://gerrit.wikimedia.org/r/1005709

gerritbot added a comment.Feb 22 2024, 6:26 PM

Change 1005683 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Refactor RebuildMessageGroupStatsJob to implement GenericParameterJob

https://gerrit.wikimedia.org/r/1005683

gerritbot added a comment.Feb 22 2024, 6:55 PM

Change 1005709 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Optimize RefreshTranslationProgressStats

https://gerrit.wikimedia.org/r/1005709

ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.20; 2024-02-27).Feb 22 2024, 7:00 PM
gerritbot added a comment.Feb 26 2024, 8:26 AM

Change 1005713 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[mediawiki/extensions/Translate@master] RefreshTranslationProgressStats: provide vanity estimate

https://gerrit.wikimedia.org/r/1005713

gerritbot added a comment.Feb 26 2024, 8:26 AM

Change 1005714 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[mediawiki/extensions/Translate@master] Rename RefreshTranslationProgressStats to add suffix

https://gerrit.wikimedia.org/r/1005714

Nikerabbit moved this task from In Progress to In Review on the Language-Team (Language-2024-January-March) board.Feb 26 2024, 8:27 AM
abi_ moved this task from In Review to Check after deployment on the Language-Team (Language-2024-January-March) board.Feb 26 2024, 5:13 PM
gerritbot added a comment.Feb 26 2024, 5:25 PM

Change 1005713 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] RefreshTranslationProgressStats: provide vanity estimate

https://gerrit.wikimedia.org/r/1005713

gerritbot added a comment.Feb 26 2024, 5:32 PM

Change 1005714 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Rename RefreshTranslationProgressStats to add suffix

https://gerrit.wikimedia.org/r/1005714

Nikerabbit closed this task as Resolved.Feb 27 2024, 11:17 AM
Nikerabbit removed a project: Patch-For-Review.
Nikerabbit moved this task from Check after deployment to Done on the Language-Team (Language-2024-January-March) board.
Nikerabbit changed the point value for this task from 2 to 4.
twn:/srv/mediawiki/workdir$ ./maintenance/run Translate:refresh-translation-progress-stats.php --language=fi
Progress: [#########-----------------------------------------] 18.12% | 1 minute and 5 seconds remaining
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL