Page MenuHomePhabricator
Create Task
Maniphest T374093

action=translationaids is slow sometimes
Closed, ResolvedPublic2 Estimated Story PointsBUG REPORT
Actions

Description

@Nikki reported on the translatewiki.net channel that some action=translations requests are slow in translatewiki (e.g. >5 seconds)

First we inspected the times keys in the API response, but they did not have any especially slow parts.

Manually debugging such request revealed more clues:

[2024-09-05T10:15:13.860128+00:00] wfDebug.DEBUG:   Start request GET /w/api.php?action=translationaids&format=json&title=MediaWiki%3ATuesday%2Fde&uselang=en-gb

# Request setup follows by loading message groups
[2024-09-05T10:15:13.904334+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:aggregate-groups|#|v [] []
[2024-09-05T10:15:13.904401+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:aggregate-groups|#|t [] []
[2024-09-05T10:15:13.909812+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:file-based-groups|#|v [] []
[2024-09-05T10:15:13.909893+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:file-based-groups|#|t [] []
[2024-09-05T10:15:13.921711+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:hook-defined-groups|#|v [] []
[2024-09-05T10:15:13.921816+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:hook-defined-groups|#|t [] []
[2024-09-05T10:15:13.922394+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:message-bundles|#|v [] []
[2024-09-05T10:15:13.922440+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:message-bundles|#|t [] []
[2024-09-05T10:15:13.922755+00:00] rdbms.DEBUG: Wikimedia\Rdbms\LoadBalancer::reuseOrOpenConnectionForNewRef: reusing connection for 0/translatewiki_net-bw_ [] []
[2024-09-05T10:15:13.923741+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:translatable-pages|#|v [] []
[2024-09-05T10:15:13.923807+00:00] memcached.DEBUG: MemCache: got WANCache:translatewiki_net-bw_:translate-mg:translatable-pages|#|t [] []

# Soon after that a lof of blogs are loaded from cache
[2024-09-05T10:15:14.049443+00:00] memcached.DEBUG: MemCache: got WANCache:global:SqlBlobStore-blob:translatewiki_net-bw_:tt%3A11296735|#|v [] [] (repeated many times with different ids)

# Then bunch of Localizations are loaded, for all custom languages?
[2024-09-05T10:15:14.155694+00:00] localisation.DEBUG: LocalisationCache::loadCoreData: got localisation for abr from source [] []
[...]
[2024-09-05T10:15:14.298275+00:00] localisation.DEBUG: LocalisationCache::loadCoreData: got localisation for zmi from source [] []

# Then we start processing machine translation requests, look at the timestamps!
[2024-09-05T10:15:14.301996+00:00] memcached.DEBUG: get(translatewiki_net-bw_:translate-service-Google) [] []
[2024-09-05T10:15:14.302273+00:00] memcached.DEBUG: get(translatewiki_net-bw_:translate-tmsug-pairs-Google) [] []
[...]
[2024-09-05T10:15:20.357548+00:00] memcached.DEBUG: get(translatewiki_net-bw_:translate-tmsug-pairs-MinT) [] []
[2024-09-05T10:15:20.367796+00:00] memcached.DEBUG: MemCache: sock i:0; got translatewiki_net-bw_:translate-tmsug-pairs-MinT [] []

# Some more stuff and shutdown happens a bit later
[2024-09-05T10:15:21.252885+00:00] session.DEBUG: Saving all sessions on shutdown [] []

The low hanging fruit here is that the tmsug-pairs-Apertium/Mint key is called thousands of times (and in case of MinT, the value is quite large) causing majority of the slowdown.

Details

SubjectRepoBranchLines +/-
MachineTranslationAid: Remove language pair checks to speed up the processmediawiki/extensions/Translatemaster+13 -8
TranslationWebService: Add process cache to store supported languagesmediawiki/extensions/Translatemaster+4 -1
Customize query in gerrit

Event Timeline

Nikerabbit created this task.Sep 5 2024, 10:28 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 5 2024, 10:28 AM
Nikerabbit renamed this task from action=translationsaids is slow sometimes to action=translationaids is slow sometimes.Sep 5 2024, 10:28 AM
Nikerabbit added a comment.Sep 5 2024, 10:32 AM

\MediaWiki\Extension\Translate\TranslatorInterface\Aid\MachineTranslationAid::populateQueries is calling isSupportedLanguages() in a loop, and there is no process cache anywhere so it hits the cache.

Nikerabbit triaged this task as Medium priority.Sep 12 2024, 8:30 AM
Nikerabbit added a project: LPL Essential (LPL Essential 2024 Jul-Oct).
Nikerabbit moved this task from Backlog to translation aids on the MediaWiki-extensions-Translate board.
hueitan claimed this task.Sep 25 2024, 7:56 AM
hueitan moved this task from Sprint Backlog to In Progress on the LPL Essential (LPL Essential 2024 Jul-Oct) board.
gerritbot added a comment.Sep 30 2024, 2:05 PM

Change #1076767 had a related patch set uploaded (by Huei Tan; author: Huei Tan):

[mediawiki/extensions/Translate@master] MachineTranslationAid: Remove language pair checks to speed up the process

https://gerrit.wikimedia.org/r/1076767

gerritbot added a project: Patch-For-Review.Sep 30 2024, 2:05 PM
Nikerabbit moved this task from In Progress to Prioritized/ Next on the LPL Essential (LPL Essential 2024 Jul-Oct) board.Tue, Oct 29, 1:54 PM
Nikerabbit removed hueitan as the assignee of this task.Thu, Nov 7, 8:42 AM
Nikerabbit added a subscriber: hueitan.
Nikerabbit edited projects, added LPL Essential (LPL Essential 2024 Nov-Dec); removed LPL Essential (LPL Essential 2024 Jul-Oct).Mon, Nov 11, 9:28 AM
Nikerabbit moved this task from Backlog to Ready for dev on the LPL Essential (LPL Essential 2024 Nov-Dec) board.
Nikerabbit set the point value for this task to 2.Mon, Nov 11, 9:33 AM
gerritbot added a comment.Mon, Nov 11, 8:26 PM

Change #1089852 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] TranslationWebService: Add process cache to store supported languages

https://gerrit.wikimedia.org/r/1089852

abi_ moved this task from Ready for dev to Needs Review on the LPL Essential (LPL Essential 2024 Nov-Dec) board.Tue, Nov 12, 4:43 AM
abi_ claimed this task.Tue, Nov 12, 4:48 AM
gerritbot added a comment.Tue, Nov 12, 11:24 AM

Change #1089852 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] TranslationWebService: Add process cache to store supported languages

https://gerrit.wikimedia.org/r/1089852

Nikerabbit moved this task from Needs Review to Pending Deployment on the LPL Essential (LPL Essential 2024 Nov-Dec) board.Tue, Nov 12, 11:30 AM
Nikerabbit removed a project: Patch-For-Review.
gerritbot added a comment.Tue, Nov 12, 11:30 AM

Change #1076767 abandoned by Nikerabbit:

[mediawiki/extensions/Translate@master] MachineTranslationAid: Remove language pair checks to speed up the process

https://gerrit.wikimedia.org/r/1076767

ReleaseTaggerBot added a project: MW-1.44-notes (1.44.0-wmf.4; 2024-11-19).Tue, Nov 12, 12:00 PM
abi_ closed this task as Resolved.Thu, Nov 14, 12:50 PM
abi_ moved this task from Pending Deployment to Done on the LPL Essential (LPL Essential 2024 Nov-Dec) board.

https://translatewiki.net/w/api.php?action=translationaids&format=json&title=MediaWiki%3ATuesday%2Fde&uselang=en-gb is much faster now. <2 seconds

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits