Page MenuHomePhabricator

CX2: Translation saving fails after adding new category
Closed, ResolvedPublic

Description

Steps to reproduce:

  1. Start translation with CX2
  2. Add new category.
  3. Watch console and error message shown to user.

Error log:

[CX] Saving Failed. Error code: internal_api_error_DBQueryError
(anonymous) @ VM452:510
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
fireWith @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
process @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
mightThrow @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:49
process @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
setTimeout (async)
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
fireWith @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
fireWith @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
process @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
mightThrow @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:49
process @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
setTimeout (async)
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
add @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
jQuery.Deferred @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:150
then @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
mightThrow @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:49
process @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
setTimeout (async)
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
fireWith @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
fireWith @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
deferred.(anonymous function) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:51
(anonymous) @ VM452:28
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
fireWith @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
done @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:126
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:129
load (async)
send @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:129
ajax @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:125
jQuery.ajax @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:143
ajax @ VM452:27
post @ VM452:26
(anonymous) @ VM452:29
mightThrow @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:49
process @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
setTimeout (async)
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
fire @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:46
add @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:47
(anonymous) @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
jQuery.Deferred @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:150
then @ load.php?debug=false&lang=gu&modules=jquery%2Cmediawiki|mediawiki.legacy.wikibits&only=scripts&skin=vector&version=18kdz1s:50
postWithToken @ VM452:28
mw.cx.TranslationController.processSaveQueue @ VM452:510
run @ load.php?debug=false&lang=gu&modules=ext.visualEditor.core|mw.config.values.wbSiteDetails|oojs-ui-core&skin=vector&version=0j942ve:972
setTimeout (async)
(anonymous) @ load.php?debug=false&lang=gu&modules=ext.visualEditor.core|mw.config.values.wbSiteDetails|oojs-ui-core&skin=vector&version=0j942ve:972
mw.cx.TranslationController.onTargetCategoriesChange @ VM452:516
oo.EventEmitter.emit @ VM452:62
mw.cx.dm.Translation.setTargetCategories @ VM452:90
mw.cx.ui.Categories.onTargetCategoriesChange @ VM452:862
oo.EventEmitter.emit @ VM452:62
OO.ui.mixin.GroupElement.addItems @ load.php?debug=false&lang=gu&modules=ext.visualEditor.core|mw.config.values.wbSiteDetails|oojs-ui-core&skin=vector&version=0j942ve:997
mw.cx.ui.CategoryMultiselectWidget.addTag @ VM452:858
mw.cx.ui.CategoryMultiselectWidget.onInputChoose @ VM452:858
oo.EventEmitter.emit @ VM452:62
mw.loader.implement.ve.ui.MWCategoryInputWidget.onLookupMenuItemChoose @ VM452:808
mw.cx.ui.CategoryInputWidget.onLookupMenuItemChoose @ VM452:855
oo.EventEmitter.emit @ VM452:62
OO.ui.SelectWidget.chooseItem @ load.php?debug=false&lang=gu&modules=ext.visualEditor.core|mw.config.values.wbSiteDetails|oojs-ui-core&skin=vector&version=0j942ve:1057
OO.ui.MenuSelectWidget.chooseItem @ load.php?debug=false&lang=gu&modules=ext.visualEditor.core|mw.config.values.wbSiteDetails|oojs-ui-core&skin=vector&version=0j942ve:1064
OO.ui.SelectWidget.onMouseUp @ load.php?debug=false&lang=gu&modules=ext.visualEditor.core|mw.config.values.wbSiteDetails|oojs-ui-core&skin=vector&version=0j942ve:1049

Event Timeline

  1. Which instance/wiki?
  2. Can you get the actual query error from PHP logs/logstash?
  1. en -> gu. Production.
  1. Getting more info, with debug enabled,
mw.Api error:  internal_api_error_DBQueryError {error: {…}, servedby: "mw1287"}
error:
 code: "internal_api_error_DBQuery Error"
 info: "[WzEPAwpAEDQAAJNFjJQAAACS] Database query error."

Nothing useful in logstash.

I am seeing this in Logstash that looks related:

ContentTranslation\CategoriesStorageManager::exists	10.64.48.19	1290	The MariaDB server is running with the --read-only option so it cannot execute this statement (10.64.48.19)	SELECT  cxc_content  FROM `cx_corpora`    WHERE cxc_translation_id = '496574' AND cxc_section_id = 'CX_CATEGORY_METADATA'   FOR UPDATE

https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-2018.06.25/mediawiki?id=AWQ4jz6R8VFZOIjHFOo2&_g=h@e8b7b86

10.64.48.19 is db1064 which is a slave. So it is correct to have read_only there.
Are you trying to insert on a slave??

Change 441976 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/ContentTranslation@master] Run category query with FOR UPDATE on master DB

https://gerrit.wikimedia.org/r/441976

Change 441976 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Run category query with FOR UPDATE on master DB

https://gerrit.wikimedia.org/r/441976

Pginer-WMF triaged this task as Medium priority.Jun 27 2018, 7:10 AM
Vvjjkkii renamed this task from CX2: Translation saving fails after adding new category to 8baaaaaaaa.Jul 1 2018, 1:02 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed Nikerabbit as the assignee of this task.
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot renamed this task from 8baaaaaaaa to CX2: Translation saving fails after adding new category.Jul 2 2018, 10:05 AM
CommunityTechBot closed this task as Resolved.
CommunityTechBot assigned this task to Nikerabbit.
CommunityTechBot lowered the priority of this task from High to Medium.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added a subscriber: Aklapper.