Description
Details
Related Objects
- Mentioned In
- rOMWC720e7207cb2d: Add missing extra namespaces from prod config to labs version
rEFLW0e6b88570d0e: Add debugging code for invalid titles in WikiReference
rMEXTe62daaf6d29f: Updated mediawiki/extensions Project: mediawiki/extensions/Flow…
T110407: Output of wmf-beta-update-databases.py is not clear on errors
Event Timeline
Traceback (most recent call last): File "/usr/local/bin/wmf-beta-update-databases.py", line 98, in <module> sys.exit(main()) File "/usr/local/bin/wmf-beta-update-databases.py", line 95, in main run_updates(dblist, args.batch) File "/usr/local/bin/wmf-beta-update-databases.py", line 51, in run_updates do_wait(procs) File "/usr/local/bin/wmf-beta-update-databases.py", line 32, in do_wait raise Exception("command: ", cmd, "output: ", f.read()) Exception: ('command: ', '/usr/local/bin/mwscript update.php --wiki=enwiki --quick', 'output: ', '') Build step 'Execute shell' marked build as failure
This means that /usr/local/bin/mwscript update.php --wiki=enwiki --quick ended with a >0 exit code (1). Not sure yet why.
Results from manual run on deployment-bastion:
$ sudo -u jenkins-deploy /usr/local/bin/mwscript update.php --wiki=enwiki --quick Catchable fatal error: Argument 7 passed to Flow\Model\WikiReference::__construct() must be an instance of Title, null given, called in /mnt/srv/mediawiki-staging/php-master/extensions/Flow/includes/Model/WikiReference.php on line 67 and defined in /mnt/srv/mediawiki-staging/php-master/extensions/Flow/includes/Model/WikiReference.php on line 24
Change 236071 had a related patch set uploaded (by Mattflaschen):
Add debugging code for invalid titles in WikiReference
Change 236071 merged by jenkins-bot:
Add debugging code for invalid titles in WikiReference
2015-09-04 19:26:30 deployment-bastion enwiki Flow INFO: Flow\Model\WikiReference::makeTitle: Invalid title. Namespace: 100, Title text: Contents/Health_and_fitness {"private":false}
mysql> SELECT workflow_wiki, ref_src_object_id, ref_src_object_type, ref_src_workflow_id, ref_src_namespace, ref_src_title, ref_target_namespace, ref_target_title, ref_type, ref_src_wiki FROM flow_wiki_ref JOIN flow_workflow ON ref_src_workflow_id = workflow_id WHERE ref_target_namespace = 100 AND ref_target_title = 'Contents/Health_and_fitness'; +---------------+-------------------+---------------------+---------------------+-------------------+---------------+----------------------+-----------------------------+----------+--------------+ | workflow_wiki | ref_src_object_id | ref_src_object_type | ref_src_workflow_id | ref_src_namespace | ref_src_title | ref_target_namespace | ref_target_title | ref_type | ref_src_wiki | +---------------+-------------------+---------------------+---------------------+-------------------+---------------+----------------------+-----------------------------+----------+--------------+ | enwiki | �n@��!w� | post | �븖�!w� | 1 | Flow | 100 | Contents/Health_and_fitness | link | enwiki | +---------------+-------------------+---------------------+---------------------+-------------------+---------------+----------------------+-----------------------------+----------+--------------+ 1 row in set (0.00 sec)
> var_export( gettype( Title::makeTitle( 100, 'Contents/Health_and_fitness' ) ) ); 'object' > var_export( Title::makeTitle( 100, 'Contents/Health_and_fitness' )->getArticleId() ) 0 > var_export( gettype( Flow\Model\WikiReference::makeTitle( 100, 'Contents/Health_and_fitness' ) ) ) 'NULL'
It doesn't exist, but that doesn't explain the error.
Title::makeTitleSafe fails, but Title::makeTitle returns a Title. This is because MWNamespace::exists( 100 ) is false (there is no such namespace on enwiki Beta currently).
Change 236206 had a related patch set uploaded (by Mattflaschen):
Add missing extra namespaces from prod config to labs version
It does exist on prod, so that patch should restore the config on Labs so that's a valid title again.
Change 236206 merged by jenkins-bot:
Add missing extra namespaces from prod config to labs version
This is weird. It switched from red to gray (https://integration.wikimedia.org/ci/view/Beta/job/beta-update-databases-eqiad/).
In https://integration.wikimedia.org/ci/view/Beta/job/beta-update-databases-eqiad/2692/consoleFull (the last red), it failed on enwiki, which is what led to me fixing https://gerrit.wikimedia.org/r/236206 .
In https://integration.wikimedia.org/ci/view/Beta/job/beta-update-databases-eqiad/2693/consoleFull the last output is:
00:20:18 ...Update 'FlowUpdateWorkflowPageId' already logged as completed. 00:20:18 ...Update 'FlowCreBuild timed out (after 45 minutes). Marking the build as aborted.
This sounds like FlowCreateTemplates.php (which is also immediately after FlowUpdateWorkflowPageId.php in Hooks.php).
But then it gets truncated (due to the build timing out?). I think it may actually be another wiki (enwiki?) timing out (probably on FlowFixLinks), not en_rtlwiki, despite the last "Going to run" being en_rtlwiki. They run in parallel.
I just started:
sudo -u jenkins-deploy /usr/local/bin/mwscript update.php --wiki=enwiki --quick
(this time in a screen session).
It successfully completed in the screen session (1 h 27 min 11 s. for the whole update.php run), so it should be marked as completed, and won't run automatically in update.php again.
I also triggered a run through the Jenkins interface, which was successful.
However, it looks like my manual shell run was running at the same time as the hourly Jenkins update.php one. So it may be in an inconsistent state.
I'm re-running it now in shell with force. It shouldn't conflict this time because it's marked as completed so update.php skips it:
sudo -u jenkins-deploy /usr/local/bin/mwscript extensions/Flow/maintenance/FlowFixLinks.php --wiki=enwiki --force
- ref_target_namespace=100 exists betalabs enwiki; testwiki does not have it.
- Since Sep05 builds are green on https://integration.wikimedia.org/ci/view/Beta/job/beta-update-databases-eqiad/