Page MenuHomePhabricator

dispatchChanges.php does not work with the Wikibase Docker distribution
Closed, ResolvedPublicBUG REPORT

Description

With the Wikibase Docker distribution, dispatchChanges.php doesn't seem to work:

$ docker exec artbase-wikibase php extensions/Wikibase/repo/maintenance/dispatchChanges.php
OutOfBoundsException from line 60 of /var/www/html/extensions/Wikibase/lib/includes/SettingsArray.php: Attempt to get non-existing setting "repoDatabase"
#0 /var/www/html/extensions/Wikibase/repo/maintenance/dispatchChanges.php(93): Wikibase\Lib\SettingsArray->getSetting('repoDatabase')
#1 /var/www/html/extensions/Wikibase/repo/maintenance/dispatchChanges.php(247): Wikibase\Repo\Maintenance\DispatchChanges->getClientWikis(Array)
#2 /var/www/html/maintenance/doMaintenance.php(107): Wikibase\Repo\Maintenance\DispatchChanges->execute()
#3 /var/www/html/extensions/Wikibase/repo/maintenance/dispatchChanges.php(403): require_once('/var/www/html/m...')
#4 {main}

What is repoDatabase, and what value should it contain?

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ubuntu@wikibase:~/wikibase$ docker exec -u www-data artbase-wikibase php extensions/Wikibase/repo/maintenance/dispatchChanges.php --max-time=110
OutOfBoundsException from line 60 of /var/www/html/extensions/Wikibase/lib/includes/SettingsArray.php: Attempt to get non-existing setting "repoDatabase"
#0 /var/www/html/extensions/Wikibase/repo/maintenance/dispatchChanges.php(93): Wikibase\Lib\SettingsArray->getSetting('repoDatabase')
#1 /var/www/html/extensions/Wikibase/repo/maintenance/dispatchChanges.php(247): Wikibase\Repo\Maintenance\DispatchChanges->getClientWikis(Array)
#2 /var/www/html/maintenance/doMaintenance.php(107): Wikibase\Repo\Maintenance\DispatchChanges->execute()
#3 /var/www/html/extensions/Wikibase/repo/maintenance/dispatchChanges.php(403): require_once('/var/www/html/m...')
#4 {main}

Looks like this might have been a fix for this in 1.36, but it is not in 1.35

https://github.com/wikimedia/Wikibase/commit/bf4ccd9ff8e75db1c35aa625684974cfb5c34582

# TODO can probably remove when using 1.36 of Wikibase
$wgWBClientSettings['repoDatabase'] = false;

After adding the LocalSettings changed proposed above ($wgWBClientSettings['repoDatabase'] = false;), changes are getting dispatched, but very irregularly.

The dispatchChanges.php script in most cases is unable to get a "database lock":

ubuntu@wikibase:~/wikibase$ docker exec -u www-data artbase-wikibase php extensions/Wikibase/repo/maintenance/dispatchChanges.php --max-time=60
10:49:33 Starting loop for unlimited passes or 60 seconds
10:49:33 Posted 1 changes to wiki, up to ID 136916, timestamp 20210527104904. Lag is 29 seconds. Next ID is 136916.
10:49:33 Posted 1 changes to artbase, up to ID 136916, timestamp 20210527104904. Lag is 29 seconds. Next ID is 136916.
10:49:33 Wikibase\Repo\Store\Sql\SqlChangeDispatchCoordinator::selectClient: Could not lock any of the candidate client wikis for dispatching
10:49:43 Wikibase\Repo\Store\Sql\SqlChangeDispatchCoordinator::selectClient: Could not lock any of the candidate client wikis for dispatching
10:49:53 Wikibase\Repo\Store\Sql\SqlChangeDispatchCoordinator::selectClient: Could not lock any of the candidate client wikis for dispatching
10:50:03 Wikibase\Repo\Store\Sql\SqlChangeDispatchCoordinator::selectClient: Could not lock any of the candidate client wikis for dispatching
10:50:13 Wikibase\Repo\Store\Sql\SqlChangeDispatchCoordinator::selectClient: Could not lock any of the candidate client wikis for dispatching
10:50:24 Wikibase\Repo\Store\Sql\SqlChangeDispatchCoordinator::selectClient: Could not lock any of the candidate client wikis for dispatching
10:50:34 Done, exiting after 10 passes and 60.152252912521 seconds.

Also it looks like changes are dispatched to 'wiki', which I do not know the origins of, nor am aware of having it set up anywhere.

In general it works, but laggy, and creating big log files. :)

For dispatchChanges.php to work on a Wikibase install, the following conditions have to be met:

  1. The local wiki has to be registered in the "sites" table. This can be done by running addSite, for instance:
$ docker exec -ti wikibase php ./maintenance/addSite.php \
  --language en \
  --filepath 'https://artbase.rhizome.org/w/$1' \
  --pagepath 'https://artbase.rhizome.org/wiki/$1' \
  --server https://artbase.rhizome.org \
  --interwiki-id artbase \
  artbase Rhizome
  1. Add this line to LocalSettings.php (this step will become obsolete with Wikibase version 1.36):
$wgWBClientSettings['repoDatabase'] = false;

Now it should be possible to add local sitelinks within your own wiki.

  1. Perhaps during some unlucky previous experiments, the connection to the local wiki is registered wrongly, which might then introduce big delays into dispatching. Truncate the table wb_changes_dispatch in your database for good measure:
MySQL [wiki]> truncate wb_changes_dispatch;
  1. several scripts have to be run in cronjobs. It seems like good practice right now to run these cronjobs on the docker host machine, not inside the wikibase docker container. Here is how this looks for us:
$ crontab -l
# m  h  dom mon dow   command
  *  *  *   *   *     docker exec -u www-data wikibase php maintenance/runJobs.php
  *  *  *   *   *     docker exec -u www-data wikibase php extensions/Wikibase/repo/maintenance/dispatchChanges.php
  3  3  8   *   *     docker exec -u www-data wikibase php extensions/Wikibase/repo/maintenance/pruneChanges.php
Addshore added a subscriber: danshick-wmde.

Tagging @danshick-wmde as I am sure we should make some docs changes etc after working through this!

Tagging @danshick-wmde as I am sure we should make some docs changes etc after working through this!

Thanks. This comes at a great time, since I just updated the page that covers dispatchChanges.

To clarify, on the current main branch this is deprecated, I believe in both the 1.36 and 1.37 compatible releases this will not be deprecated.

https://github.com/wikimedia/Wikibase/blob/master/RELEASE-NOTES-1.38#L10-L13