Page MenuHomePhabricator

Update role::wikidata to work with new (no)build process
Closed, ResolvedPublic

Description

Follow up to T173818: [Epic] Kill the Wikidata build step:

The wikidata role clones https://gerrit.wikimedia.org/r/wikidata/build-resources to provision wikidata in the VM. This should be replaced with cloning all of the individual extensions and any settings changes needed to go with that.

puppet/modules/role/manifests/wikidata.pp
mediawiki::extension { 'WikidataBuildResources':
    remote       => 'https://gerrit.wikimedia.org/r/wikidata/build-resources',
    entrypoint   => 'Wikidata.php',
    composer     => true,
    needs_update => true,
    settings     => template('role/wikidata/shared.php.erb'),
}

mediawiki::settings { 'WikiData-Init':
    priority => $::LOAD_EARLY,
    values   => template('role/wikidata/init.php.erb'),
}

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 1 2017, 12:11 AM

Just for reference (until this is fixed)

If you get errors like:

[Exception MWException] (/vagrant/mediawiki/includes/resourceloader/ResourceLoaderFileModule.php:810) ResourceLoaderFileModule::readScriptFiles: script file not found: "/vagrant/mediawiki/extensions/WikidataBuildResources/extensions/Wikibase/view/lib/resources/wikibase-data-values/lib/../../../wikibase-data-values/lib/util/util.inherit.js"

The quick and dirty fix is to checkout the new submodules of wikibase that vagrant currently doesn't do

Change 394689 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/vagrant@master] Stop using Wikidata build resources

https://gerrit.wikimedia.org/r/394689

With the patch above, after enabling wikidata role in vagrant, I get this:

==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: enwiki:  Creating wb_property_info table ...done.
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: enwiki:  Populating wb_property_info
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: [7872dc7cdf2bf52f2b118482] [no req]   Wikimedia\Rdbms\DBQueryError from line 1185 of /vagrant/mediawiki/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Query: SELECT  page_title,page_id  FROM `page` LEFT JOIN `wb_property_info` ON ((CONCAT('P',pi_property_id) = page_title))   WHERE (page_id > 1) AND (page_namespace = ) AND (pi_property_id IS NULL)  ORDER BY page_id ASC LIMIT 100   FOR UPDATE
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Function: Wikibase\PropertyInfoTableBuilder::rebuildPropertyInfo
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Error: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') AND (pi_property_id IS NULL)  ORDER BY page_id ASC LIMIT 100   FOR UPDATE' at line 1 (127.0.0.1)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: 
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Backtrace:
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #0 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(997): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #1 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(1380): Wikimedia\Rdbms\Database->query(string, string)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #2 /vagrant/mediawiki/extensions/Wikibase/repo/includes/Store/Sql/PropertyInfoTableBuilder.php(178): Wikimedia\Rdbms\Database->select(array, array, array, string, array, array)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #3 /vagrant/mediawiki/extensions/Wikibase/repo/includes/Store/Sql/DatabaseSchemaUpdater.php(210): Wikibase\PropertyInfoTableBuilder->rebuildPropertyInfo()
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #4 [internal function]: Wikibase\Repo\Store\Sql\DatabaseSchemaUpdater::rebuildPropertyInfo(MysqlUpdater)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #5 /vagrant/mediawiki/includes/installer/DatabaseUpdater.php(482): call_user_func_array(array, array)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #6 /vagrant/mediawiki/includes/installer/DatabaseUpdater.php(450): DatabaseUpdater->runUpdates(array, boolean)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #7 /vagrant/mediawiki/maintenance/update.php(202): DatabaseUpdater->doUpdates(array)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #8 /vagrant/mediawiki/maintenance/doMaintenance.php(94): UpdateMediaWiki->execute()
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #9 /vagrant/mediawiki/maintenance/update.php(247): require_once(string)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #10 /var/www/w/MWScript.php(95): require_once(string)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #11 {main}

This repeats for other wikis too. Finally, I get this:

==> default: Error: /Stage[main]/Mediawiki/Exec[update_all_databases]: Failed to call refresh: /usr/local/bin/foreachwiki update.php --quick --doshared returned 1 instead of one of [0]
==> default: Error: /Stage[main]/Mediawiki/Exec[update_all_databases]: /usr/local/bin/foreachwiki update.php --quick --doshared returned 1 instead of one of [0]

Change 400999 had a related patch set uploaded (by BryanDavis; owner: Addshore):
[mediawiki/vagrant@stretch-migration] wikidata: Install wikibase using individual extensions

https://gerrit.wikimedia.org/r/400999

With the patch above, after enabling wikidata role in vagrant, I get this:

(...snip...)

AND (page_namespace = )

This is caused by the entityNamespaces settings not being picked up properly. I think I have fixed this in the version of the patch that I cherry-picked to the stretch-migration branch: https://gerrit.wikimedia.org/r/400999

That patch should apply cleanly on the master branch as well for anyone who wants to test it without converting their local MediaWiki=Vagrant deployment to a Stretch VM yet.

With this patch, stretch-migration seems to be able to install Wikidata and I can add items there without any error messages.

Change 400999 merged by jenkins-bot:
[mediawiki/vagrant@stretch-migration] wikidata: Install wikibase using individual extensions

https://gerrit.wikimedia.org/r/400999

bd808 closed this task as Resolved.Jan 1 2018, 4:36 AM
bd808 claimed this task.

Fixed on stretch-migration branch. This will become master soon™.

Change 394689 abandoned by Addshore:
WIP Stop using Wikidata build resources

https://gerrit.wikimedia.org/r/394689

Change 401667 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@stretch-migration] wikidata: remove role::articleplaceholder conflict

https://gerrit.wikimedia.org/r/401667

Change 401667 merged by jenkins-bot:
[mediawiki/vagrant@stretch-migration] wikidata: remove role::articleplaceholder conflict

https://gerrit.wikimedia.org/r/401667