Page MenuHomePhabricator
Create Task
Maniphest T181769

Update role::wikidata to work with new (no)build process
Closed, ResolvedPublic
Actions

Description

Follow up to T173818: [Epic] Kill the Wikidata build step:

The wikidata role clones https://gerrit.wikimedia.org/r/wikidata/build-resources to provision wikidata in the VM. This should be replaced with cloning all of the individual extensions and any settings changes needed to go with that.

puppet/modules/role/manifests/wikidata.pp
mediawiki::extension { 'WikidataBuildResources':
    remote       => 'https://gerrit.wikimedia.org/r/wikidata/build-resources',
    entrypoint   => 'Wikidata.php',
    composer     => true,
    needs_update => true,
    settings     => template('role/wikidata/shared.php.erb'),
}

mediawiki::settings { 'WikiData-Init':
    priority => $::LOAD_EARLY,
    values   => template('role/wikidata/init.php.erb'),
}

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
wikidata: remove role::articleplaceholder conflictmediawiki/vagrantstretch-migration+1 -5
WIP Stop using Wikidata build resourcesmediawiki/vagrantmaster+46 -10
wikidata: Install wikibase using individual extensionsmediawiki/vagrantstretch-migration+114 -51
Customize query in gerrit

Related Objects
Search...

Event Timeline

bd808 created this task.Dec 1 2017, 12:11 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 1 2017, 12:11 AM
bd808 added a parent task: T173818: [Epic] Kill the Wikidata build step.Dec 1 2017, 12:12 AM
Smalyshev subscribed.Dec 1 2017, 9:29 PM
Addshore added a comment.Dec 1 2017, 9:32 PM

Just for reference (until this is fixed)

If you get errors like:

[Exception MWException] (/vagrant/mediawiki/includes/resourceloader/ResourceLoaderFileModule.php:810) ResourceLoaderFileModule::readScriptFiles: script file not found: "/vagrant/mediawiki/extensions/WikidataBuildResources/extensions/Wikibase/view/lib/resources/wikibase-data-values/lib/../../../wikibase-data-values/lib/util/util.inherit.js"

The quick and dirty fix is to checkout the new submodules of wikibase that vagrant currently doesn't do

gerritbot subscribed.Dec 1 2017, 9:48 PM

Change 394689 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/vagrant@master] Stop using Wikidata build resources

https://gerrit.wikimedia.org/r/394689

gerritbot added a project: Patch-For-Review.Dec 1 2017, 9:48 PM
Addshore mentioned this in T181838: Mark extension-Wikidata & wikidata-build-resources on Gerrit as ARCHIVED.Dec 6 2017, 11:43 PM
bd808 mentioned this in T183307: wikidata role produces error when trying to create item.Dec 20 2017, 5:46 AM
Smalyshev added a comment.Dec 21 2017, 10:08 PM

With the patch above, after enabling wikidata role in vagrant, I get this:

==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: enwiki:  Creating wb_property_info table ...done.
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: enwiki:  Populating wb_property_info
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: [7872dc7cdf2bf52f2b118482] [no req]   Wikimedia\Rdbms\DBQueryError from line 1185 of /vagrant/mediawiki/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Query: SELECT  page_title,page_id  FROM `page` LEFT JOIN `wb_property_info` ON ((CONCAT('P',pi_property_id) = page_title))   WHERE (page_id > 1) AND (page_namespace = ) AND (pi_property_id IS NULL)  ORDER BY page_id ASC LIMIT 100   FOR UPDATE
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Function: Wikibase\PropertyInfoTableBuilder::rebuildPropertyInfo
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Error: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') AND (pi_property_id IS NULL)  ORDER BY page_id ASC LIMIT 100   FOR UPDATE' at line 1 (127.0.0.1)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: 
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: Backtrace:
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #0 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(997): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #1 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(1380): Wikimedia\Rdbms\Database->query(string, string)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #2 /vagrant/mediawiki/extensions/Wikibase/repo/includes/Store/Sql/PropertyInfoTableBuilder.php(178): Wikimedia\Rdbms\Database->select(array, array, array, string, array, array)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #3 /vagrant/mediawiki/extensions/Wikibase/repo/includes/Store/Sql/DatabaseSchemaUpdater.php(210): Wikibase\PropertyInfoTableBuilder->rebuildPropertyInfo()
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #4 [internal function]: Wikibase\Repo\Store\Sql\DatabaseSchemaUpdater::rebuildPropertyInfo(MysqlUpdater)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #5 /vagrant/mediawiki/includes/installer/DatabaseUpdater.php(482): call_user_func_array(array, array)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #6 /vagrant/mediawiki/includes/installer/DatabaseUpdater.php(450): DatabaseUpdater->runUpdates(array, boolean)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #7 /vagrant/mediawiki/maintenance/update.php(202): DatabaseUpdater->doUpdates(array)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #8 /vagrant/mediawiki/maintenance/doMaintenance.php(94): UpdateMediaWiki->execute()
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #9 /vagrant/mediawiki/maintenance/update.php(247): require_once(string)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #10 /var/www/w/MWScript.php(95): require_once(string)
==> default: Notice: /Stage[main]/Mediawiki/Exec[update_all_databases]/returns: #11 {main}

This repeats for other wikis too. Finally, I get this:

==> default: Error: /Stage[main]/Mediawiki/Exec[update_all_databases]: Failed to call refresh: /usr/local/bin/foreachwiki update.php --quick --doshared returned 1 instead of one of [0]
==> default: Error: /Stage[main]/Mediawiki/Exec[update_all_databases]: /usr/local/bin/foreachwiki update.php --quick --doshared returned 1 instead of one of [0]
gerritbot added a comment.Dec 30 2017, 4:15 AM

Change 400999 had a related patch set uploaded (by BryanDavis; owner: Addshore):
[mediawiki/vagrant@stretch-migration] wikidata: Install wikibase using individual extensions

https://gerrit.wikimedia.org/r/400999

bd808 added a comment.Dec 30 2017, 4:39 AM
In T181769#3855613, @Smalyshev wrote:

With the patch above, after enabling wikidata role in vagrant, I get this:

(...snip...)

AND (page_namespace = )

This is caused by the entityNamespaces settings not being picked up properly. I think I have fixed this in the version of the patch that I cherry-picked to the stretch-migration branch: https://gerrit.wikimedia.org/r/400999

That patch should apply cleanly on the master branch as well for anyone who wants to test it without converting their local MediaWiki=Vagrant deployment to a Stretch VM yet.

Smalyshev added a comment.Dec 30 2017, 5:44 AM

With this patch, stretch-migration seems to be able to install Wikidata and I can add items there without any error messages.

gerritbot added a comment.Jan 1 2018, 4:33 AM

Change 400999 merged by jenkins-bot:
[mediawiki/vagrant@stretch-migration] wikidata: Install wikibase using individual extensions

https://gerrit.wikimedia.org/r/400999

bd808 closed this task as Resolved.Jan 1 2018, 4:36 AM
bd808 claimed this task.

Fixed on stretch-migration branch. This will become master soon™.

gerritbot added a comment.Jan 2 2018, 5:18 PM

Change 394689 abandoned by Addshore:
WIP Stop using Wikidata build resources

https://gerrit.wikimedia.org/r/394689

gerritbot added a comment.Jan 3 2018, 4:10 AM

Change 401667 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@stretch-migration] wikidata: remove role::articleplaceholder conflict

https://gerrit.wikimedia.org/r/401667

gerritbot added a comment.Jan 3 2018, 4:12 AM

Change 401667 merged by jenkins-bot:
[mediawiki/vagrant@stretch-migration] wikidata: remove role::articleplaceholder conflict

https://gerrit.wikimedia.org/r/401667

bd808 added a parent task: T181353: [EPIC] Migrate MediaWiki-Vagrant base image to Debian Stretch.Jan 9 2018, 4:45 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits