Page MenuHomePhabricator
Create Task
Maniphest T158751

Make populateSitesTable.php more robust
Closed, DeclinedPublic
Actions

Description

As part of T158730, it'd be good to make populateSitesTable.php more robust

https://wikitech.wikimedia.org/w/index.php?title=Add_a_wiki&oldid=1560435#Wikidata doesn't inspire much confidence!

That script is known to be troublesome, you might want to ask Marius (hoo) or Katie (aude) run it for you or just create a ticket (that may be done anytime after the wiki was created).

Beware: The script sometimes fails with a duplicate key conflict. In that case, go to the wiki's master database and empty the sites and site_identifiers tables, then run the script again. It's probably also wise to backup these tables from Wikidata and at least one Wikipedia before running the script across the whole fleet. Breaking the sites, site_identifiers tables will break page rendering of many wikis!

Related Objects
Search...

Event Timeline

Reedy created this task.Feb 22 2017, 10:32 AM
Reedy added subscribers: aude, hoo.Feb 22 2017, 10:47 AM
Lydia_Pintscher moved this task from incoming to ready to go on the Wikidata board.May 5 2017, 3:02 PM
Lydia_Pintscher triaged this task as Medium priority.Jun 11 2017, 4:56 PM
hoo added a subtask: T122520: Error running `extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https` on jbowiki.Apr 20 2018, 3:15 PM
hoo mentioned this in T171013: Clarify populateSitesTable.php instructions in Add a Wiki.Apr 20 2018, 3:23 PM
Addshore added a project: [DEPRECATED] wdwb-tech.Jan 22 2019, 3:55 PM
Reedy mentioned this in T235487: Enable Wikidata, cxserver, and Parsoid on new wikis as early as possible.Oct 15 2019, 12:41 PM
Krinkle closed subtask T122520: Error running `extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https` on jbowiki as Declined.May 17 2020, 3:45 PM
Addshore moved this task from Inbox to Legacy Inbox on the [DEPRECATED] wdwb-tech board.Mar 17 2021, 5:19 PM
Addshore moved this task from Legacy Inbox to Research on the [DEPRECATED] wdwb-tech board.
Addshore moved this task from Research to Investigate & Discuss on the [DEPRECATED] wdwb-tech board.Jul 16 2021, 8:38 AM
Addshore added subscribers: Ladsgroup, Addshore.Jul 16 2021, 8:55 AM

Looks like this text is still on the page:

In order to be able to link the new wiki from Wikidata, and to allow interwiki links from Wikidata to the new wiki, run extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https on at least all Wikidata clients (including wikidatawiki itself and testwikidata).

foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https

That script is known to be troublesome, you might want to ask Marius (hoo) or Amir Sarabadani (Amir1) run it for you or just create a ticket (that may be done anytime after the wiki was created).

Beware: The script sometimes fails with a duplicate key conflict. In that case, go to the wiki's master database and empty the sites and site_identifiers tables, then run the script again. It's probably also wise to backup these tables from Wikidata and at least one Wikipedia before running the script across the whole fleet. Breaking the sites, site_identifiers tables will break page rendering of many wikis!

@hoo @Ladsgroup would you be able to specify some changes that we might be able to make & clarify the problem exactly?

Ladsgroup added a comment.Jul 16 2021, 11:32 AM

I've been running it for years now, I have seen it break I think last month too while it's rather rare. The problem is that you need to run this for +1k wikis and well, some would not work for any number of reasons and it would break the whole wiki. The proper fix for this is to have a centralized sitestore (Something along the lines of T113034: RFC: Overhaul Interwiki map, unify with Sites and WikiMap) so only one run would be needed after each wiki creation.

hoo added a comment.Jul 16 2021, 1:00 PM

I'm not aware of a problem of any concrete issues we could tackle, other than T122520 which seems fine now. Mid-term we should really try to get rid of this, though, as Amir pointed out.

Addshore closed this task as Declined.Jul 20 2021, 9:03 AM

In that case I'm going to decline this in favour of T113034

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits