Page MenuHomePhabricator

Create and populate babel database table on Wikimedia wikis
Closed, ResolvedPublic

Description

  • Schema of babel database table: https://gerrit.wikimedia.org/r/#/c/309937/5/babel.sql.
  • This table will be created on all wikis.
  • All contents of the table are public and can be safely replicated to labs
  • The usage of this table is guarded by a configuration setting that defaults to false.
  • Contact person: @Legoktm

Next steps:

  • Final review by DBA before deployment
  • Create table on all wikis
  • Run script to populate database
  • Set $wgBabelUseDatabase = true

Event Timeline

Legoktm created this task.

If the table is fully public, and contain no user information or other private information, then I am not a blocker. The table looks fine.

$wgBabelUseDatabase should be enabled first, then run the script, otherwise you will lose some records (assuming a hook on the parsing function does that). I would be very interested being on the loop on how those records get filled in, specially by the initial script.

Thanks for the quick review!

After a page is edited that uses this feature, a refreshLinks job is submitted to the job queue. That job will run the normal link updates, and will also now update the babel table if necessary.

I was just asking @aaron about recommendations on how to do the initial population. The main consideration (IMO) is that this feature is on a limited set of pages (user pages that are not subpages, e.g. "User:Legoktm"). So I think I will probably end up writing a script that queries those pages, and then runs the refreshlinks code on them, which will also update the babel table. We have existing scripts like refreshLinks.php, but I don't think those can be filtered to just the subset of pages we want to refresh.

Mentioned in SAL (#wikimedia-operations) [2016-09-15T23:38:20Z] <legoktm> legoktm@terbium:~$ foreachwiki extensions/WikimediaMaintenance/createExtensionTables.php babel # T145366

Are there any blockers for this? Just the script that queries those pages, and then runs the refreshlinks code on them? Can I help with that?

Please ping the DBAs before running the script so we can be aware of this (also if this is a long running script please schedule it on https://wikitech.wikimedia.org/wiki/Deployments)

Mentioned in SAL (#wikimedia-operations) [2017-07-11T00:07:25Z] <legoktm> running mwscript refreshLinks.php --wiki=metawiki --namespace=2 on terbium (T145366)

I'm wondering what script actually needs running. I don't see a specific maintenance script in the extensions folder...

The comments above suggest it is refreshLinks.. But that's only been run on metawiki.

@Legoktm Does it need running everywhere else? Or are just expecting usual page churn/paging to have fixed this?

If so, do we just need to do $wgBabelUseDatabase = true in CommonSettings to finish this off?

I'm wondering what script actually needs running. I don't see a specific maintenance script in the extensions folder...

The comments above suggest it is refreshLinks.. But that's only been run on metawiki.

@Legoktm Does it need running everywhere else? Or are just expecting usual page churn/paging to have fixed this?

Meta was the only important one since we plan on having all the other wikis fall back to the central one. It could be run against the other wikis, but that's not necessary.

If so, do we just need to do $wgBabelUseDatabase = true in CommonSettings to finish this off?

Yep.

Guess $wgBabelCentralDb and $wgBabelCentralApi need setting too to meta? Guarded by CentralAuth being enabled.

But set $wgBabelUseDatabase = true; unconditionally?

Change 368429 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Make babel use Database and SUL wikis use metawiki

https://gerrit.wikimedia.org/r/368429

Guess $wgBabelCentralDb and $wgBabelCentralApi need setting too to meta? Guarded by CentralAuth being enabled.

Yeah, see T95877#3480316

But set $wgBabelUseDatabase = true; unconditionally?

Yep :)

Change 368429 merged by jenkins-bot:
[operations/mediawiki-config@master] Make babel use Database and SUL wikis use metawiki

https://gerrit.wikimedia.org/r/368429

Mentioned in SAL (#wikimedia-operations) [2017-08-01T23:47:27Z] <reedy@tin> Synchronized wmf-config/CommonSettings.php: Make Babel use databasey stuff T145366 (duration: 00m 46s)

Reedy claimed this task.
Reedy updated the task description. (Show Details)
Reedy removed a project: Patch-For-Review.