Page MenuHomePhabricator

Installation: Script that populates the sites table (interwiki) fails because of violated not null constraints.
Closed, ResolvedPublic5 Estimated Story Points

Description

Error descripton
maintenance/populateInterwiki.php crashes on Postgres installations by null constraints on iw_api and iw_wikiid

  • Mediawiki Master, 1.36.1
  • with a PostgreSQL (12) database

the step 'Run the script that populates the sites table:'

php maintenance/populateInterwiki.php

fails with a not-null constraint violation error for the fields

  • iw_api
  • iw_wikiid

Expected behavior
The script does not fail

Possible fix
in PopulateInterwiki::doPopulate() extend the database insert by empty strings for iw_api and iw_wikiid

if ( !$row ) {
  $dbw->insert(
    'interwiki',
    [
      'iw_prefix' => $prefix,
      'iw_url' => $d['url'],
      'iw_local' => 1,
      'iw_api' => '', // FIX??
      'iw_wikiid' => '' // FIX??
    ],
    __METHOD__,
    [ 'IGNORE' ]
  );
}

Event Timeline

Addshore set the point value for this task to 5.Aug 4 2021, 10:31 AM

I also got warnings from MySQL about those fields when manually inserting rows for Not Wikilambda: see Not Wikilambda:Setup (the part around the two MySQL code snippets).

Hello @OkayIguess

Thank you for reporting the issue.

I take it you are running wikibase cloned from gerrit. Could you share what tag/branch/commit you are at when running this command?

I've tried setting up something similar locally.

Wikibase REL1_36 at e8af88803f80d2a1f64692fc64235029c191855b (possibly newer than what you have)
Mediawiki at b6cf6b337262ed84b76e96ed0f7ccbb702a46e9c (1.36.1 tag)
Postgres: 12

After running installation + update

php extensions/Wikibase/lib/maintenance/populateSitesTable.php

I have the following count in my sites table.

default=# select count(*) from mediawiki.sites;
 count 
-------
   977
(1 row)

default=# \q
root@db-postgres:/# exit

Could it be that you are running the mediawiki script to populate interwiki links? (The suggested fix seems to mention that script rather)

root@mediawiki:/var/www/mediawiki# php maintenance/populateInterwiki.php 
Xdebug: [Step Debug] Could not connect to debugging client. Tried: 172.17.0.1:9000 (fallback through xdebug.client_host/xdebug.client_port) :-(
Wikimedia\Rdbms\DBQueryError from line 1719 of /var/www/mediawiki/includes/libs/rdbms/database/Database.php: Error 23502: ERROR:  null value in column "iw_api" violates not-null constraint
DETAIL:  Failing row contains (advisory, https://advisory.wikimedia.org/wiki/$1, null, null, 1, 0).

Function: PopulateInterwiki::doPopulate
Query: INSERT INTO "interwiki" (iw_prefix,iw_url,iw_local) VALUES ('advisory','https://advisory.wikimedia.org/wiki/$1',1) ON CONFLICT DO NOTHING

#0 /var/www/mediawiki/includes/libs/rdbms/database/Database.php(1703): Wikimedia\Rdbms\Database->getQueryException('ERROR:  null va...', '23502', 'INSERT INTO "in...', 'PopulateInterwi...')
#1 /var/www/mediawiki/includes/libs/rdbms/database/Database.php(1678): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('ERROR:  null va...', '23502', 'INSERT INTO "in...', 'PopulateInterwi...')
#2 /var/www/mediawiki/includes/libs/rdbms/database/Database.php(1244): Wikimedia\Rdbms\Database->reportQueryError('ERROR:  null va...', '23502', 'INSERT INTO "in...', 'PopulateInterwi...', false)
#3 /var/www/mediawiki/includes/libs/rdbms/database/Database.php(2383): Wikimedia\Rdbms\Database->query('INSERT INTO "in...', 'PopulateInterwi...', 128)
#4 /var/www/mediawiki/includes/libs/rdbms/database/DatabasePostgres.php(562): Wikimedia\Rdbms\Database->doInsertNonConflicting('interwiki', Array, 'PopulateInterwi...')
#5 /var/www/mediawiki/includes/libs/rdbms/database/Database.php(2343): Wikimedia\Rdbms\DatabasePostgres->doInsertNonConflicting('interwiki', Array, 'PopulateInterwi...')
#6 /var/www/mediawiki/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->insert('interwiki', Array, 'PopulateInterwi...', Array)
#7 /var/www/mediawiki/includes/libs/rdbms/database/DBConnRef.php(369): Wikimedia\Rdbms\DBConnRef->__call('insert', Array)
#8 /var/www/mediawiki/maintenance/populateInterwiki.php(144): Wikimedia\Rdbms\DBConnRef->insert('interwiki', Array, 'PopulateInterwi...', Array)
#9 /var/www/mediawiki/maintenance/populateInterwiki.php(69): PopulateInterwiki->doPopulate(Array, false)
#10 /var/www/mediawiki/maintenance/doMaintenance.php(112): PopulateInterwiki->execute()
#11 /var/www/mediawiki/maintenance/populateInterwiki.php(159): require_once('/var/www/mediaw...')
#12 {main}

This seems to be not a Wikibase error (Wikibase was disabled in the above installation). I think the suggested fix could do the trick however since there is no testing for postgres in CI we could probably only assert that adding the defaults does not crash MariaDB/Mysql.

https://www.mediawiki.org/wiki/Manual:PostgreSQL

This seems to still be a bug on master of mediawiki, but can't find any ticket. Should we use this one?

(MW Postgres documentation mentions some maintenance scripts not working but no specifics)

toan updated the task description. (Show Details)
toan updated the task description. (Show Details)
toan subscribed.

Change 812078 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/core@master] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812078

Change 812078 merged by jenkins-bot:

[mediawiki/core@master] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812078

Change 812357 had a related patch set uploaded (by Zabe; author: Umherirrender):

[mediawiki/core@REL1_38] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812357

Change 812358 had a related patch set uploaded (by Zabe; author: Umherirrender):

[mediawiki/core@REL1_37] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812358

Change 812359 had a related patch set uploaded (by Zabe; author: Umherirrender):

[mediawiki/core@REL1_35] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812359

Change 812357 merged by jenkins-bot:

[mediawiki/core@REL1_38] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812357

Change 812359 merged by jenkins-bot:

[mediawiki/core@REL1_35] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812359

Change 812358 merged by jenkins-bot:

[mediawiki/core@REL1_37] populateInterwiki: Include not null columns iw_api/iw_wikiid

https://gerrit.wikimedia.org/r/812358