Page MenuHomePhabricator

meta_p.wiki table corrupt (contains many NULL entries for 'url' field)
Closed, ResolvedPublic

Description

Pages like https://tools.wmflabs.org/guc/?user=KrinkleBot are missing headings and have broken links because .url is NULL for many rows (292 of 862).

This is a regression since last month. Possibly triggered by T103663.

Source code: https://github.com/wikimedia/labs-tools-guc/blob/cfab32ef03e4ba906dd02432a75a2f04cde8a55a/lb/wikicontribs.php#L255-L273

Event Timeline

Krinkle created this task.Jul 24 2015, 9:43 PM
Krinkle updated the task description. (Show Details)
Krinkle raised the priority of this task from to Unbreak Now!.
Krinkle added a subscriber: Krinkle.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 24 2015, 9:43 PM
Krinkle added a comment.EditedJul 24 2015, 9:48 PM

The .name field is also NULL for the same rows.

scfc added a subscriber: scfc.Jul 24 2015, 11:11 PM

(Not sure if it is already possible or worthy of a RFE task, but if there was an "API" to InitialiseSettings.php independent of actual MediaWiki start-up, i. e. if something like:

require('mediawiki/includes/this.php');
require('mediawiki/includes/that.php');
require('path/to/InitialiseSettings.php');
foreach($argv as $wiki) {
    print(json_encode($wiki, getConfigSetting($wiki, 'wgSomething')));
}

would (always) work, that would be tremendously helpful in this and other cases.)

Krenair claimed this task.Jul 24 2015, 11:11 PM

Change 226939 had a related patch set uploaded (by Alex Monk):
Also look at wikipedia.dblist

https://gerrit.wikimedia.org/r/226939

"Unbreak Now!"

MariaDB [(none)]> SELECT dbname, lang, name, family, url FROM meta_p.wiki WHERE family = 'wikipedia' ORDER BY dbname;
+------------------+------+------+-----------+------+
| dbname           | lang | name | family    | url  |
+------------------+------+------+-----------+------+
| aawiki           | en   | NULL | wikipedia | NULL |
| abwiki           | en   | NULL | wikipedia | NULL |
| acewiki          | en   | NULL | wikipedia | NULL |
| afwiki           | en   | NULL | wikipedia | NULL |
| akwiki           | en   | NULL | wikipedia | NULL |
| alswiki          | en   | NULL | wikipedia | NULL |
| amwiki           | en   | NULL | wikipedia | NULL |
| angwiki          | en   | NULL | wikipedia | NULL |
| anwiki           | en   | NULL | wikipedia | NULL |
| arcwiki          | en   | NULL | wikipedia | NULL |
| arwiki           | en   | NULL | wikipedia | NULL |
| arzwiki          | en   | NULL | wikipedia | NULL |
| astwiki          | en   | NULL | wikipedia | NULL |
| aswiki           | en   | NULL | wikipedia | NULL |
| avwiki           | en   | NULL | wikipedia | NULL |
| aywiki           | en   | NULL | wikipedia | NULL |
| azbwiki          | en   | NULL | wikipedia | NULL |
| azwiki           | en   | NULL | wikipedia | NULL |
| barwiki          | en   | NULL | wikipedia | NULL |
| bat_smgwiki      | en   | NULL | wikipedia | NULL |
| bawiki           | en   | NULL | wikipedia | NULL |
| bclwiki          | en   | NULL | wikipedia | NULL |
| bewiki           | en   | NULL | wikipedia | NULL |
| be_x_oldwiki     | en   | NULL | wikipedia | NULL |
| bgwiki           | en   | NULL | wikipedia | NULL |
| bhwiki           | en   | NULL | wikipedia | NULL |
| biwiki           | en   | NULL | wikipedia | NULL |
| bjnwiki          | en   | NULL | wikipedia | NULL |
| bmwiki           | en   | NULL | wikipedia | NULL |
| bnwiki           | en   | NULL | wikipedia | NULL |
| bowiki           | en   | NULL | wikipedia | NULL |
| bpywiki          | en   | NULL | wikipedia | NULL |
| brwiki           | en   | NULL | wikipedia | NULL |
| bswiki           | en   | NULL | wikipedia | NULL |
| bugwiki          | en   | NULL | wikipedia | NULL |
| bxrwiki          | en   | NULL | wikipedia | NULL |
| cawiki           | en   | NULL | wikipedia | NULL |
| cbk_zamwiki      | en   | NULL | wikipedia | NULL |
| cdowiki          | en   | NULL | wikipedia | NULL |
| cebwiki          | en   | NULL | wikipedia | NULL |
| cewiki           | en   | NULL | wikipedia | NULL |
| chowiki          | en   | NULL | wikipedia | NULL |
| chrwiki          | en   | NULL | wikipedia | NULL |
| chwiki           | en   | NULL | wikipedia | NULL |
| chywiki          | en   | NULL | wikipedia | NULL |
| ckbwiki          | en   | NULL | wikipedia | NULL |
| cowiki           | en   | NULL | wikipedia | NULL |
| crhwiki          | en   | NULL | wikipedia | NULL |
| crwiki           | en   | NULL | wikipedia | NULL |
| csbwiki          | en   | NULL | wikipedia | NULL |
| cswiki           | en   | NULL | wikipedia | NULL |
| cuwiki           | en   | NULL | wikipedia | NULL |
| cvwiki           | en   | NULL | wikipedia | NULL |
| cywiki           | en   | NULL | wikipedia | NULL |
| dawiki           | en   | NULL | wikipedia | NULL |
| dewiki           | en   | NULL | wikipedia | NULL |
| diqwiki          | en   | NULL | wikipedia | NULL |
| dsbwiki          | en   | NULL | wikipedia | NULL |
| dvwiki           | en   | NULL | wikipedia | NULL |
| dzwiki           | en   | NULL | wikipedia | NULL |
| eewiki           | en   | NULL | wikipedia | NULL |
| elwiki           | en   | NULL | wikipedia | NULL |
| emlwiki          | en   | NULL | wikipedia | NULL |
| enwiki           | en   | NULL | wikipedia | NULL |
| eowiki           | en   | NULL | wikipedia | NULL |
| eswiki           | en   | NULL | wikipedia | NULL |
| etwiki           | en   | NULL | wikipedia | NULL |
| euwiki           | en   | NULL | wikipedia | NULL |
| extwiki          | en   | NULL | wikipedia | NULL |
| fawiki           | en   | NULL | wikipedia | NULL |
| ffwiki           | en   | NULL | wikipedia | NULL |
| fiu_vrowiki      | en   | NULL | wikipedia | NULL |
| fiwiki           | en   | NULL | wikipedia | NULL |
| fjwiki           | en   | NULL | wikipedia | NULL |
| fowiki           | en   | NULL | wikipedia | NULL |
| frpwiki          | en   | NULL | wikipedia | NULL |
| frrwiki          | en   | NULL | wikipedia | NULL |
| frwiki           | en   | NULL | wikipedia | NULL |
| furwiki          | en   | NULL | wikipedia | NULL |
| fywiki           | en   | NULL | wikipedia | NULL |
| gagwiki          | en   | NULL | wikipedia | NULL |
| ganwiki          | en   | NULL | wikipedia | NULL |
| gawiki           | en   | NULL | wikipedia | NULL |
| gdwiki           | en   | NULL | wikipedia | NULL |
| glkwiki          | en   | NULL | wikipedia | NULL |
| glwiki           | en   | NULL | wikipedia | NULL |
| gnwiki           | en   | NULL | wikipedia | NULL |
| gomwiki          | en   | NULL | wikipedia | NULL |
| gotwiki          | en   | NULL | wikipedia | NULL |
| guwiki           | en   | NULL | wikipedia | NULL |
| gvwiki           | en   | NULL | wikipedia | NULL |
| hakwiki          | en   | NULL | wikipedia | NULL |
| hawiki           | en   | NULL | wikipedia | NULL |
| hawwiki          | en   | NULL | wikipedia | NULL |
| hewiki           | en   | NULL | wikipedia | NULL |
| hifwiki          | en   | NULL | wikipedia | NULL |
| hiwiki           | en   | NULL | wikipedia | NULL |
| howiki           | en   | NULL | wikipedia | NULL |
| hrwiki           | en   | NULL | wikipedia | NULL |
| hsbwiki          | en   | NULL | wikipedia | NULL |
| htwiki           | en   | NULL | wikipedia | NULL |
| huwiki           | en   | NULL | wikipedia | NULL |
| hywiki           | en   | NULL | wikipedia | NULL |
| hzwiki           | en   | NULL | wikipedia | NULL |
| iawiki           | en   | NULL | wikipedia | NULL |
| idwiki           | en   | NULL | wikipedia | NULL |
| iewiki           | en   | NULL | wikipedia | NULL |
| igwiki           | en   | NULL | wikipedia | NULL |
| iiwiki           | en   | NULL | wikipedia | NULL |
| ikwiki           | en   | NULL | wikipedia | NULL |
| ilowiki          | en   | NULL | wikipedia | NULL |
| iowiki           | en   | NULL | wikipedia | NULL |
| iswiki           | en   | NULL | wikipedia | NULL |
| itwiki           | en   | NULL | wikipedia | NULL |
| iuwiki           | en   | NULL | wikipedia | NULL |
| jawiki           | en   | NULL | wikipedia | NULL |
| jbowiki          | en   | NULL | wikipedia | NULL |
| jvwiki           | en   | NULL | wikipedia | NULL |
| kaawiki          | en   | NULL | wikipedia | NULL |
| kabwiki          | en   | NULL | wikipedia | NULL |
| kawiki           | en   | NULL | wikipedia | NULL |
| kbdwiki          | en   | NULL | wikipedia | NULL |
| kgwiki           | en   | NULL | wikipedia | NULL |
| kiwiki           | en   | NULL | wikipedia | NULL |
| kjwiki           | en   | NULL | wikipedia | NULL |
| kkwiki           | en   | NULL | wikipedia | NULL |
| klwiki           | en   | NULL | wikipedia | NULL |
| kmwiki           | en   | NULL | wikipedia | NULL |
| knwiki           | en   | NULL | wikipedia | NULL |
| koiwiki          | en   | NULL | wikipedia | NULL |
| kowiki           | en   | NULL | wikipedia | NULL |
| krcwiki          | en   | NULL | wikipedia | NULL |
| krwiki           | en   | NULL | wikipedia | NULL |
| kshwiki          | en   | NULL | wikipedia | NULL |
| kswiki           | en   | NULL | wikipedia | NULL |
| kuwiki           | en   | NULL | wikipedia | NULL |
| kvwiki           | en   | NULL | wikipedia | NULL |
| kwwiki           | en   | NULL | wikipedia | NULL |
| kywiki           | en   | NULL | wikipedia | NULL |
| ladwiki          | en   | NULL | wikipedia | NULL |
| lawiki           | en   | NULL | wikipedia | NULL |
| lbewiki          | en   | NULL | wikipedia | NULL |
| lbwiki           | en   | NULL | wikipedia | NULL |
| lezwiki          | en   | NULL | wikipedia | NULL |
| lgwiki           | en   | NULL | wikipedia | NULL |
| lijwiki          | en   | NULL | wikipedia | NULL |
| liwiki           | en   | NULL | wikipedia | NULL |
| lmowiki          | en   | NULL | wikipedia | NULL |
| lnwiki           | en   | NULL | wikipedia | NULL |
| lowiki           | en   | NULL | wikipedia | NULL |
| lrcwiki          | en   | NULL | wikipedia | NULL |
| ltgwiki          | en   | NULL | wikipedia | NULL |
| ltwiki           | en   | NULL | wikipedia | NULL |
| lvwiki           | en   | NULL | wikipedia | NULL |
| maiwiki          | en   | NULL | wikipedia | NULL |
| map_bmswiki      | en   | NULL | wikipedia | NULL |
| mdfwiki          | en   | NULL | wikipedia | NULL |
| mgwiki           | en   | NULL | wikipedia | NULL |
| mhrwiki          | en   | NULL | wikipedia | NULL |
| mhwiki           | en   | NULL | wikipedia | NULL |
| minwiki          | en   | NULL | wikipedia | NULL |
| miwiki           | en   | NULL | wikipedia | NULL |
| mkwiki           | en   | NULL | wikipedia | NULL |
| mlwiki           | en   | NULL | wikipedia | NULL |
| mnwiki           | en   | NULL | wikipedia | NULL |
| mowiki           | en   | NULL | wikipedia | NULL |
| mrjwiki          | en   | NULL | wikipedia | NULL |
| mrwiki           | en   | NULL | wikipedia | NULL |
| mswiki           | en   | NULL | wikipedia | NULL |
| mtwiki           | en   | NULL | wikipedia | NULL |
| muswiki          | en   | NULL | wikipedia | NULL |
| mwlwiki          | en   | NULL | wikipedia | NULL |
| myvwiki          | en   | NULL | wikipedia | NULL |
| mywiki           | en   | NULL | wikipedia | NULL |
| mznwiki          | en   | NULL | wikipedia | NULL |
| nahwiki          | en   | NULL | wikipedia | NULL |
| napwiki          | en   | NULL | wikipedia | NULL |
| nawiki           | en   | NULL | wikipedia | NULL |
| ndswiki          | en   | NULL | wikipedia | NULL |
| nds_nlwiki       | en   | NULL | wikipedia | NULL |
| newiki           | en   | NULL | wikipedia | NULL |
| newwiki          | en   | NULL | wikipedia | NULL |
| ngwiki           | en   | NULL | wikipedia | NULL |
| nlwiki           | en   | NULL | wikipedia | NULL |
| nnwiki           | en   | NULL | wikipedia | NULL |
| novwiki          | en   | NULL | wikipedia | NULL |
| nowiki           | en   | NULL | wikipedia | NULL |
| nrmwiki          | en   | NULL | wikipedia | NULL |
| nsowiki          | en   | NULL | wikipedia | NULL |
| nvwiki           | en   | NULL | wikipedia | NULL |
| nywiki           | en   | NULL | wikipedia | NULL |
| ocwiki           | en   | NULL | wikipedia | NULL |
| omwiki           | en   | NULL | wikipedia | NULL |
| orwiki           | en   | NULL | wikipedia | NULL |
| oswiki           | en   | NULL | wikipedia | NULL |
| pagwiki          | en   | NULL | wikipedia | NULL |
| pamwiki          | en   | NULL | wikipedia | NULL |
| papwiki          | en   | NULL | wikipedia | NULL |
| pawiki           | en   | NULL | wikipedia | NULL |
| pcdwiki          | en   | NULL | wikipedia | NULL |
| pdcwiki          | en   | NULL | wikipedia | NULL |
| pflwiki          | en   | NULL | wikipedia | NULL |
| pihwiki          | en   | NULL | wikipedia | NULL |
| piwiki           | en   | NULL | wikipedia | NULL |
| plwiki           | en   | NULL | wikipedia | NULL |
| pmswiki          | en   | NULL | wikipedia | NULL |
| pnbwiki          | en   | NULL | wikipedia | NULL |
| pntwiki          | en   | NULL | wikipedia | NULL |
| pswiki           | en   | NULL | wikipedia | NULL |
| ptwiki           | en   | NULL | wikipedia | NULL |
| quwiki           | en   | NULL | wikipedia | NULL |
| rmwiki           | en   | NULL | wikipedia | NULL |
| rmywiki          | en   | NULL | wikipedia | NULL |
| rnwiki           | en   | NULL | wikipedia | NULL |
| roa_rupwiki      | en   | NULL | wikipedia | NULL |
| roa_tarawiki     | en   | NULL | wikipedia | NULL |
| rowiki           | en   | NULL | wikipedia | NULL |
| ruewiki          | en   | NULL | wikipedia | NULL |
| ruwiki           | en   | NULL | wikipedia | NULL |
| rwwiki           | en   | NULL | wikipedia | NULL |
| sahwiki          | en   | NULL | wikipedia | NULL |
| sawiki           | en   | NULL | wikipedia | NULL |
| scnwiki          | en   | NULL | wikipedia | NULL |
| scowiki          | en   | NULL | wikipedia | NULL |
| scwiki           | en   | NULL | wikipedia | NULL |
| sdwiki           | en   | NULL | wikipedia | NULL |
| sewiki           | en   | NULL | wikipedia | NULL |
| sgwiki           | en   | NULL | wikipedia | NULL |
| shwiki           | en   | NULL | wikipedia | NULL |
| simplewiki       | en   | NULL | wikipedia | NULL |
| siwiki           | en   | NULL | wikipedia | NULL |
| skwiki           | en   | NULL | wikipedia | NULL |
| slwiki           | en   | NULL | wikipedia | NULL |
| smwiki           | en   | NULL | wikipedia | NULL |
| snwiki           | en   | NULL | wikipedia | NULL |
| sowiki           | en   | NULL | wikipedia | NULL |
| sqwiki           | en   | NULL | wikipedia | NULL |
| srnwiki          | en   | NULL | wikipedia | NULL |
| srwiki           | en   | NULL | wikipedia | NULL |
| sswiki           | en   | NULL | wikipedia | NULL |
| stqwiki          | en   | NULL | wikipedia | NULL |
| stwiki           | en   | NULL | wikipedia | NULL |
| suwiki           | en   | NULL | wikipedia | NULL |
| svwiki           | en   | NULL | wikipedia | NULL |
| swwiki           | en   | NULL | wikipedia | NULL |
| szlwiki          | en   | NULL | wikipedia | NULL |
| tawiki           | en   | NULL | wikipedia | NULL |
| tetwiki          | en   | NULL | wikipedia | NULL |
| tewiki           | en   | NULL | wikipedia | NULL |
| tgwiki           | en   | NULL | wikipedia | NULL |
| thwiki           | en   | NULL | wikipedia | NULL |
| tiwiki           | en   | NULL | wikipedia | NULL |
| tkwiki           | en   | NULL | wikipedia | NULL |
| tlwiki           | en   | NULL | wikipedia | NULL |
| tnwiki           | en   | NULL | wikipedia | NULL |
| towiki           | en   | NULL | wikipedia | NULL |
| tpiwiki          | en   | NULL | wikipedia | NULL |
| trwiki           | en   | NULL | wikipedia | NULL |
| tswiki           | en   | NULL | wikipedia | NULL |
| ttwiki           | en   | NULL | wikipedia | NULL |
| tumwiki          | en   | NULL | wikipedia | NULL |
| twwiki           | en   | NULL | wikipedia | NULL |
| tyvwiki          | en   | NULL | wikipedia | NULL |
| tywiki           | en   | NULL | wikipedia | NULL |
| udmwiki          | en   | NULL | wikipedia | NULL |
| ugwiki           | en   | NULL | wikipedia | NULL |
| ukwiki           | en   | NULL | wikipedia | NULL |
| urwiki           | en   | NULL | wikipedia | NULL |
| uzwiki           | en   | NULL | wikipedia | NULL |
| vecwiki          | en   | NULL | wikipedia | NULL |
| vepwiki          | en   | NULL | wikipedia | NULL |
| vewiki           | en   | NULL | wikipedia | NULL |
| viwiki           | en   | NULL | wikipedia | NULL |
| vlswiki          | en   | NULL | wikipedia | NULL |
| vowiki           | en   | NULL | wikipedia | NULL |
| warwiki          | en   | NULL | wikipedia | NULL |
| wawiki           | en   | NULL | wikipedia | NULL |
| wowiki           | en   | NULL | wikipedia | NULL |
| wuuwiki          | en   | NULL | wikipedia | NULL |
| xalwiki          | en   | NULL | wikipedia | NULL |
| xhwiki           | en   | NULL | wikipedia | NULL |
| xmfwiki          | en   | NULL | wikipedia | NULL |
| yiwiki           | en   | NULL | wikipedia | NULL |
| yowiki           | en   | NULL | wikipedia | NULL |
| zawiki           | en   | NULL | wikipedia | NULL |
| zeawiki          | en   | NULL | wikipedia | NULL |
| zhwiki           | en   | NULL | wikipedia | NULL |
| zh_classicalwiki | en   | NULL | wikipedia | NULL |
| zh_min_nanwiki   | en   | NULL | wikipedia | NULL |
| zh_yuewiki       | en   | NULL | wikipedia | NULL |
| zuwiki           | en   | NULL | wikipedia | NULL |
+------------------+------+------+-----------+------+
291 rows in set (0.01 sec)
MariaDB [(none)]> SELECT dbname, lang, name, family, url FROM meta_p.wiki WHERE (url IS NULL OR name IS NULL) AND family != 'wikipedia';
+-------------+------+------+-------------+--------------------------+
| dbname      | lang | name | family      | url                      |
+-------------+------+------+-------------+--------------------------+
| ukwikimedia | en   | NULL | wikimedia   | https://uk.wikimedia.org |
| centralauth | en   | NULL | centralauth | NULL                     |
+-------------+------+------+-------------+--------------------------+
2 rows in set (0.01 sec)

The sitematrix API is also broken:

new mw.Api().get( {
	action: 'sitematrix',
	smsiteprop: 'dbname|url',
	smlangprop: 'site'
} )
.done( function ( data ) {
	var sites = {};
	$.each( data.sitematrix, function () {
		$.each( this.site || ( $.isArray( this ) ? this : [] ), function () {
			sites[this.dbname] = this.url;
		} );
	} );
	console.log( sites.enwiki );
} );
null

Tools using those data, such as https://tools.wmflabs.org/dewkin/, do not work correctly.

Ricordisamoa raised the priority of this task from "Unbreak Now!" to "Unbreak Before Now!".

Mjbmr set Security to None.Jul 26 2015, 10:21 AM
Mjbmr added a subscriber: Mjbmr.
scfc added a comment.Jul 26 2015, 6:39 PM

(Filed T106989 for my ramblings in T106897#1481393.)

Ricordisamoa raised the priority of this task from "Unbreak Now!" to "Unbreak Before Now!".

Surely that level of priority is reserved for tasks in which the wikis are broken for most readers or something similarly catastrophic.

In general, if you think an issue is extremely urgent, you can join #wikimedia-operations on freenode and poke someone about it. Or you can try direct e-mail.

In this particular case, maybe awarding a "The World Burns" token would be sufficient to express your frustration?

Ricordisamoa raised the priority of this task from "Unbreak Now!" to "Unbreak Before Now!".

Surely that level of priority is reserved for tasks in which the wikis are broken for most readers or something similarly catastrophic.

I didn't actually set any priority here.

In this particular case, maybe awarding a "The World Burns" token would be sufficient to express your frustration?

I'm sorry but I use "The World Burns" with a different meaning (as you can see in my Phabricator prophile).

In this particular case, maybe awarding a "The World Burns" token would be sufficient to express your frustration?

The World is not burning but we cannot effectively patrol lots of smaller wikis, so it's not burning but at least being filled with garbage ;)

yuvipanda reassigned this task from Krenair to coren.Jul 27 2015, 6:04 PM
yuvipanda added a subscriber: yuvipanda.

Change 226939 merged by coren:
maintain-replicas: Also look at wikipedia.dblist

https://gerrit.wikimedia.org/r/226939

coren moved this task from To Do to Doing on the Labs-Sprint-107 board.Jul 27 2015, 7:37 PM
Krenair closed this task as Resolved.Jul 27 2015, 8:30 PM

Looks fixed to me

coren moved this task from Doing to Done on the Labs-Sprint-107 board.Jul 28 2015, 12:27 AM

This was already fixed:

MariaDB [meta_p]> select * from wiki where url is null;
+-------------+------+------+-------------+------+------+-----------+-----------+----------+-----------------+------------------+--------------+--------------+
| dbname      | lang | name | family      | url  | size | slice     | is_closed | has_echo | has_flaggedrevs | has_visualeditor | has_wikidata | is_sensitive |
+-------------+------+------+-------------+------+------+-----------+-----------+----------+-----------------+------------------+--------------+--------------+
| centralauth | en   | NULL | centralauth | NULL |    1 | s7.labsdb |         0 |        0 |               0 |                0 |            0 |            0 |
+-------------+------+------+-------------+------+------+-----------+-----------+----------+-----------------+------------------+--------------+--------------+
1 row in set (0.00 sec)

That's the only row which is supposed to be null.

Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald TranscriptNov 25 2016, 12:59 AM