Page MenuHomePhabricator

Remove deleted wiki's entries from global tables
Closed, ResolvedPublic

Description

de-labs.wikimedia.org and en-labs.wikimedia.org were closed in the past. Subdomains were removed from DNS but there are still entries in the GlobaleUsage table, see URL. These entries are misleading.

As far as I see there is no maintenance script to fix this error.

I suggest either

a) Remove these entries per manual SQL magic

and/or

b) Write a maintenance script to do this


Version: wmf-deployment
Severity: normal
URL: https://commons.wikimedia.org/wiki/File:Caprese-1.jpg
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=54894

Details

Reference
bz49858

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 22 2014, 2:10 AM
bzimport set Reference to bz49858.
bzimport added a subscriber: Unknown Object (MLST).
Raymond created this task.Jun 20 2013, 6:33 AM

Deleted wiki names from https://git.wikimedia.org/blob/operations%2Fmediawiki-config.git/HEAD/deleted.dblist

MariaDB [commonswiki_p]> SELECT COUNT(*) FROM globalimagelinks WHERE gil_wiki IN ('bawiktionary', 'chwikimedia', 'closed_zh_twwiki', 'de_labswikimedia', 'dkwikibooks', 'dkwiktionary', 'en_labswikimedia', 'flaggedrevs_labswikimedia', 'liquidthreads_labswikimedia', 'nomcomwiki', 'ptwikimedia', 'readerfeedback_labswikimedia', 'ru_sibwiki', 'strategyappswiki', 'tlhwiki', 'tlhwiktionary', 'tokiponawiki', 'tokiponawikibooks', 'tokiponawikiquote', 'tokiponawikisource', 'tokiponawiktionary');
+----------+

COUNT(*)

+----------+

191913

+----------+
1 row in set (0.53 sec)

Expanding the scope of this bug a bit to include CentralAuth's localuser and localnames tables. According to TS user scfc_de, localuser has 3865 entries from deleted wikis and localnames has 4128.

A script should be written for deleting wikis (a script for adding wikis exists), where this maintainance work can be added.

(In reply to comment #0)

a) Remove these entries per manual SQL magic
and/or
b) Write a maintenance script to do this

Only option B seems reasonable. :-)

(In reply to comment #2)

Expanding the scope of this bug a bit to include CentralAuth's localuser and
localnames tables. According to TS user scfc_de, localuser has 3865 entries
from deleted wikis and localnames has 4128.

Hmmm. Are you thinking about one maintenance script with an array of tables to check or are you thinking about individual maintenance scripts (i.e., one script for CentralAuth, one script for GlobalUsage, etc.)?

Either approach could work, but depending on the path forward, it may make sense to split this bug out into smaller, more discrete action items.

Reedy added a comment.Oct 3 2013, 8:23 PM

GlobalUseage has been dealt with in bug 54894, simple script created and ran on the cluster.

Other than CentralAuth, do we have anything else that needs cleaning up?

Change 89484 had a related patch set uploaded by Reedy:
Remove deleted wikis entries from CentralAuth tables

https://gerrit.wikimedia.org/r/89484

Change 89484 merged by jenkins-bot:
Remove deleted wikis entries from CentralAuth tables

https://gerrit.wikimedia.org/r/89484

Reedy added a comment.Oct 22 2013, 7:36 PM

reedy@tin:/a/common/php-1.22wmf22$ mwscript extensions/WikimediaMaintenance/removeDeletedWikis.php --wiki=commonswiki
bawiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
chwikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
39
39 localnames rows deleted
localuser:
25
25 localuser rows deleted
closed_zh_twwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
de_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
500
634
634 localnames rows deleted
localuser:
500
594
594 localuser rows deleted
dkwikibooks:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
2
2 localnames rows deleted
localuser:
1
1 localuser rows deleted
dkwiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
5
5 localnames rows deleted
localuser:
4
4 localuser rows deleted
en_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
500
1000
1500
1962
1962 localnames rows deleted
localuser:
500
1000
1500
1785
1785 localuser rows deleted
flaggedrevs_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
500
894
894 localnames rows deleted
localuser:
500
856
856 localuser rows deleted
liquidthreads_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
3
3 localuser rows deleted
nomcomwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
ptwikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
200
200 localnames rows deleted
localuser:
195
195 localuser rows deleted
readerfeedback_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
392
392 localnames rows deleted
localuser:
360
360 localuser rows deleted
ru_sibwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
3
3 localuser rows deleted
strategyappswiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tlhwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
13
13 localuser rows deleted
tlhwiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
31
31 localuser rows deleted
tokiponawiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawikibooks:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawikiquote:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawikisource:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
Done.
reedy@tin:/a/common/php-1.22wmf22$