Page MenuHomePhabricator

updateCollation.php doubles the progress counts when using --force vs --dry-run
Open, Needs TriagePublic

Description

Running updateCollation.php --dry-run takes some time to run, and prints progress with the number of rows that would be updated.

However, when running --force it takes a considerably longer time to complete, and it outputs a lot of more progress counters than with --dry-run. In fact, it *doubles* the progress counts, and also the stats if called with --verbose-stats.

Is the script doing anything two times when run with the --force flag?

According to Manual:updateCollation.php the counts should be the number of rows in the categorylinks table. That's correct when running with --dry-run but doubles it when running with --force.

A real example output:

:.../mediawiki-1.34.1$ nice sudo -u php-fpm php maintenance/updateCollation.php --dry-run --verbose-stats

...

Selecting next 100 rows... processing...90896 rows would be updated so far.
Selecting next 100 rows... processing...90996 rows would be updated so far.
Selecting next 100 rows... processing...91096 rows would be updated so far.
Selecting next 100 rows... processing...91179 rows would be updated so far.

Sort key size histogram
Raw data: 0, 0, 0, 0, 0, 0, 1, 18, 82, 149, 327, 399, 469, 390, 374, 315, 332, 415, 505, 439, 484, 569, 549, 630, 802, 658, 676, 609, 598, 558, 714, 859, 1110, 999, 1096, 1046, 1243, 1180, 1340, 1277, 1551, 1595, 1324, 1349, 1953, 1892, 1822, 1970, 2189, 2131, 2333, 2166, 2148, 2070, 2670, 2441, 2412, 1980, 2197, 1891, 1955, 1923, 1357, 1594, 1331, 1375, 1197, 1259, 1139, 963, 983, 976, 970, 715, 912, 758, 777, 681, 738, 602, 537, 561, 493, 517, 438, 345, 285, 335, 298, 242, 242, 214, 183, 220, 195, 139, 226, 200, 141, 182, 117, 100, 121, 110, 133, 89, 53, 78, 87, 58, 67, 41, 54, 42, 24, 42, 19, 23, 43, 22, 8, 22, 32, 11, 10, 23, 7, 10, 4, 16, 3, 11, 5, 10, 20, 30, 2, 4, 2, 10, 13, 17, 3, 1, 10, 1, 1, 2, 1, 4, 2, 0, 1, 1, 0, 0, 0, 3, 0, 0, 1, 0, 1, 1, 0, 0, 11, 1, 0, 1, 0, 1, 10, 1, 0, 1, 1, 1, 23, 3, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 2, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1

0-9:       250        |
10-20:     4449       |***********
21-30:     6363       |****************
31-41:     13296      |**********************************
42-51:     19129      |*************************************************
52-62:     23044      |************************************************************
63-72:     11787      |******************************
73-83:     7291       |******************
84-93:     2802       |*******
94-104:    1664       |****
105-114:   593        |*
115-124:   232        |
125-135:   139        |
136-145:   63         |
146-156:   12         |
157-166:   17         |
167-177:   17         |
178-187:   29         |
188-198:   5          |
199-209:   1          |


:.../mediawiki-1.34.1$ nice sudo -u php-fpm php maintenance/updateCollation.php --force --verbose-stats

...

Selecting next 100 rows... processing...181900 done.
Selecting next 100 rows... processing...182000 done.
Selecting next 100 rows... processing...182100 done.
Selecting next 100 rows... processing...182200 done.
Selecting next 100 rows... processing...182300 done.
Selecting next 100 rows... processing...182366 done.
182366 rows processed

Sort key size histogram
Raw data: 0, 0, 0, 0, 0, 0, 2, 36, 164, 298, 654, 798, 938, 780, 748, 630, 664, 830, 1010, 878, 968, 1138, 1098, 1260, 1604, 1316, 1352, 1218, 1196, 1116, 1428, 1718, 2220, 1998, 2192, 2092, 2486, 2360, 2680, 2554, 3102, 3190, 2648, 2698, 3906, 3784, 3644, 3940, 4378, 4262, 4666, 4332, 4296, 4140, 5340, 4882, 4824, 3960, 4394, 3782, 3910, 3846, 2714, 3188, 2662, 2750, 2394, 2518, 2278, 1926, 1966, 1952, 1940, 1430, 1824, 1516, 1554, 1362, 1476, 1204, 1074, 1122, 986, 1034, 876, 690, 570, 670, 596, 484, 484, 428, 366, 440, 390, 278, 452, 400, 282, 364, 234, 200, 242, 220, 266, 178, 106, 156, 174, 116, 134, 82, 108, 84, 48, 84, 38, 46, 86, 44, 16, 44, 64, 22, 20, 46, 14, 20, 8, 32, 6, 22, 10, 20, 40, 60, 4, 8, 4, 20, 26, 34, 6, 2, 20, 2, 2, 4, 2, 8, 4, 0, 2, 2, 0, 0, 0, 6, 0, 0, 2, 0, 2, 2, 0, 0, 22, 2, 0, 2, 0, 2, 20, 2, 0, 2, 2, 2, 46, 6, 2, 0, 0, 2, 0, 0, 0, 2, 0, 2, 0, 0, 0, 0, 0, 4, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2

0-9:       500        |
10-20:     8898       |***********
21-30:     12726      |****************
31-41:     26592      |**********************************
42-51:     38258      |*************************************************
52-62:     46088      |************************************************************
63-72:     23574      |******************************
73-83:     14582      |******************
84-93:     5604       |*******
94-104:    3328       |****
105-114:   1186       |*
115-124:   464        |
125-135:   278        |
136-145:   126        |
146-156:   24         |
157-166:   34         |
167-177:   34         |
178-187:   58         |
188-198:   10         |
199-209:   2          |