Page MenuHomePhabricator

Duplicate rows in ipblocks table causing exceptions from maintenance/update.php in MW 1.35
Closed, DeclinedPublicBUG REPORT

Description

If the ipblocks table from a prior-version MediaWiki installation contains any duplicate records (ie: two blocks against the same user or address), the upgrade from MW 1.31 (or earlier) to MW 1.35 fails with a fatal error, as it attempts to create a "unique index" on this field without verifying the sanity of the existing data in the table.

Steps to Reproduce:
On an existing MW 1.31 installation which contains duplicate entries in the ipblocks table (such as two blocks against the same user, or against the same address), any attempt to run maintenance/update.php to "upgrade" to the current MW version (1.35) fails with a fatal error.

The update script attempts to place a "unique index" on the ipblocks.ipb_address field. This operation fails if the existing data in the table contains duplicate records.

No idea how the dupes originally got in there, but this is an existing install which has been live since MW 1.6 (in 2006) and none of the earlier versions seemed to care.

Actual Results:

It's impossible to complete the "upgrade" to MW 1.35 without manually going into the ipblocks table, finding the duplicates with something like:

SELECT DISTINCT CONCAT(ipb_address,''), SUM(1) as count from ipblocks group by ipb_address having count > 1 order by count desc, ipb_address limit 50

and then manually, individually deleting each of them.

Expected Results:

The script needs to detect the duplicates, then either get rid of them (perhaps by copying everything to another table with GROUP BY or INSERT IGNORE) or not attempt to apply the unique index to an existing table which doesn't comply with the prerequisite that the ipb_address actually be unique.

Event Timeline

Do you have an old version of that table?

Noting the unique wasn't just on ipb_address

UNIQUE INDEX /*i*/ipb_address_unique (ipb_address(255), ipb_user, ipb_auto)

Not surprising, since tables.sql for MediaWiki 1.31 has:

CREATE UNIQUE INDEX /*i*/ipb_address ON /*_*/ipblocks (ipb_address(255), ipb_user, ipb_auto, ipb_anon_only);

Excluding a column from a multi-column unique index has the potential to cause duplicate key errors.

This comes from T251188, a very confusingly worded task. The duplicate key issue was brought up on T251188#6105539 but no public disclosure about what actions were made to mitigate that (for example, which duplicate row to delete)

Duplicate entries can be removed running those statements:

CREATE TEMPORARY TABLE dupeipbocks AS (select min(ipb_id) as blockid from /*_*/ipblocks group by ipb_address, ipb_user, ipb_auto having count(*) > 1);

delete from /*_*/ipblocks where ipb_id in (select blockid from dupeipbocks);

DROP TEMPORARY TABLE dupeipbocks;

This retrieves the duplicate rows, and selects the lower block ID for each duplicate. Stores them to a temporary table and then deletes them from the ipblocks table. This will leave only the most recent block for each duplicate.

I suggest adding those SQL statements to the file: maintenance/archives/patch-ipblocks-rename-ipb_address.sql, before the ALTER statement.