Page MenuHomePhabricator

db1072 change_tag schema and dataset is not consistent
Closed, ResolvedPublic

Description

This is a subtask of T155999 but in no way a blocker- we just discovered it when preparing for it.

We discovered that tags on db1072 are duplicated:

SELECT ct_rc_id, ct_tag FROM change_tag GROUP BY ct_rc_id, ct_tag HAVING count(*) > 1;

returns more than 65K results, while only 5 (NULL) values everywhere. This is possible because the table definition is wrongly:

CREATE TABLE `change_tag` (
  `ct_rc_id` int(11) DEFAULT NULL,
  `ct_log_id` int(11) DEFAULT NULL,
  `ct_rev_id` int(11) DEFAULT NULL,
  `ct_tag` varbinary(255) NOT NULL DEFAULT '',
  `ct_params` blob,
  KEY `change_tag_rc_tag` (`ct_rc_id`,`ct_tag`),
  KEY `change_tag_log_tag` (`ct_log_id`,`ct_tag`),
  KEY `change_tag_rev_tag` (`ct_rev_id`,`ct_tag`),
  KEY `change_tag_tag_id` (`ct_tag`,`ct_rc_id`,`ct_rev_id`,`ct_log_id`)
) ENGINE=InnoDB DEFAULT CHARSET=binary

(there are no unique-key restrictions).

So the propsed fix is:

  • For DBA s: fix the table - probably requires a full reimage of the dataset
  • For change_tag maintainers - even despite the lack of restrictions, which is a guarantee, this is an example of writes being done in an unsafe way- despite the operational problem, it is not an excuse for duplicate results to be present there. The fact the duplicates are even tried to be inserted means there is persistence model problem too.

We can create separate subtasks for both aspects of the problem, if that helps.

Event Timeline

Adding @TTO and @Cenarium because they may know the actual right people to add to this ticket (probably not them) for the mediawiki bug side of things.

It looks like the author of ChangeTags::updateTags intended for the DB unique key to act as protection against duplicate entries. This pattern (using INSERT IGNORE on a table with a unique key) is widespread in MediaWiki core, so I don't think it's necessarily fair to pick on ChangeTags for doing it :)

is widespread in MediaWiki core

And we should kill them with fire! :-P

I got the architecture comittee to agree with me and document that as a bug a few months ago: https://www.mediawiki.org/wiki/Development_policy#Database_policy

Non-deterministic queries and unsafe statements for binlog should be avoided as they would return/write different results in a replication environment.

jcrespo assigned this task to Marostegui.

This is resolved, leaving T156226 open for pending issues, non-related.

SELECT ct_rc_id, ct_tag FROM change_tag GROUP BY ct_rc_id, ct_tag HAVING count(*) > 1;
+----------+-----------------------+
| ct_rc_id | ct_tag                |
+----------+-----------------------+
|     NULL | WPCleaner             |
|     NULL | contenttranslation    |
|     NULL | visualeditor          |
|     NULL | visualeditor-switched |
|     NULL | wikilove              |
+----------+-----------------------+
5 rows in set (1 min 7.15 sec)