Page MenuHomePhabricator

Old references are not removed properly when removing them from the content
Closed, ResolvedPublic2 Estimated Story Points

Description

This Flow topic makes use of a template in the summary section to facilitate sorting into categories (and to help with summarizing the topic in general). The template is located here.

I recently updated the names of categories in the template, and tried purging the cache of the flow topic (via ?action=purge) to update the categories. This did absolutely nothing, so I tried removing the template and adding it back in, and now it uses a bizarre mix of the old and new category names:

All approved Wikipedia article requests
Approved Wikipedia article requests
WikiProject United States requested articles

"Approved Wikipedia article requests" and "WikiProject United States requested articles" are artifacts of an older version of the template. "All approved Wikipedia article requests" is rightfully displayed, but "WikiProject United States approved article requests" is oddly missing.

Also happens when templates are not involved. See https://en.wikipedia.org/w/index.php?title=Topic:Selbmxmvhx79462g&topic_showPostId=selchds8is4qutoz#flow-post-selchds8is4qutoz .

Event Timeline

Harej raised the priority of this task from to Needs Triage.
Harej updated the task description. (Show Details)
Harej added a project: StructuredDiscussions.
Harej subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Mattflaschen-WMF renamed this task from Regular cache purging does not seem to work with Flow to Old categories are not removed properly when removing and re-adding a template.Mar 31 2015, 4:10 PM
Mattflaschen-WMF reopened this task as Open.
Mattflaschen-WMF set Security to None.

The part about transclusions not updating automatically is T59991: Changes in templates are not immediately reflected in Flow posts which transclude those templates.

However, removing and re-adding the template should still work.

Mattflaschen-WMF renamed this task from Old categories are not removed properly when removing and re-adding a template to Old categories are not removed properly when removing and re-adding a template to summary.Mar 31 2015, 4:12 PM
EBernhardson subscribed.
Mattflaschen-WMF renamed this task from Old categories are not removed properly when removing and re-adding a template to summary to Old categories are not removed properly when removing and re-adding a template to summary (or just adding new categories).Mar 31 2015, 10:42 PM
Mattflaschen-WMF updated the task description. (Show Details)
Mattflaschen-WMF renamed this task from Old categories are not removed properly when removing and re-adding a template to summary (or just adding new categories) to Old categories are not removed properly when removing and re-adding a template to summary or header (or just adding new categories).Aug 7 2015, 11:51 PM
DannyH lowered the priority of this task from High to Medium.Aug 11 2015, 11:54 PM
Catrope raised the priority of this task from Medium to High.Aug 18 2015, 5:39 PM
Mattflaschen-WMF renamed this task from Old categories are not removed properly when removing and re-adding a template to summary or header (or just adding new categories) to Old wiki references are not removed properly when removing them from the content.Aug 19 2015, 6:31 AM

This is apparently broken for all wiki references: https://www.mediawiki.org/wiki/Special:WhatLinksHere/obscure_red_link

Mattflaschen-WMF renamed this task from Old wiki references are not removed properly when removing them from the content to Old references are not removed properly when removing them from the content.Aug 19 2015, 6:34 AM

I believe I've figured out why this is happening (see below), but I'm still working on it.

  • FeatureIndex::onAfterInsert (the one that updates cache data) seems to have some faults (doesn't re-compact). Apart from references, we never delete anything, so we probably never noticed.
  • TopKIndex::removeFromIndex (and probably others) currently fails to locate a row in cache because the data in cache includes a currently unused ref_src_wiki column from when cache got populated from DB. We have some code to normalize rows, but that'll be hard to reach from there...
  • TopKIndex::removeFromIndex may have other issues when finding an occurence of a value in index (e.g. different order of keys?)
  • ReferenceRecorder::onAfterInsert probably shouldn't call LinksTableUpdater::doUpdate right away - that should be deferred until data has been committed (unless our buffered cache is that good, but still...). Otherwise, it compares the same data and can't figure out that categories have been removed.
  • TopKIndex::removeFromIndex (and probably others) currently fails to locate a row in cache because the data in cache includes a currently unused ref_src_wiki column from when cache got populated from DB. We have some code to normalize rows, but that'll be hard to reach from there...

Should we block this on T107204: Separate reference tables by wiki so we don't have to work around it in the meantime?

Change 233432 had a related patch set uploaded (by Matthias Mullie):
[WIP] Fix references/storage

https://gerrit.wikimedia.org/r/233432

In T94569#1566271, @Mattflaschen wrote:

Should we block this on T107204: Separate reference tables by wiki so we don't have to work around it in the meantime?

I wanted to fix it for real because if we ever need to do schema changes again, we'd run into the same problem (and lose time trying to figure this out again) and there is no way to work around it until the schema changes are done.

Change 233432 merged by jenkins-bot:
Fix removal/addition of categorylinks

https://gerrit.wikimedia.org/r/233432

Checked in betalabs with the sample template from

The part about transclusions not updating automatically is T59991: Changes in templates are not >immediately reflected in Flow posts which transclude those templates.

T59991 filed Nov 2014) is still Open.

However, removing and re-adding the template should still work.

Removing and re-adding the template works now - the changes in templates will be displayed on pages that use them.