Do not rerender whole client page when only sitelinks are updated
Open, NormalPublic

Description

When sitelinks are updated, the content of the page on the client wiki typically does not need to be rerendered. When a sitelink changes, we should change the cached ParserOutput with the new list of sitelinks, without touching the generated HTML.

Considerations: rendering wikitext to HTML is expensive. Updates to sitelinks are frequent.

Details

Reference
bz42783
bzimport raised the priority of this task from to Normal.
bzimport set Reference to bz42783.
bzimport added a subscriber: Unknown Object (MLST).
Denny created this task.Dec 6 2012, 3:01 PM
  • Bug 47159 has been marked as a duplicate of this bug. ***
Lydia_Pintscher removed a subscriber: Unknown Object (MLST).
Lydia_Pintscher edited subscribers, added: hoo, aude; removed: Wikidata-bugs.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 22 2015, 2:47 PM
thiemowmde closed this task as Declined.Aug 26 2016, 12:40 PM
thiemowmde added a subscriber: thiemowmde.

In my time in the team I never heard anybody talking about this being an actual issue we need to work on. We are indeed rerendering the whole HTML page. We do caching on so many levels to make this fast. To what "the page itself" refers, I can't tell.

daniel reopened this task as Open.Aug 26 2016, 8:18 PM

Reopening. Should at least be discussed before closing.

@thiemowmde This is about pages on the client (i'll update the decsription). When only sitelinks change, all we'd need to do is to update the list of sitelinks in the ParserOutput object, and not touch the HTML. Since generating HTML from Wikitext is quite expensive (and our caching doesn't help with that), this seems to be a worth-while optimization. Changes to sitelinks are one of the most common types of updates we have on wikidata. We should avoid re-rendering all linked pages when adding a sitelink.

Maybe this is too complicated, or not worth the effort, but we should at least discuss it again before closing this.

daniel renamed this task from Do not rerender whole page when only sitelinks are updated to Do not rerender whole client page when only sitelinks are updated.Aug 26 2016, 8:20 PM
daniel updated the task description. (Show Details)

Change 307123 had a related patch set uploaded (by Hoo man):
[RfC] Update the langlinks in the canonical parser cache if possible

https://gerrit.wikimedia.org/r/307123

hoo added a comment.Aug 28 2016, 1:11 PM

I found that implementing this is rather easy, thus I quickly hacked it up (and verified it manually, it doesn't discard parser cache entries). My patch is not polished and there are some remaining issues (especially regarding the timing with RefreshlinksJobs, but that's solvable).

Please have a look and decide whether we actually want to do this.

Given that it's possible to have Lua modules that do something with the sitelinks, how can we distinguish cases where a changed sitelink is used exclusively in the "other languages" section from cases where the same edit on wikidata.org affects other parts of the same page?

hoo added a comment.Aug 29 2016, 10:38 AM

Given that it's possible to have Lua modules that do something with the sitelinks, how can we distinguish cases where a changed sitelink is used exclusively in the "other languages" section from cases where the same edit on wikidata.org affects other parts of the same page?

We use EntityUsage::SITELINK_USAGE for the language links in the sidebar, but EntityUsage::OTHER_USAGE if the sitelinks are used in wikitext. Because of this, we can know when only the sidebar needs to be updated. See the constants in EntityUsage for details.

thiemowmde assigned this task to hoo.
thiemowmde moved this task from Proposed to Doing on the Wikidata-Sprint-2016-08-30 board.
thiemowmde moved this task from Proposed to Doing on the Wikidata-Sprint-2016-08-16 board.
hoo added a comment.Sep 14 2016, 12:11 AM

I didn't receive any input on my (RfC) change in over two weeks, I perceive this as a strong lack of interest in actually implementing this. If so, shall we decline this?

I believe most people (including me) have not looked at https://gerrit.wikimedia.org/r/307123 for two reasons: Tests fail for weird reasons, and the commit message is hard to understand. It's titled "Update the langlinks in the canonical parser cache if possible". "Possible" depending on what? What is a "canonical" cache? I never heard this term.

What are the acceptance criteria, if any? Do we really know this is an issue we want or should fix? Do we have numbers? How big of an issue is this? Personally I can't tell, even if I can imagine the impact. Sorry. I'm asking because I feel this is what's needed to get this moving.