Page MenuHomePhabricator

Invalidation of Gadgets cache is broken after switch to APC
Closed, ResolvedPublic

Description

The invalidation of the gadgets cache is broken because APC is per-server, and doesn't clear the cache on all servers. Caused by https://gerrit.wikimedia.org/r/#/c/210260/

Event Timeline

Legoktm created this task.May 18 2015, 7:05 PM
Legoktm raised the priority of this task from to Unbreak Now!.
Legoktm updated the task description. (Show Details)
Legoktm added subscribers: Legoktm, ori, matmarex.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 18 2015, 7:05 PM
Krenair updated the task description. (Show Details)May 18 2015, 7:08 PM
Krenair set Security to None.

The current plan is to have https://gerrit.wikimedia.org/r/#/c/207730/ fix this by using WANCache.

Note that the list is cached for 24 hours only, so updates will take about that long to propagate.

He7d3r added a subscriber: He7d3r.May 22 2015, 4:58 PM

Any progress? I've just had confused Wikimedians ask me what kind of magic incantation they need to add a gadget, because just doing it no longer works. Can't we revert the bad change?

I would bump the priority, but it's already been "Unbreak now!" for a week. :(

matmarex assigned this task to ori.May 23 2015, 9:09 PM
matmarex added a subscriber: greg.
Ankry added a subscriber: Zdzislaw.May 24 2015, 5:56 AM

Please, do something, roll-back 210260 or... disable completely the gadget extension. In the current state it is not possible to manage and supervise (introduced) changes in Gadgets extension. I have made a few changes (not knowing about the problems), ... waiting for the effects... and nothing happens - how do I test the changes if I do not know when they will appear in the users (and my) preferences tab?
Additional side effects of the problem (as I noticed) are such that changes in "Gadgets-definition" (i.e. new gadgets) occur in the users preferences tab for a moment (randomly) and the next time when I refresh the browser they disappear.
Testing the changes in gadgets-definition immediately as they are added is critical, and I can not imagine waiting "for 24 hours" every time when I change gadgets-definition.

Regards, Z.

The fix mentioned above was backported and deployed to WMF sites:
23:30 logmsgbot: ori Synchronized php-1.26wmf7/extensions/Gadgets: b592efa5fe: Update Gadgets for I6da3eede0: Conversion to using WAN cache (duration: 00m 13s)
Is it really still broken?

I have checked a while ago in plwikisource, and I receive randomly two various versions of the page (gadget list): sometimes the newest one, sometimes a few yesterday revisions older (a gadget was renamed yesterday).

Zdzislaw added a comment.EditedMay 24 2015, 12:16 PM

@Krenair Yes, the problem still exists. I made a change in "MediaWiki:Gadgets-definition" a half-hour ago, and (on pl source) the changes are not visible. But, what we can observe is a state (of Special:Gadgets and Preferences) before the change, and randomly... state of "a few changes ago".
@Ankry: I made a change at 11:37 , so... the one of the version you can receive it's not "the newest one" but "before the change" :)

Currently, there is a third version that we can receive for a moment (randomly) - version before all the changes of the last two days (without new Gadgets), but... not the newest one :(

Regards, Z.

@ori: Any input here, regarding the last comment? Thanks in advance.

Is this fixed now? It's been "Unbreak Now!" for two weeks now. @ori, are you working on this? The lac of updates here is disappointing :(

ori closed this task as Resolved.Jun 5 2015, 12:46 AM

Is this fixed now? It's been "Unbreak Now!" for two weeks now. @ori, are you working on this? The lac of updates here is disappointing :(

It was hard for me to pinpoint exactly when the fix rolled out because of the wmf7 / 8 flapping. It should be fixed now, yes.

Ankry added a comment.Jun 5 2015, 3:14 PM

I confirm, gadget definition page updates are visible immediately, no delay (1.26wmf8 (dab1803) on plwikisource).