Page MenuHomePhabricator
Create Task
Maniphest T164860

Update Echo's caching strategy for multi-dc compatibility
Closed, ResolvedPublic
Actions

Description

As suggested in T164505#3237739 and following comments, Echo should use a purge-on-write, cache-from-slave WANCache approach to caching, rather than its current recache-on-write strategy. This is probably necessary to make Echo work in a multi-DC environment, and will also address T164505: Read queries detected on the x1 master by Echo extension more fully.

Details

SubjectRepoBranchLines +/-
NotifUser: Remove basically-unused talk notification cachemediawiki/extensions/Echomaster+3 -97
[WIP] Make "echo-new-talk-notification" key use getWithSetCallback()mediawiki/extensions/Echomaster+52 -39
NotifUser: Redo caching strategy for multi-DC compatibilitymediawiki/extensions/Echomaster+174 -195
CachedList: Use getWithSetCallback()mediawiki/extensions/Echomaster+15 -19
Customize query in gerrit

Related Objects
Search...

Event Timeline

Catrope created this task.May 9 2017, 6:22 PM
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptMay 9 2017, 6:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Catrope added a subscriber: aaron.May 9 2017, 6:23 PM

@aaron Do I understand correctly that Echo's current strategy, which is to overwrite (as opposed to purge) entries in WANObjectCache, needs to be fixed for multi-DC?

Catrope added a parent task: T88445: MediaWiki active/active datacenter investigation and work (tracking).May 9 2017, 6:23 PM
Catrope mentioned this in T164505: Read queries detected on the x1 master by Echo extension.
Catrope added a comment.May 9 2017, 7:44 PM

Potentially related: T138341: Consider replacing echo_unread_wikis table with a cache

aaron added a comment.May 9 2017, 8:14 PM

Sounds correct to me.

• jmatazzoni moved this task from Untriaged to Near-Term Interest on the Collaboration-Team-Triage board.Jun 13 2017, 7:30 PM
Krinkle added a project: Sustainability (MediaWiki-MultiDC).Feb 21 2018, 11:25 PM
Catrope claimed this task.May 23 2018, 1:51 PM
gerritbot subscribed.May 29 2018, 11:46 PM

Change 436204 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/Echo@master] CachedList: Use getWithSetCallback()

https://gerrit.wikimedia.org/r/436204

gerritbot added a project: Patch-For-Review.May 29 2018, 11:46 PM

Change 436205 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/Echo@master] NotifUser: Redo caching strategy for multi-DC compatibility

https://gerrit.wikimedia.org/r/436205

gerritbot added a comment.May 30 2018, 9:55 PM

Change 436204 merged by jenkins-bot:
[mediawiki/extensions/Echo@master] CachedList: Use getWithSetCallback()

https://gerrit.wikimedia.org/r/436204

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)).May 30 2018, 10:00 PM
Catrope added a comment.May 31 2018, 12:55 AM

These patches fix most of the issue, but there's still one left: flagCacheWithNewTalkNotification() and flagCacheWithNoTalkNotification() still call WANObjectCache::set(). It's a lot harder to figure out what that cache key actually means (and what the set callback for this key would look like), so I'm going to come back to this a little later.

gerritbot added a comment.May 31 2018, 11:35 PM

Change 436697 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/Echo@master] NotifUser: Remove basically-unused talk notification cache

https://gerrit.wikimedia.org/r/436697

gerritbot added a comment.Jun 2 2018, 5:10 PM

Change 436205 merged by jenkins-bot:
[mediawiki/extensions/Echo@master] NotifUser: Redo caching strategy for multi-DC compatibility

https://gerrit.wikimedia.org/r/436205

gerritbot added a comment.Jun 21 2018, 11:34 AM

Change 441377 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/extensions/Echo@master] [WIP] Make "echo-new-talk-notification" key use getWithSetCallback()

https://gerrit.wikimedia.org/r/441377

gerritbot added a comment.Jul 13 2018, 2:19 PM

Change 441377 abandoned by Aaron Schulz:
[WIP] Make "echo-new-talk-notification" key use getWithSetCallback()

https://gerrit.wikimedia.org/r/441377

Restricted Application added a project: Growth-Team. · View Herald TranscriptJul 13 2018, 2:19 PM
JTannerWMF moved this task from Inbox to Needs Discussion on the Growth-Team board.Jul 18 2018, 6:32 PM
SBisson edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.Jul 20 2018, 6:42 PM
SBisson moved this task from Incoming to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.
gerritbot added a comment.Aug 10 2018, 2:05 AM

Change 436697 merged by jenkins-bot:
[mediawiki/extensions/Echo@master] NotifUser: Remove basically-unused talk notification cache

https://gerrit.wikimedia.org/r/436697

ReleaseTaggerBot edited projects, added MW-1.32-notes (WMF-deploy-2018-08-14 (1.32.0-wmf.17)); removed MW-1.32-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)).Aug 10 2018, 3:00 AM
aaron added a comment.Aug 10 2018, 8:02 PM

Can this task be closed?

Catrope closed this task as Resolved.Aug 13 2018, 7:01 PM
In T164860#4495353, @aaron wrote:

Can this task be closed?

Yes, the last patch for it was just merged, I think we're done now.

Krinkle edited projects, added Performance-Team (Radar); removed Patch-For-Review.Aug 13 2018, 7:37 PM
Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.
Jdforrester-WMF edited projects, added MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)); removed MW-1.32-notes (WMF-deploy-2018-08-14 (1.32.0-wmf.17)).Aug 14 2018, 8:55 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL