Page MenuHomePhabricator

Eliminate creating of temp tables from Omnimailing.load task
Closed, ResolvedPublic1 Estimated Story Points

Description

In response to Jeff identifying a pattern of creating temp tables when creating mailings I have added a bypass on the getRecipients part when creating the Mailings in this job.

As a result of this I also identified that the replace api does not work as I thought it did (it creates & deletes rather than creates or updates). As a result I have replaces this with an internal function. We also use replace on addresses & emails in core - although currently only on lower traffic jobs. If we were to adopt @Ejegg's idea of doing more matching we would want to change that

Event Timeline

Change 383734 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Fix Omnimailing.load job to bypass creating recipients & to use internal replace, add test.

https://gerrit.wikimedia.org/r/383734

Change 383734 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Fix Omnimailing.load job to bypass creating recipients & to use internal replace, add test.

https://gerrit.wikimedia.org/r/383734

This appears to have entirely fixed the temp table bloat issue!

Let's hope it was the cause of the replag (it feels right given I can actually construct a narrative that fits with what we know for this - ie. the temp table bloat can explain how the replag related to uptime)