Page MenuHomePhabricator

Add async email-sending strategy
Closed, ResolvedPublic


Right now TWLight sends email as part of the request-response loop. This will do for now but will rapidly become untenable with scale. The djmail system can have other backends swapped in - its expected option for async email sending is to use celery, which requires setting up celery and a broker for it and editing settings/ accordingly.

Event Timeline

Samwalton9 lowered the priority of this task from Normal to Low.Feb 3 2017, 6:45 PM
Samwalton9 added a subscriber: jsn.sherman.
Samwalton9 added a subscriber: Samwalton9.

@jsn.sherman you might want to take a look at this to decide on priority, I'm not sure how much of a concern this will be.

After doing some research on this, I recommend ditching direct emails for communicating with editors and coordinators, and instead leverage the email user api to have the wiki infrastructure send the mail on our behalf.

We'll still need to send our own mails to partners, but that should be much lower volume. We could even bring those out of the platform by just generating mailto links with the correct information filled in.

Samwalton9 raised the priority of this task from Low to Normal.Oct 19 2017, 6:37 PM
Samwalton9 raised the priority of this task from Normal to High.

Django has some decent email testing tools that we should probably be using. DJmail, which is our current mail handling library, has some asynchronous options that don't have external dependencies, but I'll need to get at least some basic email testing in place before we can reconfigure it.

Quick update: I'm currently testing using an async djmail backend for sending mail. Currently, it offers all of the hoped for performance benefits of an async mail queue, minus the capability to actually transfer any email messages. In other words, I'm making progress, but I still have a little work to do.

async email config pushed to prod. Testing turned up a few more cases where there the user salutation ended up being the numerical ID T169525. It was down to the fact that we're using a separate system for app comments than for the rest of the things that generate emails. That's now been resolved as well.

This should really only make a perceivable difference in page loads that are causing a lot of emails to be sent. Eg. waitlisting a partner with a lot of applications previously took a very long time to complete, or timed out.

Samwalton9 closed this task as Resolved.Jan 8 2018, 6:11 PM