Page MenuHomePhabricator

Outbound mail from Greenhouse is broken
Closed, ResolvedPublic

Description

When sending email out of Greenhouse.io, it advises that "It's unlikely that this email will be delivered." The default From address is careers@wikimedia.org. For this to work, the wikimedia.org SPF record would have to include Greenhouse's SPF policy. The exact recommended SPF policy can apparently only be viewed by a Greenhouse admin, which I'm not, but according to a screenshot in the documentation it would be something like include:mg-spf.greenhouse.io

https://support.greenhouse.io/hc/en-us/articles/201111684

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

This has been discussed in bigger requests a couple of times before (T103893, T84201) for Greenhouse specfically, plus a bunch of other times for other third-party services. The TL;DR is that we don't really like whitelisting in SPF/DKIM/DMARC for wikimedia.org for all of the third-party services that we use, because that opens up attack vectors like email spoofing, CEO fraud to entities that we do not control nor are able to vet their security. The alternative we had proposed before was to use a separate subdomain (careers.wikimedia.org). It's still non-ideal, but it's better than allowing them and others like them to send emails us as <insert ED name>@wikimedia.org for instance.

Does that make sense? I'm going to leave the task open in case you disagree and we can discuss further :)

How about I change the task title so that it can stay open? Because the real problem here is that outbound email is broken, I don't care whether SPF or a subdomain is used to fix it. There's no MX record for careers.wikimedia.org.

tstarling renamed this task from SPF for Greenhouse to Outbound mail from Greenhouse is broken.Mar 8 2018, 5:16 AM

Change 417350 had a related patch set uploaded (by Herron; owner: Herron):
[operations/dns@master] WIP: add careers.wikimedia.org spf/dkim/mx records

https://gerrit.wikimedia.org/r/417350

Here's a patch to get the ball rolling on a subdomain for this. It's WIP since we will need an admin on the greenhouse account to supply the correct dkim/spf/mx values.

So we need a Greenhouse admin to go to Configure > Email Settings, then enter "careers.wikimedia.org" for the domain and click "Register". Then click "Email your I.T. dept" and send it to kherron@wikimedia.org.

Change 417350 had a related patch set uploaded (by Herron; owner: Herron):
[operations/dns@master] WIP: add careers.wikimedia.org spf/dkim/mx records

https://gerrit.wikimedia.org/r/417350

Is there any reason we cannot merge this in advance of the greenhouse.io settings change?

An email was sent to the recruitment team regarding this settings change over a week ago, and I'll send a followup email to them later today. It seems to be blocked on that, plus the patchset that I think is fine for us to merge now.

Is there any reason we cannot merge this in advance of the greenhouse.io settings change?

The change is immature as-is, unfortunately. Before merging we will want confirmation that the SPF settings proposed via gerrit are correct, and to include the correct values for dkim and mx (after they are obtained by a greenhouse admin)

Tim emailed about this a couple of weeks ago, and I sent out another email to them regarding this. Hopefully it gets some movement soon.

RobH triaged this task as High priority.Apr 2 2018, 2:54 PM
herron changed the task status from Open to Stalled.Apr 10 2018, 5:26 PM
herron moved this task from Backlog to Up Next on the Mail board.

Hi All,

Has this subdomain been created? Either careers.wikimedia.org or gh-mail.wikimedia.org? Can it be re-iterated why the domain creation is stalled?

It looks like we need the domain setup before it's setup in greenhouse, otherwise mailflow in green house will probably not flow if we change the domain there.

Thanks,
Byron

Hey @bbogaert, there is an unmerged WIP patch to create the subdomain here https://gerrit.wikimedia.org/r/417350. It's stalled waiting for details/confirmation from a greenhouse admin about what MX, SPF, DMARC, etc. records need to be used.

The steps outlined in https://phabricator.wikimedia.org/T189065#4045191 should do the trick.

If you'd like we could coordinate a time to apply the greenhouse and DNS changes together.

Hi @herron,

I have a meeting with Lisa in recruiting for 1 pm Pacific on Monday, May

  1. I'll be doing the green house changes with her then. Can we coordinate

for this time or we can reschedule?

Thanks,
Byron

1 pm Pacific on Monday, May 21. I'll be doing the green house changes with her then.

Sounds good! Added a reminder to my calendar.

Change 417350 merged by Herron:
[operations/dns@master] add gh-mail.wikimedia.org (greenhouse.io) spf/mx records

https://gerrit.wikimedia.org/r/417350

SPF and MX records for greenhouse have been configured using the subdomain gh-mail.wikimedia.org and verified through the greenhouse web admin interface.

gh-mail.wikimedia.org was used because the greenhouse system automatically prepends gh-mail to the (sub)domain which prevented us from configuring careers.wikimedia.org

The new records are:

gh-mail.wikimedia.org descriptive text "v=spf1 include:mg-spf.greenhouse.io ~all"
gh-mail.wikimedia.org mail is handled by 10 mxa.mailgun.org.
gh-mail.wikimedia.org mail is handled by 10 mxb.mailgun.org.

Ideally we will use DKIM on this subdomain as well, however DKIM support for gh-mail.wikimedia.org is not yet in place on the greenhouse system.

As I understand, greenhouse support is now being engaged to finalize the config for this subdomain. At which point DKIM details should become available and we can move forward with that portion of the config. @bbogaert please correct me if that's inaccurate.

Thanks!

Following up -- Will we be able to enable DKIM for gh-mail.wikimedia.org?

ArielGlenn added a subscriber: ArielGlenn.

At this point might it be useful to poke the greenhouse support folks again?

herron claimed this task.

Well, looks like no DKIM.