Page MenuHomePhabricator

Outgoing mail to not working on new labs instance
Closed, ResolvedPublic


I created a new lab instance ( to run some cron jobs. I'd like to be able to receive e-mail when those jobs fail. However, outgoing mail appears broken. When I run

$ echo -n "Test" | mail -s Test

The message fails when an error in /var/log/exim4/mainlog:

2016-06-06 22:10:18 1bA2iw-0008Qs-CW "" from env-from rewritten as "" by rule 1
2016-06-06 22:10:18 1bA2iw-0008Qs-CW <= U=runner P=local S=562
2016-06-06 22:10:18 1bA2iw-0008Qs-CW [2620:0:861:3:208:80:154:76] Network is unreachable
2016-06-06 22:10:18 1bA2iw-0008Qs-CW ** R=smart_route T=remote_smtp: SMTP error from remote mail server after RCPT TO:<>: host []: 550-Verification failed for <>\n550-Cannot route to remote domain\n550 Sender verify failed
2016-06-06 22:10:18 1bA2iw-0008Qx-E8 <= <> R=1bA2iw-0008Qs-CW U=Debian-exim P=local S=1653

Event Timeline

yuvipanda renamed this task from Outgoing mail not working on new labs instance to Outgoing mail to not working on new labs instance.Jun 6 2016, 10:42 PM

It seems to work fine for non mail addresses:

2016-06-06 22:26:49 1bA2yv-0000bk-RQ "" from env-from rewritten as "" by
 rule 1
2016-06-06 22:26:49 1bA2yv-0000bk-RQ <= U=root P=local S=585
2016-06-06 22:26:49 1bA2yv-0000bk-RQ [2620:0:861:3:208:80:154:76] Network is unreachable
2016-06-06 22:26:49 1bA2yv-0000bk-RQ => R=smart_route T=remote_smtp S=603 [] C="250 OK id=1bA2yv-0001qK-S1" DT=0s
2016-06-06 22:26:49 1bA2yv-0000bk-RQ Completed
mailx -r ''

works well and delivers it fine.

On 0295b935034c685936f5a79c273696df6bb4521c I mentioned:

On a later step, Labs could get its own mail relays entirely; this split config would allow for this.

For mails in Labs to properly work after this change, should become a mail domain (MX records + a simple alias file defining at least the root alias).

The former is T41785, open for almost 4 years now :(

The latter was never done and, as expected, Labs email is broken. Ideally we'd fix this properly, but we should probably continue the interim hacks as to not wait another e.g. 4 years to unbreak Labs emails…

In short, this behavior essentially happens because does not exist and our (production) mailservers perform sender verification (= reject emails that originate from invalid addresses).

I just set up as a valid email domain and pointed root@ and a few other aliases to the Cloud-Services team (with 5ac81c7150ee6dc0a4375b146adc1c961180513a + a few puppet-private commits). What's left here is to add a few records on the, which is managed by Designate and I'm not entirely sure how to do that — Cc @Andrew.

The records that we need are: 1H IN MX 10 1H IN MX 50 1H IN TXT "v=spf1 mx ?all"

Mentioned in SAL [2016-06-07T00:39:37Z] <Krenair> Created MX and SPF records directly for for

The records that we need are: 1H IN MX 10 1H IN MX 50 1H IN TXT "v=spf1 mx ?all"

Horizon was perfectly happy with those MX records (just, as an admin of the wmflabsdotorg project, create the MX record with an empty 'name' (subdomain) field - just leaving the static '' text)
However, because of this part of the designatedashboard code:
... it wasn't so happy to make the TXT record for us. I ended up having to use python-designateclient (the old v1, because silver) to do it. The code ended up being something like this:

from keystoneclient.auth.identity import generic
from keystoneclient import session as keystone_session
from designateclient import v1
from designateclient.v1.records import Record

auth = generic.Password(

session = keystone_session.Session(auth=auth)
client = v1.Client(session=session)
domain_id =[0]['id'] # 553ef162-add7-4a5c-b115-9cabca662746
record = Record(name="", type="TXT", data="\"v=spf1 mx ?all\"", ttl=3600)
result = client.records.create(domain_id, record)

(I did it once in the python console, so never actually tested this whole block of code together)

This is working now. Thank you!