Page MenuHomePhabricator

Document mail relay service which can be used to send outbound email from Toolforge Kubernetes containers
Closed, ResolvedPublic

Description

Containers running on the Toolforge Kubernetes cluster do not install and configure a local mailer service like the exim service that is installed on grid engine nodes. Tools running in Kubernetes should instead send email using an external SMTP server. The mail.tools.wmflabs.org service name should be usable for this. This service name is used as the public MX (mail exchange) host for inbound SMTP messages to the tools.wmflabs.org domain and should always point to a server that can process both inbound and outbound email for Toolforge.


Original bug report:

Currently, it seems that it's not possible to send email from PHP (with either the php5.6 or php7.2 container). We only want to be able to send to the tool maintainers' email address (for T204860).

Following the instructions for mailing from tools gets the error "/usr/sbin/exim: No such file or directory".

This might already be possible, in which case I'd love someone to tell me how to do it. Thanks!

(This is a follow-up from T188318#4628389.)

Event Timeline

Samwilson created this task.Oct 1 2018, 5:17 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2018, 5:17 AM
Legoktm renamed this task from Make it possible to send mail from PHP tools to Make it possible to send mail from PHP webservices running on kubernetes.Oct 1 2018, 5:20 AM

Other images don't have these commands either:

tools.zhuyifei1999-test@tools-bastion-02:~$ webservice --backend kubernetes python shell
If you don't see a command prompt, try pressing enter.
tools.zhuyifei1999-test@interactive:~$ 
tools.zhuyifei1999-test@interactive:~$ which exim
tools.zhuyifei1999-test@interactive:~$ which mail
tools.zhuyifei1999-test@interactive:~$

However, it is very rare for a tool to send mail afaik, so installing it for every container seems unnecessary, imo. Can you utilize something like mail() or some libraries in PHP?

Yes, should be easy to configure to use a different server to send the mail, but I'm not quite sure what there is available (the PHP mail() function by default sends with whatever the default is for the OS, usually sendmail, so it fails with the same problem). Is there an SMTP account for each tool user? What server do we send with?

Good question. I believe in the case of grid nodes and bastions it is localhost:

root@tools-exec-1401:~# ps ufww -u Debian-exim
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Debian-+  1320  0.0  0.0  50216   676 ?        Ss   May11   0:39 /usr/sbin/exim4 -bd -q10m
root@tools-exec-1401:~# lsof -i -a -p 1320
COMMAND  PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
exim4   1320 Debian-exim    3u  IPv4   8086      0t0  TCP localhost:smtp (LISTEN)
exim4   1320 Debian-exim    4u  IPv6   8087      0t0  TCP ip6-localhost:smtp (LISTEN)

It does get tricky in the case of k8s since these listen on localhost, and containers can't access the host.

FWIW, these are the puppet config for toolforge exim4:
https://github.com/wikimedia/puppet/blob/06e4457944b6ae62584bd8968f94becf8bcba4f4/modules/toollabs/manifests/init.pp#L130
https://github.com/wikimedia/puppet/blob/b347052863d4d2e87b37d6c2d9f44f833cfd9dc2/modules/toollabs/templates/route-to-mail-relay.exim4.conf.erb

Considering tools-mail is an incoming relay rather than outgoing, I have no idea if tools-mail will work.

Is there an smtp service/server available that could be used?

bd808 added a subscriber: bd808.Oct 2 2018, 12:05 AM

Is there an smtp service/server available that could be used?

I think that mail.tools.wmflabs.org should work as a relay host for sending outbound mail from inside Toolforge. I did this really basic test using telnet and the message ended up in my inbox:

$ ssh tools-bastion-02.tools.eqiad.wmflabs
$ telnet mail.tools.wmflabs.org 25
Trying 10.68.16.27...
Connected to mail.tools.wmflabs.org.
Escape character is '^]'.
220 mail.tools.wmflabs.org ESMTP
HELO tools.wmflabs.org
250 mail.tools.wmflabs.org Hello bd808 at tools-bastion-02.tools.eqiad.wmflabs [10.68.16.44]
mail from: bd808@tools.wmflabs.org
250 OK
rcpt to: bdavis@wikimedia.org
250 Accepted
data
354 Enter message, ending with "." on a line by itself
subject: test of mail.tools.wmflabs.org as relay host

This is a test to see if outbound email works.
Bryan
.
250 OK id=1g7886-0000dN-CZ
quit
221 mail.tools.wmflabs.org closing connection
Connection closed by foreign host.
$
bd808 renamed this task from Make it possible to send mail from PHP webservices running on kubernetes to Document mail relay service which can be used to send outbound email from Toolforge Kubernetes containers.Oct 2 2018, 1:19 AM
bd808 triaged this task as Normal priority.
bd808 added a project: Documentation.
bd808 updated the task description. (Show Details)
bd808 added a comment.Oct 2 2018, 1:22 AM

Here's a test from inside a python k8s container as well:

$ webservice --backend=kubernetes python shell
tools.bd808-test2@interactive:~$ python
Python 2.7.9 (default, Jun 29 2016, 13:08:31)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import smtplib
>>> server = smtplib.SMTP('mail.tools.wmflabs.org')
>>> server.set_debuglevel(1)
>>> server.sendmail('bd808@tools.wmflabs.org', 'bdavis@wikimedia.org', '''\
... Subject: testing email from k8s container
...
... This is a test sent using python from inside a Toolforge k8s container.''')
send: 'ehlo [192.168.200.10]\r\n'
reply: '250-mail.tools.wmflabs.org Hello tools-worker-1006.tools.eqiad.wmflabs [10.68.17.89]\r\n'
reply: '250-SIZE 52428800\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-PIPELINING\r\n'
reply: '250 HELP\r\n'
reply: retcode (250); Msg: mail.tools.wmflabs.org Hello tools-worker-1006.tools.eqiad.wmflabs [10.68.17.89]
SIZE 52428800
8BITMIME
PIPELINING
HELP
send: 'mail FROM:<bd808@tools.wmflabs.org> size=114\r\n'
reply: '250 OK\r\n'
reply: retcode (250); Msg: OK
send: 'rcpt TO:<bdavis@wikimedia.org>\r\n'
reply: '250 Accepted\r\n'
reply: retcode (250); Msg: Accepted
send: 'data\r\n'
reply: '354 Enter message, ending with "." on a line by itself\r\n'
reply: retcode (354); Msg: Enter message, ending with "." on a line by itself
data: (354, 'Enter message, ending with "." on a line by itself')
send: 'Subject: testing email from k8s container\r\n\r\nThis is a test sent using python from inside a Toolforge k8s container.\r\n.\r\n'
reply: '250 OK id=1g79Jo-0002zd-Qm\r\n'
reply: retcode (250); Msg: OK id=1g79Jo-0002zd-Qm
data: (250, 'OK id=1g79Jo-0002zd-Qm')
{}
>>>

I have updated the main issue summary to mark this as a Documentation issue.

Thanks @bd808.

I've had a go at updating https://wikitech.wikimedia.org/wiki/Help:Toolforge#Mail_from_Tools see what you think.

And my testing of GrantMetrics has successfully sent mails.

herron closed this task as Resolved.Nov 9 2018, 9:54 PM
herron added a subscriber: herron.

This looks to be resolved but please re-open if not!