Other images don't have these commands either:

tools.zhuyifei1999-test@tools-bastion-02:~$ webservice --backend kubernetes python shell
If you don't see a command prompt, try pressing enter.
tools.zhuyifei1999-test@interactive:~$ 
tools.zhuyifei1999-test@interactive:~$ which exim
tools.zhuyifei1999-test@interactive:~$ which mail
tools.zhuyifei1999-test@interactive:~$

However, it is very rare for a tool to send mail afaik, so installing it for every container seems unnecessary, imo. Can you utilize something like mail() or some libraries in PHP?

Yes, should be easy to configure to use a different server to send the mail, but I'm not quite sure what there is available (the PHP mail() function by default sends with whatever the default is for the OS, usually sendmail, so it fails with the same problem). Is there an SMTP account for each tool user? What server do we send with?

Good question. I believe in the case of grid nodes and bastions it is localhost:

root@tools-exec-1401:~# ps ufww -u Debian-exim
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Debian-+  1320  0.0  0.0  50216   676 ?        Ss   May11   0:39 /usr/sbin/exim4 -bd -q10m
root@tools-exec-1401:~# lsof -i -a -p 1320
COMMAND  PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
exim4   1320 Debian-exim    3u  IPv4   8086      0t0  TCP localhost:smtp (LISTEN)
exim4   1320 Debian-exim    4u  IPv6   8087      0t0  TCP ip6-localhost:smtp (LISTEN)

It does get tricky in the case of k8s since these listen on localhost, and containers can't access the host.

FWIW, these are the puppet config for toolforge exim4:
https://github.com/wikimedia/puppet/blob/06e4457944b6ae62584bd8968f94becf8bcba4f4/modules/toollabs/manifests/init.pp#L130
https://github.com/wikimedia/puppet/blob/b347052863d4d2e87b37d6c2d9f44f833cfd9dc2/modules/toollabs/templates/route-to-mail-relay.exim4.conf.erb

Considering tools-mail is an incoming relay rather than outgoing, I have no idea if tools-mail will work.

Is there an smtp service/server available that could be used?

It kind of looks like there might be?
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Nodes#misc_nodes

In T205813#4630214, @dbarratt wrote:

Is there an smtp service/server available that could be used?

I think that mail.tools.wmflabs.org should work as a relay host for sending outbound mail from inside Toolforge. I did this really basic test using telnet and the message ended up in my inbox:

$ ssh tools-bastion-02.tools.eqiad.wmflabs
$ telnet mail.tools.wmflabs.org 25
Trying 10.68.16.27...
Connected to mail.tools.wmflabs.org.
Escape character is '^]'.
220 mail.tools.wmflabs.org ESMTP
HELO tools.wmflabs.org
250 mail.tools.wmflabs.org Hello bd808 at tools-bastion-02.tools.eqiad.wmflabs [10.68.16.44]
mail from: bd808@tools.wmflabs.org
250 OK
rcpt to: bdavis@wikimedia.org
250 Accepted
data
354 Enter message, ending with "." on a line by itself
subject: test of mail.tools.wmflabs.org as relay host

This is a test to see if outbound email works.
Bryan
.
250 OK id=1g7886-0000dN-CZ
quit
221 mail.tools.wmflabs.org closing connection
Connection closed by foreign host.
$

Here's a test from inside a python k8s container as well:

$ webservice --backend=kubernetes python shell
tools.bd808-test2@interactive:~$ python
Python 2.7.9 (default, Jun 29 2016, 13:08:31)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import smtplib
>>> server = smtplib.SMTP('mail.tools.wmflabs.org')
>>> server.set_debuglevel(1)
>>> server.sendmail('bd808@tools.wmflabs.org', 'bdavis@wikimedia.org', '''\
... Subject: testing email from k8s container
...
... This is a test sent using python from inside a Toolforge k8s container.''')
send: 'ehlo [192.168.200.10]\r\n'
reply: '250-mail.tools.wmflabs.org Hello tools-worker-1006.tools.eqiad.wmflabs [10.68.17.89]\r\n'
reply: '250-SIZE 52428800\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-PIPELINING\r\n'
reply: '250 HELP\r\n'
reply: retcode (250); Msg: mail.tools.wmflabs.org Hello tools-worker-1006.tools.eqiad.wmflabs [10.68.17.89]
SIZE 52428800
8BITMIME
PIPELINING
HELP
send: 'mail FROM:<bd808@tools.wmflabs.org> size=114\r\n'
reply: '250 OK\r\n'
reply: retcode (250); Msg: OK
send: 'rcpt TO:<bdavis@wikimedia.org>\r\n'
reply: '250 Accepted\r\n'
reply: retcode (250); Msg: Accepted
send: 'data\r\n'
reply: '354 Enter message, ending with "." on a line by itself\r\n'
reply: retcode (354); Msg: Enter message, ending with "." on a line by itself
data: (354, 'Enter message, ending with "." on a line by itself')
send: 'Subject: testing email from k8s container\r\n\r\nThis is a test sent using python from inside a Toolforge k8s container.\r\n.\r\n'
reply: '250 OK id=1g79Jo-0002zd-Qm\r\n'
reply: retcode (250); Msg: OK id=1g79Jo-0002zd-Qm
data: (250, 'OK id=1g79Jo-0002zd-Qm')
{}
>>>

I have updated the main issue summary to mark this as a Documentation issue.

Thanks @bd808.

I've had a go at updating https://wikitech.wikimedia.org/wiki/Help:Toolforge#Mail_from_Tools see what you think.

And my testing of GrantMetrics has successfully sent mails.

Garbage. The change to the documentation does not explain how to send an email if there is no sendmail, no exim, no MIME::Lite.

Revert the documentation and fix the problem.

In T205813#8319178, @Hawkeye7 wrote:

Garbage. The change to the documentation does not explain how to send an email if there is no sendmail, no exim, no MIME::Lite.

Revert the documentation and fix the problem.

Use whatever language you wants mail function. PHP, Python, etc all have functions for sending mail very easily using an external host. You can use mail.tools.wmflabs.org

In T205813#8319188, @RhinosF1 wrote:

In T205813#8319178, @Hawkeye7 wrote:

Garbage. The change to the documentation does not explain how to send an email if there is no sendmail, no exim, no MIME::Lite.

Revert the documentation and fix the problem.

Use whatever language you wants mail function. PHP, Python, etc all have functions for sending mail very easily using an external host. You can use mail.tools.wmflabs.org

I commented on your migration ticket with the correct package to request for the Perl image.

Perl has a mail module, MIME::Lite, but while it is available on the Grid, it has not been provided in the Kubernetes image.

In T205813#8319500, @Hawkeye7 wrote:

Perl has a mail module, MIME::Lite, but while it is available on the Grid, it has not been provided in the Kubernetes image.

I am aware. I have commented on your other task with the correct package that needs to be enabled.

I do have a workaround: I could use Net::SMTP. This is a low-level module, but it is included with a standard Perl 6.32 install.I have tested this with the tf-perl532 image. Using it would require me to write an new email wrapper for my bots to use. Currently my wrapper supports sendmail, exim and MIME::Lite.