jmail can be used in ~/.forward to process incoming mail (cf. https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Processing_email_programatically). It is currently not puppetized. The source is on tools-mail in /usr/local/bin/jmail.
Description
Details
Subject | Repo | Branch | Lines +/- | |
---|---|---|---|---|
Package jmail | labs/toollabs | ubuntu/precise | +124 -7 | |
Package jmail | labs/toollabs | master | +124 -7 |
Related Objects
- Mentioned In
- T94792: Remove support for precise OGE exec hosts
Event Timeline
Current script:
#! /bin/bash # # Copyright © 2013 Marc-André Pelletier <mpelletier@wikimedia.org> # # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # # THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES # WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF # MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR # ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES # WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN # ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF # OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. # cat >>/tmp/jmail.log <<EOF $(date): cmdline: $@ $(id) ---- EOF email="/data/project/.system/store/mail/.deliver.$(date +%s).$$" prog="$1" shift cd $HOME exe=$(PATH="$PATH:$HOME" /usr/bin/which "$prog") if [ ! -x "$exe" ]; then echo "Processing program unavailable" >&2 exit 1 fi trap "/bin/rm -f '$email' '$email.out'" 0 if ! cat >$email 2>/dev/null; then exit 1 fi /usr/bin/qsub -N mail.$(/usr/bin/id -nu) \ -sync y -b y -m n \ -o "$email.out" -j y -i "$email" \ -q mailq -l h_vmem=500M -r n \ "$exe" "$@" >/dev/null rv=$? cat "$email.out" exit $rv
Actually, there seems to be a bug in that script: The first run of cat >>/tmp/jmail.log after a reboot creates /tmp/jmail.log being owned by that particular user (mostly drtrigonbot) and subsequent invocations by other users will fail. But:
scfc@tools-mail:~$ sudo find -H /data/project -mindepth 2 -maxdepth 2 -type f -name .forward\* -exec fgrep jmail {} + /data/project/tsreports-dev/.forward:|jmail /data/project/tsreports-dev/test.py /data/project/csbot/.forward.test:|jmail mailtest /data/project/drtrigonbot/.forward.subster:|jmail cat >> ~/data/subster/mail_inbox /data/project/bd808-test/.forward.jmail:|jmail tee -a /data/project/bd808-test/jmail.spool scfc@tools-mail:~$
so the chance is pretty slim that someone would have noticed :-).
I believe the part about email="/data/project/.system/store/mail/.deliver.$(date +%s).$$" should also be replaced by files created by mktemp in the tool's home directory. I wasn't aware of this use of /data/project/.system/store/mail (https://gerrit.wikimedia.org/r/#/c/326306/ as proof :-)) and I think per-tool storage is a better idea. Also I would only remove the files on success. There seem to be some confusion on which signals trap something 0 is executed by which shells.
It would be swell to have this rewritten in python and added to the labs/toollabs.git repo as part of the jobutils deb.
Email as an API seems like an idea fraught with peril to me. Can we remove this entirely? What's the must-have use case here?
jmail is documented on wiki: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Processing_email_programatically
I don't know if this was something carried over from Toolserver or a local invention.
(I disagree that mail APIs are worse than others; in fact IMHO the inherent queue system saves much trouble. Being able to use Phabricator or Debian Bugs asynchronously is priceless.)
It looks as if drtrigonbot is the only user/tool that uses jmail for non-testing purposes. @DrTrigon, do you actively use that?
I have put the rewrite at /usr/local/bin/jmail.new and set a symbolic link from /usr/local/bin/jmail. I ran several tests and I'm confident that the patch works.
Change 340276 had a related patch set uploaded (by Tim Landscheidt; owner: Tim Landscheidt):
Package jmail
Mentioned in SAL (#wikimedia-labs) [2017-02-28T03:52:27Z] <scfc_de> Deployed jobtools and misctools 1.20/1.20~precise+1 (T158722).
I removed the now-obsolete files under /usr/local/bin and tested successfully /usr/bin/jmail with echo Test 15 | mail -s Test tools.scfc-test-can-be-deleted-anytime (as seen above, no existing ~/.forward referred to /usr/local/bin/jmail by its absolute path).