Page MenuHomePhabricator

"Phabricator monthly statistics" email on wikitech-l@ missing for May 2019
Closed, ResolvedPublic

Description

I just noticed that we didn't get the monthly phabricator stats email and I was wondering if it can be related to the migration from phab1001 to phab1003, as I reckon it used to run from phab1001?

Event Timeline

Most likely a cron job that was manually created? I don't think I have access to phab1001 anymore but I'll see if I can find the script.

This is 1003:

root@phab1003:/var/spool# egrep -iR "stats|metr" *
cron/crontabs/root:# Puppet Name: phabstatscron_communitymetrics
cron/crontabs/root:0 0 1 * * /usr/local/bin/community_metrics.sh

Looks like there is one. But this is the problem:

root@phab1003:/var/spool# echo "test" | mail -s "test" marostegui@xxxx
-bash: mail: command not found

Looks like we don't have the mailutils installed there and the script uses it:

# the actual email
cat <<EOF | /usr/bin/mail -r "${sndr_address}" -s "Phabricator monthly statistics - ${lastmonth}" ${rcpt_address}
Aklapper renamed this task from Phabricator May's stats missing to "Phabricator monthly statistics" email on wikitech-l@ missing for May 2019.Jun 1 2019, 8:51 PM
ArielGlenn triaged this task as Medium priority.Jun 10 2019, 7:38 AM

I see that @MoritzMuehlenhoff installed bsd-mailx manually on the box to fix future runs; this should be puppetized.

Change 516131 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] phabricator logmail requires /usr/bin/mail be installed

https://gerrit.wikimedia.org/r/516131

Change 516131 merged by ArielGlenn:
[operations/puppet@production] phabricator logmail requires /usr/bin/mail be installed

https://gerrit.wikimedia.org/r/516131

I'll leave this ticket open until we see that the next month's report has shown up.

Would it be possible to run that script once manually within this month, to get the stats for May 2019? Otherwise we'll never get them. Thanks for considering.

I just ran it, but I think it gave the delta between today and 1 month ago as most of the queries are:

SELECT COUNT(DISTINCT authorPHID) FROM maniphest_transaction WHERE
    FROM_UNIXTIME(dateCreated,'%Y%m')=date_format(NOW() - INTERVAL 1 MONTH,'%Y%m');

So if we want the ones from may we need to modify all the queries on that script to make them to pick the right range, not sure if it is worth the time?

I could probably hack the script to let one optionally specify start date and end date, is it worth it though?

No need to change any code, the script automatically "looks up" the entire previous calendar month via FROM_UNIXTIME(dateCreated,'%Y%m')=date_format(NOW() - INTERVAL 1 MONTH,'%Y%m') :)

Argh. I should get my first coffee before posting I guess, because date_format(NOW()), indeed. Sorry.

Regardless of the query...the email hasn't arrived yet and the script didn't show any errors. So probably some debugging is needed to check what the email is doing //cc @herron
From the exim logs I see the email being sent correctly though.
I have tested sending an email to myself from the CLI and that arrived correctly, however the script doesn't seem to be working correctly as the email isn't arriving to either wikitech-l or myself (I modified the script to send it to me).
Exim still marks it as correctly being sent on logs.

The email has arrived to us for may now @Marostegui

Yeah, it got some delay:

Wed, Jun 19, 2019 at 10:27 AM (Delivered after 1752 seconds)
Marostegui assigned this task to ArielGlenn.

Looks fixed then