Page MenuHomePhabricator

Fix incorrect mailing list activity of AKlapper (=Phabricator) in Technical Community Metrics user data
Closed, ResolvedPublic

Description

http://korma.wmflabs.org/browser/mls.html states that Andre Klapper is the most active mailing list participant in the last 365 days with 1066 messages, and the 2nd most active all-time participant.
I have a hard time believing that.

My guess was that these high numbers were due to emails triggered by Andre Klapper's activity in issue trackers (Phabricator; Bugzilla until Nov 2014) but looking at the currently indexed lists at https://github.com/Bitergia/mediawiki-repositories/blob/master/mlstats_mailing_lists.conf I do not see any of the four "bug notification only" mailing lists:

  • wikibugs-l became inactive in Nov 2014 due to the switch from Bugzilla to Phabricator
  • pywikibot-bugs not listed (but see below)
  • wikidata-bugs not listed
  • tools-nlwikibots not listed and very low traffic.

Looking at http://korma.wmflabs.org/browser/people.html?id=360cfd7aca56c90c4cc6ad653bb5b263db9b62f7 under "Mailing lists" it states that

  • 1) Andre Klapper sent 764 emails in May 2015
  • 2) Andre Klapper sent 1055 emails in Dec 2014.

Looking at the previous version at https://github.com/Bitergia/mediawiki-repositories/commit/7a201aea22b00e664876b8f093da23abb131efe7 I see that http://lists.wikimedia.org/pipermail/pywikipedia-bugs was indexed until Nov 2015 which is the only mailing list receiving bugmail from Phabricator.
My assumption is not covered by

However,

  • 2) https://lists.wikimedia.org/pipermail/pywikibot-bugs/2014-November/thread.html lists 638 Phabricator emails by aklapper, triggered by fixing T1282. If this was related, stats would be off by one month, but I could not find similar Phabricator mass-editing activity on Pywikibot tasks in the months around May 2015.
  • 1) Going to my mail account's "Sent" folder (which obviously also has any private, non-mailing list emails in there), I seem to have sent less than 100 emails in May 2015 (I was on holidays half of that month).

So I assume the numbers in korma are incorrect, and I'd welcome some investigation.

Event Timeline

Aklapper updated the task description. (Show Details)Apr 18 2016, 10:56 AM

This task welcomes an owner.

We're investigating it.

Aklapper claimed this task.Nov 23 2016, 8:05 PM

@Lcanasdiaz: Nothing to investigate anymore for you. I just need to fix my own entry in the mediawiki-identities DB.

Longer version:

After an hour of trying to understand


and being baffled about Email senders list showing 1 entry vs Email senders bars showing several entries (that might be the only thing here left to investigate for Bitergia but let's please wait with that), I realized that

  • Phabricator sets $UserNameInPhabricator <no-reply@phabricator.wikimedia.org> as the sender in emails, and Bitergia code obviously (and correctly) uses the email address as the unique ID for mls identities.
  • At some point, Bitergia code parsed and indexed its first date for that email address. That identity ended up as mine in the DB for reasons that don't matter (might have been code, might have been a human mistake when merging).
  • T146632: Kibana's Mailing List data sources do not include recent activity on wikitech-l mailing list also influenced my confusion.

Solution:

sortinghat mv 91a509e8460453fff991f76e2ab03ec66a140495 91a509e8460453fff991f76e2ab03ec66a140495
sortinghat profile --bot 91a509e8460453fff991f76e2ab03ec66a140495
sortinghat profile --name "Phabricator email notifications to mailing lists" 91a509e8460453fff991f76e2ab03ec66a140495

Aklapper renamed this task from Check whether mailing list activity per person on korma is in sync with current "mlstats_mailing_lists.conf" to Fix incorrect mailing list activity of AKlapper (=Phabricator) in Technical Community Metrics user data.Nov 23 2016, 8:21 PM
Aklapper added a comment.EditedNov 25 2016, 2:29 PM

Verified:
On 2016-11-23, http://korma.wmflabs.org/browser/top-contributors.html showed Andre Klapper on #47 with #2 about "Mailing Lists".
On 2016-11-25, http://korma.wmflabs.org/browser/top-contributors.html showed Andre Klapper on #47 with #62 about "Mailing Lists".

Oh, wait:
Korma seems correct now. Kibana not yet because https://wikimedia.biterg.io:443/goto/fea0b0637c3984ac5b27ff894232cb63 looks still like in the T132907#2818735 screenshot... Going to keep an eye on this.

Aklapper reopened this task as Open.Dec 21 2016, 6:20 AM

Reopening as it's still an issue in Kibana.

https://wikimedia.biterg.io/goto/fea0b0637c3984ac5b27ff894232cb63 should not show "Andre Klapper" as the sole "email senders".

Aklapper removed Aklapper as the assignee of this task.Jan 4 2017, 10:08 PM
Aklapper closed this task as Resolved.Feb 28 2017, 2:51 PM
Aklapper claimed this task.

This specific issue is not a problem anymore (maybe fixed by T146632: Kibana's Mailing List data sources do not include recent activity on wikitech-l mailing list hence resolving).

However there is a different problem now that wikimedia.biterg.io states that there have been email senders but the "Email Senders" panel is empty. I have filed T159229: "Email senders" widget empty though "Mailing Lists" widget states that there are email senders for that.