Page MenuHomePhabricator

Valid email address was unconfirmed for temporary spam blacklisting
Open, Needs TriagePublic

Description

In the last few days I noticed that I didn't get email notifications for user talk messages on my "Nemo bis" account. https://it.wikipedia.org/wiki/Speciale:Preferenze shows my email address was unconfirmed.

The last email notification I received for this account seems to have been on 2015-05-07 03.09 UTC. I suspect this has to do with the large swath of user talk messages I received (over 100 messages in about a day): http://vs.aka-online.de/cgi-bin/wppagehiststat.pl?lang=it.wikipedia&page=Discussioni_utente:Nemo_bis

IIRC in the last couple weeks my ISP tiscali.it did not have issues (usually visible from delays in dellivery) and the missing messages are not in spam mail. However I notice some of them (listed below) did end up in spam, I wonder if antispam protection caused bounces for the others.

Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:56
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:31
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:16
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:11
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 17:00
MediaWiki message delivery ti ha lasciato un mess Wikizionario <wiki@wikimedia.org> 19-04-2015 11:10
9af9af ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 26-04-2015 21:34
9af9af ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 26-04-2015 21:33
MediaWiki message delivery ti ha lasciato un mess Wikiversit  <wiki@wikimedia.org> 26-04-2015 14:24
MediaWiki message delivery ti ha lasciato un mess Wikiversit  <wiki@wikimedia.org> 26-04-2015 14:22
MediaWiki message delivery ti ha lasciato un mess MediaWiki <wiki@wikimedia.org> 26-04-2015 14:18
Starter27 ti ha lasciato un messaggio in Wikipedi Wikipedia <wiki@wikimedia.org> 26-04-2015 12:03
Never covered ti ha lasciato un messaggio in Wiki Wikipedia <wiki@wikimedia.org> 26-04-2015 10:39
Starter27 ti ha lasciato un messaggio in Wikipedi Wikipedia <wiki@wikimedia.org> 26-04-2015 10:27
Retaggio ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 27-04-2015 9:51
Giomayo ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 30-04-2015 15:51
Ettorre ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 03-05-2015 17:37
Reyon ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 05-05-2015 15:14

Related Objects

StatusAssignedTask
OpenNone
ResolvedReedy
InvalidNone
ResolvedLegoktm
OpenNone
Resolved01tonythomas
Resolved01tonythomas
Resolved01tonythomas
Resolvedcsteipp
Resolved01tonythomas
Resolved01tonythomas
InvalidNone
ResolvedNone
Resolved01tonythomas
Resolvedaaron
Resolved01tonythomas
ResolvedNone
Declined01tonythomas
Resolved01tonythomas
OpenNone
ResolvedNone
OpenNone
Resolvedherron

Event Timeline

Nemo_bis created this task.May 17 2015, 2:41 PM
Nemo_bis raised the priority of this task from to Needs Triage.
Nemo_bis updated the task description. (Show Details)
Nemo_bis added a subscriber: Nemo_bis.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 17 2015, 2:41 PM
Nemo_bis added a comment.EditedMay 17 2015, 4:15 PM

I see X-CNFS-Analysis headers so in the meanwhile I reported the issue to https://www.cloudmark.com/en/support/cloudmark-authority (case
00231264) but of course that's only part of the problem.

hoo added a subscriber: hoo.EditedMay 17 2015, 5:47 PM

Logs about this (email address replaced by ):

mysql:wikiadmin@db1029 [wikishared]> SELECT br_id, br_timestamp, br_reason FROM bounce_records WHERE br_user_email = '…';
+--------+----------------+---------------------------------------------------+
| br_id  | br_timestamp   | br_reason                                         |
+--------+----------------+---------------------------------------------------+
| 212847 | 20150504095359 | Mail delivery failed: returning message to sender |
| 212851 | 20150504150932 | Mail delivery failed: returning message to sender |
| 213222 | 20150507084307 | Mail delivery failed: returning message to sender |
| 213219 | 20150507102343 | Mail delivery failed: returning message to sender |
| 213224 | 20150507114738 | Mail delivery failed: returning message to sender |
| 213220 | 20150507155326 | Mail delivery failed: returning message to sender |
| 213236 | 20150507183107 | Mail delivery failed: returning message to sender |
+--------+----------------+---------------------------------------------------+
7 rows in set (0.00 sec)
archive/BounceHandler.log-20150508.gz:2015-05-07 15:57:09 mw1011 mediawikiwiki BounceHandler INFO: Un-subscribed global user … for exceeding Bounce Limit 5 {"private":false}
archive/BounceHandler.log-20150508.gz:2015-05-07 15:57:53 mw1011 mediawikiwiki BounceHandler INFO: Un-subscribed global user … for exceeding Bounce Limit 5 {"private":false}
archive/BounceHandler.log-20150508.gz:2015-05-07 18:31:10 mw1011 foundationwiki BounceHandler INFO: Un-subscribed … for exceeding Bounce limit 5 {"private":false}

Hmm. We are planning to include the body+headers of the bounce email too in the 'bouncehandler' logs so that a deeper study of special cases like this is possible. This would've happened due to the bouncehandler taking SPAM bounce backs as a permanent bounce. If this should be skipped, we need to study what bounce email was created and design the handler in a way it skips this one.

Well, what standard responses are there to say "please slow down messages to this address"? (I don't know one.) If the bounces were uninformative and seemingly permanent, then unsubscribing the address was the right thing to do, but maybe a "suspension" system would be needed too.

In the meanwhile, as for the sub-issue of spam blocking, I already got a fix from the spam authority: amazing support! (Quoted with permission.)

Thank you for contacting Cloudmark.
That fingerprint is for the content of messages that were reported spammy by end users.
Looking at the feedback I see:

  • 85% of reports seen were to block
  • 0% of blocks were from spam traps
  • First report : 2015-03-17 23:39:06
  • Last report : 2015-05-17 13:18:06

I have reset it, you should be good to go now.

01tonythomas set Security to None.May 18 2015, 11:46 AM
01tonythomas added a subscriber: Jgreen.

Well, what standard responses are there to say "please slow down messages to this address"? (I don't know one.)

Currently, we consider a bounce as 'Permanent Bounce' if:

  1. If we find an 'X-Failed-Recipient' in the bounce-header
  2. After https://gerrit.wikimedia.org/r/#/c/177456/, we can identify a bounce email which is following RFC3464 strictly ( for status code parsing ) - and if not found - we even do a heuristic way to figure it out ( further scan through the body )
  3. If not detected in the above steps - we stamp the bounce as temporary/fake.

I think http://www.heise.de/netze/rfc/rfcs/rfc3464.shtml#page-18 is followed closely enough but not all the headers etc. Fetching the 3 or 6 digits code should be feasible.

Reedy added a subscriber: Reedy.Nov 15 2015, 7:38 PM

We should be able to dig into this a bit more now...

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 15 2015, 7:38 PM
Nemo_bis renamed this task from Valid email address was unconfirmed: too many enotifs? to Valid email address was unconfirmed for temporary spam blacklisting.Jun 27 2017, 1:54 PM

Maybe this should be closed invalid, especially if there's already some other task on how to handle temporary failures. Mailservers often lie about the reasons they bounce or defer messages, nowadays (to avoid scraping etc.).