Page MenuHomePhabricator
Create Task
Maniphest T99444

Valid email address was unconfirmed for temporary spam blacklisting
Closed, InvalidPublic
Actions

Description

In the last few days I noticed that I didn't get email notifications for user talk messages on my "Nemo bis" account. https://it.wikipedia.org/wiki/Speciale:Preferenze shows my email address was unconfirmed.

The last email notification I received for this account seems to have been on 2015-05-07 03.09 UTC. I suspect this has to do with the large swath of user talk messages I received (over 100 messages in about a day): http://vs.aka-online.de/cgi-bin/wppagehiststat.pl?lang=it.wikipedia&page=Discussioni_utente:Nemo_bis

IIRC in the last couple weeks my ISP tiscali.it did not have issues (usually visible from delays in dellivery) and the missing messages are not in spam mail. However I notice some of them (listed below) did end up in spam, I wonder if antispam protection caused bounces for the others.

Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:56
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:31
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:16
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 18:11
Alexmar983 ti ha lasciato un messaggio in Wikiquo Wikiquote <wiki@wikimedia.org> 20-04-2015 17:00
MediaWiki message delivery ti ha lasciato un mess Wikizionario <wiki@wikimedia.org> 19-04-2015 11:10
9af9af ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 26-04-2015 21:34
9af9af ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 26-04-2015 21:33
MediaWiki message delivery ti ha lasciato un mess Wikiversit  <wiki@wikimedia.org> 26-04-2015 14:24
MediaWiki message delivery ti ha lasciato un mess Wikiversit  <wiki@wikimedia.org> 26-04-2015 14:22
MediaWiki message delivery ti ha lasciato un mess MediaWiki <wiki@wikimedia.org> 26-04-2015 14:18
Starter27 ti ha lasciato un messaggio in Wikipedi Wikipedia <wiki@wikimedia.org> 26-04-2015 12:03
Never covered ti ha lasciato un messaggio in Wiki Wikipedia <wiki@wikimedia.org> 26-04-2015 10:39
Starter27 ti ha lasciato un messaggio in Wikipedi Wikipedia <wiki@wikimedia.org> 26-04-2015 10:27
Retaggio ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 27-04-2015 9:51
Giomayo ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 30-04-2015 15:51
Ettorre ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 03-05-2015 17:37
Reyon ti ha lasciato un messaggio in Wikipedia Wikipedia <wiki@wikimedia.org> 05-05-2015 15:14

Related Objects
Search...

StatusSubtypeAssignedTask
 InvalidNoneT99444 Valid email address was unconfirmed for temporary spam blacklisting
 ResolvedReedyT99767 Log full bounce email too before unsubscribing
 InvalidNoneT54915 HTML email notifications caught by spam filters
 ResolvedLegoktmT58315 Set List-Unsubscribe header for email notifications (and maybe List-Help, List-Subscribe etc.)
 Resolved• chasempT58414 Get mail relay out of Yahoo! blacklist: apply to Yahoo for whitelisting bulk mail
 Resolved01tonythomasT48640 Add proper email bounce handling to MediaWiki (with VERP)
 Resolved01tonythomasT71019 Review and deploy BounceHandler extension to Wikimedia wikis
 Resolved01tonythomasT74683 Rename br_user column
 Resolved• csteippT71099 Security review BounceHandler extension for deployement
 Resolved01tonythomasT74686 BounceHandlerActions::handleFailingRecipient() SQL query
 Resolved01tonythomasT74684 Inconsistent returns in BounceHandler
 InvalidNoneT71098 Product review BounceHandler extension for deployement
 ResolvedNoneT74685 ApiBounceHandler improvements
 Resolved01tonythomasT71101 Architecture review for BounceHandler extension
 ResolvedaaronT71100 Performance review BounceHandler extension for Deployment
 Resolved01tonythomasT75083 Use MW style timestamps in the bounce_records table
 ResolvedNoneT71621 Deploy BounceHandler extension to beta cluster
 Declined01tonythomasT92876 Deploy BounceHandler to group1 wikis (non-wikipedias)
 Resolved01tonythomasT92877 Deploy BounceHandler everywhere (adding in wikipedias)
 OpenNoneT64704 Account creation by email often winds up marked as spam
 ResolvedNoneT58413 DomainKeys Identified Mail (DKIM) for wikipedia.org (and other projects)
 InvalidNoneT106664 Set up role accounts and feedback loops (FBL) with all providers
 ResolvedherronT168467 Improve lists.wikimedia.org DMARC compatibility

Event Timeline

Nemo_bis created this task.May 17 2015, 2:41 PM
Nemo_bis raised the priority of this task from to Needs Triage.
Nemo_bis updated the task description. (Show Details)
Nemo_bis added projects: MediaWiki-extensions-BounceHandler, Mail.
Nemo_bis subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 17 2015, 2:41 PM
Nemo_bis added a comment.EditedMay 17 2015, 4:15 PM

I see X-CNFS-Analysis headers so in the meanwhile I reported the issue to https://www.cloudmark.com/en/support/cloudmark-authority (case
00231264) but of course that's only part of the problem.

hoo subscribed.EditedMay 17 2015, 5:47 PM

Logs about this (email address replaced by …):

mysql:wikiadmin@db1029 [wikishared]> SELECT br_id, br_timestamp, br_reason FROM bounce_records WHERE br_user_email = '…';
+--------+----------------+---------------------------------------------------+
| br_id  | br_timestamp   | br_reason                                         |
+--------+----------------+---------------------------------------------------+
| 212847 | 20150504095359 | Mail delivery failed: returning message to sender |
| 212851 | 20150504150932 | Mail delivery failed: returning message to sender |
| 213222 | 20150507084307 | Mail delivery failed: returning message to sender |
| 213219 | 20150507102343 | Mail delivery failed: returning message to sender |
| 213224 | 20150507114738 | Mail delivery failed: returning message to sender |
| 213220 | 20150507155326 | Mail delivery failed: returning message to sender |
| 213236 | 20150507183107 | Mail delivery failed: returning message to sender |
+--------+----------------+---------------------------------------------------+
7 rows in set (0.00 sec)
archive/BounceHandler.log-20150508.gz:2015-05-07 15:57:09 mw1011 mediawikiwiki BounceHandler INFO: Un-subscribed global user … for exceeding Bounce Limit 5 {"private":false}
archive/BounceHandler.log-20150508.gz:2015-05-07 15:57:53 mw1011 mediawikiwiki BounceHandler INFO: Un-subscribed global user … for exceeding Bounce Limit 5 {"private":false}
archive/BounceHandler.log-20150508.gz:2015-05-07 18:31:10 mw1011 foundationwiki BounceHandler INFO: Un-subscribed … for exceeding Bounce limit 5 {"private":false}
01tonythomas subscribed.May 17 2015, 6:12 PM

Hmm. We are planning to include the body+headers of the bounce email too in the 'bouncehandler' logs so that a deeper study of special cases like this is possible. This would've happened due to the bouncehandler taking SPAM bounce backs as a permanent bounce. If this should be skipped, we need to study what bounce email was created and design the handler in a way it skips this one.

Nemo_bis added a comment.May 17 2015, 9:59 PM

Well, what standard responses are there to say "please slow down messages to this address"? (I don't know one.) If the bounces were uninformative and seemingly permanent, then unsubscribing the address was the right thing to do, but maybe a "suspension" system would be needed too.

In the meanwhile, as for the sub-issue of spam blocking, I already got a fix from the spam authority: amazing support! (Quoted with permission.)

Thank you for contacting Cloudmark.

That fingerprint is for the content of messages that were reported spammy by end users.

Looking at the feedback I see:

  • 85% of reports seen were to block
  • 0% of blocks were from spam traps
  • First report : 2015-03-17 23:39:06
  • Last report : 2015-05-17 13:18:06

I have reset it, you should be good to go now.

01tonythomas set Security to None.May 18 2015, 11:46 AM
01tonythomas added a subscriber: Jgreen.
01tonythomas mentioned this in T99767: Log full bounce email too before unsubscribing.May 20 2015, 7:21 AM
01tonythomas added a subtask: T99767: Log full bounce email too before unsubscribing.
01tonythomas added a comment.May 20 2015, 7:38 AM
In T99444#1292318, @Nemo_bis wrote:

Well, what standard responses are there to say "please slow down messages to this address"? (I don't know one.)

Currently, we consider a bounce as 'Permanent Bounce' if:

  1. If we find an 'X-Failed-Recipient' in the bounce-header
  2. After https://gerrit.wikimedia.org/r/#/c/177456/, we can identify a bounce email which is following RFC3464 strictly ( for status code parsing ) - and if not found - we even do a heuristic way to figure it out ( further scan through the body )
  3. If not detected in the above steps - we stamp the bounce as temporary/fake.
Nemo_bis mentioned this in T54915: HTML email notifications caught by spam filters.May 23 2015, 10:41 AM
Nemo_bis added a subtask: T54915: HTML email notifications caught by spam filters.
Nemo_bis added a comment.May 23 2015, 1:06 PM

I think http://www.heise.de/netze/rfc/rfcs/rfc3464.shtml#page-18 is followed closely enough but not all the headers etc. Fetching the 3 or 6 digits code should be feasible.

Reedy subscribed.Nov 15 2015, 7:38 PM

We should be able to dig into this a bit more now...

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 15 2015, 7:38 PM
Reedy closed subtask T99767: Log full bounce email too before unsubscribing as Resolved.Nov 15 2015, 8:30 PM
Nemo_bis renamed this task from Valid email address was unconfirmed: too many enotifs? to Valid email address was unconfirmed for temporary spam blacklisting.Jun 27 2017, 1:54 PM

Maybe this should be closed invalid, especially if there's already some other task on how to handle temporary failures. Mailservers often lie about the reasons they bounce or defer messages, nowadays (to avoid scraping etc.).

JTannerWMF closed subtask T54915: HTML email notifications caught by spam filters as Invalid.Feb 7 2019, 2:46 AM
Aklapper added a project: Infrastructure-Foundations.Jun 21 2021, 8:59 PM
joanna_borun closed this task as Invalid.Dec 4 2023, 4:23 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits