Page MenuHomePhabricator

HTML email notifications caught by spam filters
Closed, InvalidPublic

Description

spam notice on echo HTML email notification

It just occurred to me that all HTML notifications I've received to date from no-reply-notifications@wikimedia.org (Echo) are in my Gmail spambox. Google displays a notice saying that "Many people marked similar messages as spam". If confirmed, this seriously affects our ability to reach notification recipients.

Possibly related to T54556: lists.wikimedia.org has no SPF record


Version: unspecified
Severity: major
See Also:
T54556: lists.wikimedia.org has no SPF record
T58416: Investigate in what new Gmail "inbox tabs and category labels" MediaWiki emails are going

Attached:

Details

Reference
bz52915

Related Objects

StatusAssignedTask
OpenNone
ResolvedNone
InvalidNone
ResolvedLegoktm
OpenNone
Resolved01tonythomas
Resolved01tonythomas
Resolved01tonythomas
Resolvedcsteipp
Resolved01tonythomas
Resolved01tonythomas
InvalidNone
ResolvedNone
Resolved01tonythomas
Resolvedaaron
Resolved01tonythomas
ResolvedNone
Declined01tonythomas
Resolved01tonythomas
OpenNone
ResolvedNone
OpenNone
Resolvedherron

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 1:50 AM
bzimport added a project: Notifications.
bzimport set Reference to bz52915.
bzimport added a subscriber: Unknown Object (MLST).
DarTar created this task.Aug 16 2013, 5:37 AM
TheDJ added a comment.Aug 16 2013, 8:30 AM

Possibly this might have the same cause as the mailinglist messages that were ending up in the gmail spambox.

Created attachment 13107
Mail notifications headers compared

(In reply to comment #1)

Possibly this might have the same cause as the mailinglist messages that were
ending up in the gmail spambox.

This is supposed not to be the case, because they have different sender addresses with different SPF records and they are sent by a different IP, precisely to avoid @wikimedia.org mail, ML and wiki/other mail to be collated as single spam agent (this was done a few years ago by Mark, IIRC).

Note that Echo emails are different from standard enotif in almost every single header (subject, from, sender, reply-to, footer and of course body are all non-standard), see an example at https://www.mediawiki.org/w/index.php?title=Extension:GettingStarted/Welcome_enotif&oldid=686716

I have been told that the Echo team had discussed with ops the implications of having a different header. Indeed, I have tested it and despite the

From: Wikipedia <no-reply-notifications@wikipedia.org>
Reply-To: No Reply <no-reply-notifications@wikipedia.org>

and the absence of a Sender, the presence of

	(envelope-from <wiki@wikimedia.org>)

in the Received headers makes it pass the SPF check:

Authentication-Results: mx.google.com;
      spf=pass (google.com: domain of wiki@wikimedia.org designates 2620:0:860:2:219:b9ff:fedd:c027 as permitted sender) smtp.mail=wiki@wikimedia.org

However, there is not much to do if the emails "look like spam" and users press the spam button, except – I think – making it clear that they are not spam.

I don't know if this is the actual problem, it may be not. Most notifications (some 5k out of 16k daily) are talk pages notifications which already triggered enotiftalk before Echo. Only some 40 % of notifications goes to new users; experienced users probably already received most notifications and know how to deal with them. We don't have stats on pre-Echo enotiftalk nor on enotifwatchlist (
http://ee-dashboard.wmflabs.org/dashboards/enwiki-features

As it was noted on the wikitech-l thread about mailing list "spam", Gmail suggests you to unsubscribe from diligent email senders, rather than marking them spam, if they use the appropriate headers, in that case List-Unsubscribe and List-Subscribe. I don't know if we can use these headers to point to [[Special:Preferences#mw-prefsection-echo]] or there are better headers; it could be a starting point though, as it has no cost.

Attached:

Prioritization and scheduling of this bug is tracked on Mingle card https://mingle.corp.wikimedia.org/projects/flow/cards/232

Hello Dario and Nemo, thank you both for reporting and investigating this issue!

Dario, do we have any way to confirm how widespread this issue is, beyond your own experience?

I have not heard any reports from other users about this problem as of this time and there are no posts about it on the talk page.

For now, I have Cc:d a couple members of our Core Features team, so they can help assess the problem and recommend a possible solution.

Terry Chay, would you like to provide an engineering perspective on this issue, so we get a sense of what's feasible on your end, if anything?

Nemo suggested that we consider using List-Unsubscribe and List-Subscribe headers to point to [[Special:Preferences#mw-prefsection-echo]], if we think this would help address the issue.

Another report, this time about Yahoo!: https://en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_(technical)&oldid=579262399#WMF_notification_email_marked_as_spam_by_Yahoo
The user doesn't say if it's HTML notification but from the subject it's Echo.

So, it wasn't HTML but plain-text email: the situation with Yahoo! seems rather bad as of now, I don't know if it happened after Echo (more emails? more people clicking "spam" buttons?) or before. I filed a few bugs and more will be needed: bug 56315, bug 56413, bug 56414.

Elitre added a comment.Nov 1 2013, 4:02 PM

Silly question time.
If the sender is no-reply-notifications@wikipedia.org and an affected user sets up a Gmail filter (which explicitly allows him/her to specify such emails are never spam), wouldn't such an operation solve the issue?

(In reply to comment #7)

Silly question time.
If the sender is no-reply-notifications@wikipedia.org and an affected user
sets
up a Gmail filter (which explicitly allows him/her to specify such emails are
never spam), wouldn't such an operation solve the issue?

Yes, for that user. :) Just adding that address (or even wiki@wikimedia.org) to contacts should also work in Gmail (and perhaps Yahoo), which is why I opened bug 56416.

Elitre added a comment.Nov 1 2013, 4:11 PM

Ok, thanks, this is good to know.
Workarounds are usually better than nothing, IMHO.

legoktm reminded me of this bug. Thanks to BounceHandler, we identified at least one spam filtering service which had blocked our enotifs' *content* and get them to reverse the block: T99444#1292318
So we should work on the blockers currently filed, but also use BounceHandler to identify further issues and file them.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 23 2015, 7:54 AM
Restricted Application added a project: Growth-Team. · View Herald TranscriptSep 4 2018, 2:16 PM
JTannerWMF closed this task as Invalid.Feb 7 2019, 2:46 AM
JTannerWMF added a subscriber: JTannerWMF.

Open again if you run into the issue again