Email server's DMARC config prevents users from sending emails via Special:EmailUser
Closed, ResolvedPublic

Description

Yahoo recently changed their DMARC configuration, which has two results that are relevant for us:

  1. It's no longer possible to send e-mails From: someone@yahoo.com, even if it's allowed from an SPF point of view (i.e. 'someone@yahoo.com via wikipedia.org'). This means Yahoo users cannot send e-mail from the wiki anymore.
  1. It's no longer possible to change parts of an e-mail (e.g. the subject to add a mailing list name) sent by a Yahoo user. This means Yahoo users cannot send e-mail to a mailing list anymore. If they do try to, they will receive a flood of error mails from mail servers rejecting the e-mail.

See http://www.ietf.org/mail-archive/web/ietf/current/msg87153.html for more info on issue 2).

For issue 1), we might want to block Special:SendEmail for people with a @yahoo.com address, telling them their e-mail will not be delivered.


See Also:
T58414: Get mail relay out of Yahoo! blacklist: apply to Yahoo for whitelisting bulk mail
T61731: mailman emails taking long time for delivery, getting stuck in sodium

Details

Reference
bz64795
There are a very large number of changes, so older changes are hidden. Show Older Changes

@TheDJ does that ticket now block this one?

TheDJ added a comment.May 1 2015, 5:11 PM

@Technical13 No. As far as I can see I think all that needs to be done is to flip wgUserEmailUseReplyTo and emails will be sent using the wiki@wikimedia.org address.

revi added a subscriber: revi.May 28 2015, 12:24 PM
Elitre added a subscriber: Elitre.Oct 7 2015, 4:03 PM
Restricted Application added a subscriber: Matanya. · View Herald TranscriptOct 7 2015, 4:03 PM
NQ added a subscriber: NQ.Oct 7 2015, 4:57 PM
Restricted Application added a subscriber: Luke081515. · View Herald TranscriptJan 7 2016, 7:34 PM
Restricted Application added a subscriber: Malyacko. · View Herald TranscriptJun 10 2016, 10:29 PM
Colin added a subscriber: Colin.Sep 6 2016, 6:51 PM

Why are these email bug reports not being dealt with? One suggestion, that huge groups of people get told that special:email is broken for them, is just unacceptable. The solution is trivial:

For users with problem email providers, send the email as though it came from MediaWiki just the same as all the notification emails I currently get. And just stick the sender's email address in the subject or as the first line of the email -- it's a plain-text email anyway so formatting it to add an extra line is easy.

This is the work of an hour or so. Please. I shouldn't have to change my email provider in order to use MediaWiki, and I'm not sympathetic to the whole "spoofing the sender" thing you did anyway.

Change 322243 had a related patch set uploaded (by Legoktm):
Set $wgUserEmailUseReplyTo = true;

https://gerrit.wikimedia.org/r/322243

Huji added a comment.Nov 27 2016, 12:01 AM

r322243 will possibly fix this for Wikimedia, by changing the sender address to the same one used by the wiki itself (which we know works) but sticking the sending user's email address into the reply-to field.

However, that will not fix the issue completely. If I am a non-WMF user of MW, I have no way to know about this issue or the possible fix for it. Therefore, I am going to also assign this task to MediaWiki, and I hope it is kept open until the latter concern is addressed, may it be through documentation change, adding it to the list of "known issues", or even changing MediaWiki code such that it defaults to using reply-to when certain conditions are met.

Huji added a comment.Nov 27 2016, 12:06 AM

Another thought:

If an email is tried to be sent a few times and it bounces (as appears to be the case here when the sender is a Yahoo! address) and $wgUserEmailUseReplyTo is false, I think what should really happen is MediaWiki should return the email to the sender (i.e. an email from noreply@wikimedia.org to that Yahoo! address) and say "MediaWiki was unable to send this message, because it boucned. It appears that your email server may not allow MediaWiki to send emails on your behalf; please use a different email provider, or notify your wiki admins to enable the $wgUserEmailUseReplyTo option".

I think this should be incorporated into Extension:BounceHandler. I am proposing this, because then non-WMF wiki's would have a way to know what to do when emails don't go through (... at least as long as they also use Extension:BounceHanlder).

Huji renamed this task from DMARC: Users cannot send emails via a wiki's [[Special:EmailUser]] to Email server's DMARC config prevents users from sending emails via Special:EmailUser.

Change 323672 had a related patch set uploaded (by Legoktm):
Set $wgUserEmailUseReplyTo = true; on group0 wikis

https://gerrit.wikimedia.org/r/323672

Titou awarded a token.Nov 29 2016, 3:06 PM
Titou rescinded a token.

Change 323672 merged by jenkins-bot:
Set $wgUserEmailUseReplyTo = true; on group0 wikis

https://gerrit.wikimedia.org/r/323672

Mentioned in SAL (#wikimedia-operations) [2016-12-01T01:48:00Z] <legoktm@tin> Synchronized wmf-config/InitialiseSettings.php: Set $wgUserEmailUseReplyTo = true; on group0 wikis - T66795 (duration: 00m 46s)

The proposed fix is now live on test wikis and mediawiki.org. Here's what the emails now look like:

To: Legoktm <legoktm.wikipedia@gmail.com>
Subject: Wikipedia email from user "Legoemailtest1"
From: Wikipedia <wiki@wikimedia.org>
Reply-To: Legoemailtest1 <testwiki@legoktm.com>

From is now set to Wikipedia, but the Reply-To is the account I sent it from. If you've had issues sending emails in the past feel free to send me test emails from test.wikipedia.org and I can let you know if they've been received. In about a week I'll be deploying this to all wikis for all users pending everything goes well.

Huji added a comment.Dec 1 2016, 2:01 AM

Should From be Wikipedia <wiki@wikimedia.org> or something like either Legoktm <wiki@wikimedia.org> or Wikipedia on behalf of Legoktm <wiki@wikimedia.org>? I am asking because from is what will show up in your inbox, and if it is always "Wikipedia" it may not be super useful.

This comment was removed by Xaosflux.
Xaosflux added a comment.EditedDec 1 2016, 2:34 AM

Perhaps something like username-projectname@wikimedia.org and include which project generated the mail? Examples:
xaosflux-enwiki@wikimedia.org; xaosflux-meta@wikimedia.org; etc./

Then if there is a abuse from a project - you will know which project admins need to revoke email access form a user.

MaxSem added a subscriber: MaxSem.Dec 1 2016, 2:38 AM

The problem here is that usernames are case-sensitive and allow more characters than email addresses. It can't work if there's no simple 1:1 mapping between them.

Perhaps something like username-projectname@wikimedia.org and include which project generated the mail? Examples:
xaosflux-enwiki@wikimedia.org; xaosflux-meta@wikimedia.org; etc./

Then if there is a abuse from a project - you will know which project admins need to revoke email access form a user.

The email footer will still say "This email was sent by Legoemailtest1 to Legoktm by the "Email this user" function at Wikipedia." at the bottom.

Maybe the footer needs work too - that doesn't tell you if it came from say enwiki, eswiki, dewikibooks, etc

czar awarded a token.Dec 1 2016, 4:29 AM
czar removed a subscriber: czar.

Maybe the footer needs work too - that doesn't tell you if it came from say enwiki, eswiki, dewikibooks, etc

Do you want to split that into a separate task then? I don't really see how it's related to this task as the current emails don't include that information anyways.

The last few suggestions are only tangentially relevant. Let's not stall this task with scope creep.

TheDJ added a comment.Dec 1 2016, 4:39 PM

I just realized something... As emails will now appear to be sent by Wikimedia, it might be less clear to people that by reply'ing, they will disclose their own email address...

That might be problematic...

I just realized something... As emails will now appear to be sent by Wikimedia, it might be less clear to people that by reply'ing, they will disclose their own email address...

That might be problematic...

Disclosing you email address while replying to an email in an email provider or software, is that not the usual behavior?

The point is that they'll be disclosing to an address other than the sender (wikimedia). But I agree this is a non-issue

Elitre added a comment.Dec 1 2016, 6:29 PM

Notified en:WP/VPT as that's the place where such issues were reported in the past: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Previous_issues_in_sending_emails_from_the_site .

Izno added a subscriber: Izno.Dec 1 2016, 6:44 PM
Huji added a comment.Dec 2 2016, 12:06 AM

I think the footer should be expanded to also say "by replying to this email, you will be disclosing your email address to the original sender of this message".

I am totally fine with updating the footer message to make it clearer that you're revealing your own email address, but I don't consider it a blocker to this task - it should be split to a separate one.

Paladox added a subscriber: Paladox.Dec 2 2016, 9:39 PM

I'm aiming to deploy this everywhere tomorrow (that way T152242 will have been deployed everywhere too)

Change 322243 merged by jenkins-bot:
Set $wgUserEmailUseReplyTo = true; everywhere

https://gerrit.wikimedia.org/r/322243

Mentioned in SAL (#wikimedia-operations) [2016-12-08T22:03:10Z] <legoktm@tin> Synchronized wmf-config/InitialiseSettings.php: Set $wgUserEmailUseReplyTo = true; everywhere - T66795 (duration: 00m 56s)

Legoktm closed this task as Resolved.Dec 8 2016, 10:11 PM
Legoktm claimed this task.

Change 326044 had a related patch set uploaded (by Legoktm):
Set $wgUserEmailUseReplyTo = true by default

https://gerrit.wikimedia.org/r/326044

@Legoktm - I have tested it (ru wiki) - it works! =) Thanks.

Huji added a comment.Dec 9 2016, 3:18 AM

A user at FA WP also just tested it with a Yahoo! sender address and it worked. No surprise, of course, because Yahoo! email servers were not involved at all.

@Legoktm - I have sent 3 emails with copy to me. Two of these emails I have sent to me.

After 10 minutes I have received 0 emails and 0 copies. The correct result should be 3 copy and 2 emails. Its on ru wiki.

Any explanations?

Sorry - it was only a time delay!

AnnaMariaKoshka added a comment.EditedDec 11 2016, 1:27 PM

@Legoktm - after some time of tesing - it doesnt work.

If I send the emails to me - its ok.

If I send to other users - I receive the copy of my message (email) and the user receive the notification in wiki. This notification said that I have sent the email but he got no email. :(

@AnnaMariaKoshka: Feel free to create a separate task with more information that allows investigation. I assume the user checked their spam folder?

Johan added a subscriber: Johan.Dec 12 2016, 7:09 PM

I changed the email address on one of my accounts (one I use when I give presentations and don't want to log in with my normal account on unknown computers) to a Yahoo address and failed to replicate the problem; both sending and receiving worked fine.

Change 326044 merged by jenkins-bot:
Set $wgUserEmailUseReplyTo = true by default

https://gerrit.wikimedia.org/r/326044