Page MenuHomePhabricator

Make SPF for wikimedia.org more strict
Closed, ResolvedPublic

Description

As suggested by Platonides in T127247, change our SPF for wikimedia.org from ?all to -all

http://www.openspf.org/SPF_Record_Syntax

Mechanisms can be prefixed with one of four qualifiers:

"+"	Pass
"-"	Fail
"~"	SoftFail
"?"	Neutral
The "all" mechanism
all
This mechanism always matches. It usually goes at the end of the SPF record.

Examples:

"v=spf1 mx -all"
Allow domain's MXes to send mail for the domain, prohibit all others.

"v=spf1 -all"
The domain sends no mail at all.

"v=spf1 +all"
The domain owner thinks that SPF is useless and/or doesn't care.

Currently WMF has

v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ?all

Which means all other mail servers are considered "neutral"

v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 -all

Would make it so all other mail servers are prohibited if they're not in the list before hand

Details

Related Gerrit Patches:

Event Timeline

csteipp created this task.Apr 20 2016, 6:38 PM
csteipp created this object with visibility "Custom Policy".
csteipp created this object with edit policy "Custom Policy".
Reedy updated the task description. (Show Details)May 5 2017, 5:57 PM
Reedy updated the task description. (Show Details)May 5 2017, 6:20 PM
Reedy updated the task description. (Show Details)May 5 2017, 6:27 PM
Reedy added a subscriber: Reedy.May 5 2017, 6:31 PM

See also T131930 for doing it for toolserver.org

Restricted Application added a project: Traffic. · View Herald TranscriptMay 5 2017, 8:03 PM

Proposal from earlier this month was to bump from ? (neutral) to ~ (soft-fail), and then determine impact before moving further. Did this go anywhere?

Reedy added a subscriber: herron.Jun 7 2017, 1:40 AM

So, a reason to potentially looking at getting this done quicker...

@JAufrecht had an email "from" Katherine (which was marked as spam) requesting "I need the complete emails and names of our donors."

@herron As you're now apparently the "maintainer" for mail, could you look at getting this actioned. Whether per James we go in stages (ie to soft fail first). And similarly T131930

greg added a subscriber: greg.Jun 7 2017, 4:49 AM
faidon added a subscriber: faidon.Jun 7 2017, 12:43 PM

@herron (new operations engineer) is working on all of those SPF/spoofed email issues indeed.

This email spoofed to look like it's from Katherine is interesting (not particularly surprising though, it was a matter of time until such a thing happened).

The fact that this email went to spam is a good thing -- it probably means that SPF/DKIM checks worked for Gmail to classify this as illegitimate. Can we get the full email, with headers please? There are other actionables with regards to this email, I'll follow-up separately.

Reedy added a comment.Jun 7 2017, 4:34 PM

I did ask for Joel to post the headers in the email thread, but no reply yet...

Here's the headers:

Delivered-To: jaufrecht@wikimedia.org
Received: by 10.182.19.137 with SMTP id f9csp592120obe;
        Sat, 27 May 2017 01:30:59 -0700 (PDT)
X-Received: by 10.233.237.79 with SMTP id c76mr6688841qkg.76.1495873859008;
        Sat, 27 May 2017 01:30:59 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1495873858; cv=none;
        d=google.com; s=arc-20160816;
        b=VLzPee2ZmHQelLfwJhniPGaKKHbW9INQ9cH8wt7Qr1jTHMFzR/BHqKW3kvc4Y6L0i4
         6UoZqKWMZRk3mZNUIpd+NPc6dKcSLvGmkFQl8+P+/PeFj6sMN2cUBxhxsSCfHGnTf0cf
         1bxrp08gKmwfdkWqokMn7ZR6dVCqHT3nEA6lv8I8AZrp4tzNILDUZHNEAfdfNIIf5k4v
         Ggptg+1JkPznA+PX1d1btgimeyQJIWfTxjGfgXMntahSkVn0lC/cqlARoMLxfQq2AHUa
         MdJ/TqEV2yNOpKxYZNBdn4IIzgWb5O8xuWoIRyn14BATPTJt0Rw8ZX2Gj+0RwgiK5hT7
         SMlg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=date:message-id:reply-to:errors-to:importance:from:subject:to
         :arc-authentication-results;
        bh=Kmbe0XiRS79oX6TRGC6/UqRyqchwx0YIt60VSfyP/XE=;
        b=ZEUqmMNC+p+kfjFuVyRCWLo7ZqbnKZtet3jeI3ymHY6/6O3eK95xspx9NhiQmwaxpP
         GSVNecd7dDBk1xfAwunNErh+vY0EIzP9oqO0uAx/PzyMrAY7NkGQkkb6AIiT4HyAU4En
         j8gz/+yykQPdUyTEGh8BLe+aK4urQb6aFEh+6c0kTdOVmlfutQESZ03aqYx9XcoDDzwm
         n2IZKbgVgrWq1RfQiHmL8CWQBYbuYeuYZf/6UXNSg5sCQ3x+ifc+JVJazptLMM6NbDh4
         VK6sg6DbjDAOakUqxZ3TvMdPaNXtl+GAqZvVXoJKPn93a8P5glynst2Kk8k7629EbUx4
         yjpw==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=neutral (google.com: 2a01:5e0:36:5001::1491:8ce5 is neither permitted nor denied by domain of kmaher@wikimedia.org) smtp.mailfrom=kmaher@wikimedia.org
Return-Path: <kmaher@wikimedia.org>
Received: from mx1001.wikimedia.org (mx1001.wikimedia.org. [208.80.154.76])
        by mx.google.com with ESMTPS id t39si3223269qtb.79.2017.05.27.01.30.58
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Sat, 27 May 2017 01:30:58 -0700 (PDT)
Received-SPF: neutral (google.com: 2a01:5e0:36:5001::1491:8ce5 is neither permitted nor denied by domain of kmaher@wikimedia.org) client-ip=2a01:5e0:36:5001::1491:8ce5;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 2a01:5e0:36:5001::1491:8ce5 is neither permitted nor denied by domain of kmaher@wikimedia.org) smtp.mailfrom=kmaher@wikimedia.org
Received: from [2a01:5e0:36:5001::1491:8ce5] (port=55885 helo=emkei.cz) by mx1001.wikimedia.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <kmaher@wikimedia.org>) id 1dEX7h-00042J-PZ; Sat, 27 May 2017 08:30:58 +0000
Received: by emkei.cz (Postfix, from userid 33) id 713266BB65; Sat, 27 May 2017 10:30:27 +0200 (CEST)
To: 
Subject: Notice
From: Katherine Maher <kmaher@wikimedia.org>
X-Priority: 3 (Normal)
Importance: Normal
Errors-To: williambills@protonmail.com
Reply-To: Katherine Maher <williambills@protonmail.com>
Content-Type: text/html; charset=utf-8
Message-Id: <20170527083028.713266BB65@emkei.cz>
Date: Sat, 27 May 2017 10:30:27 +0200 (CEST)
X-Spam-Score: 4.4 (++++)
X-Spam-Report: Spam detection software, running on the system "mx1001.wikimedia.org", has identified this incoming email as possible spam.
  The original message has been attached to this so you can view it or label similar future email.
  If you have any questions, see the administrator of that system for details.
  Content preview:
  I need the complete emails and names of our donors. Katherine
    Maher Wikimedia Foundation Executive Director [...]
   Content analysis details:
   (4.4 points, 4.0 required)
   pts rule name
              description ---- ---------------------- --------------------------------------------------
  0.0 HTML_MESSAGE
           BODY: HTML included in message -1.9 BAYES_00
               BODY: Bayes spam probability is 0 to 1%
                             [score: 0.0083]
  0.7 MIME_HTML_ONLY
         BODY: Message only has text/html MIME parts
  0.7 MIME_HEADER_CTYPE_ONLY 'Content-Type' found without required MIME
                             headers
  0.4 HTML_MIME_NO_HTML_TAG
  HTML-only message, but there is no HTML tag
  1.6 REPLYTO_WITHOUT_TO_CC
  No description available.
  0.8 RDNS_NONE
              Delivered to internal network by a host with no rDNS
  2.1 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From

<p>I need the complete emails and names of our donors.</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p><em>Katherine Maher</em></p>
<p><em>Wikimedia Foundation Executive Director</em></p>

Could we also look at setting up a more restrictive DMARC record for our domain? Has it been considered before?

Our current DMARC:

v=DMARC1; p=none; sp=none; rua=mailto:dmarc-rua@wikimedia.org; ruf=mailto:dmarc-ruf@wikimedia.org;

This means:

  • p=none means there is no policy
  • sp=none means we do nothing with messages that fail the DKIM signature

G-Suite How To on DMARC: https://support.google.com/a/answer/2466580?hl=en&ref_topic=2759254

How to deploy DMARC: https://support.google.com/a/answer/2466563

Reedy added a comment.Jun 7 2017, 5:55 PM

I'd suggest a separate ticket for it :)

I'd suggest a separate ticket for it :)

@Reedy OK! I have created ticket https://phabricator.wikimedia.org/T167337 for this task.

jrbs added a subscriber: jrbs.Jun 7 2017, 7:49 PM
Bawolff added a subscriber: Bawolff.Jun 9 2017, 8:00 PM

Looking at common email providers on the internet, gmail.com and hotmail.com seem to set very similar DMARC, DKIM, and SPF records as us, except that they do ~all (SOFTFAIL all) instead of neutral all. Perhaps following suit and setting ~all (As a permanent setting, not as a testing thing until eventual -all) would be the best bet here.

herron added a comment.Jun 9 2017, 9:42 PM

Agreed that soft fail is a good course of action. We may wish to apply a stricter setting in the future but should evaluate after running soft fail for some time.

I went ahead and submitted a change to update wikimedia.org SPF from ?all to ~all (https://gerrit.wikimedia.org/r/#/c/358132). Ready for review.

Reedy added a comment.Jun 9 2017, 10:30 PM
$ grep spf wikimedia.org -B3
_dmarc.lists    1H  IN TXT "v=DMARC1; p=none; rua=mailto:dmarc-rua@wikimedia.org, ruf=mailto:dmarc-ruf@wikimedia.org;"

; SPF txt and rr records
wikimedia.org.      600 IN TXT  "v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ?all"

donate          5M  IN TXT  "v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ?all"
--
lists           1H  IN A    208.80.154.75
lists           1H  IN AAAA 2620:0:861:3::2
lists           1H  IN MX   10 lists
lists           1H  IN TXT  "v=spf1 mx ?all"
--
benefactors     600 IN DYNA     geoip!text-addrs
                1H  IN MX       10 mx1001
                1H  IN MX       50 mx2001
                1H  IN TXT      "v=spf1 include:wikimedia.org include:spf.mandrillapp.com -all"
--
phabricator     600 IN DYNA     geoip!misc-addrs
phabricator     1H  IN MX       10 mx1001
phabricator     1H  IN MX       50 mx2001
phabricator     1H  IN TXT      "v=spf1 mx ip4:10.64.32.150 ip6:2620:0:861:103:10:64:32:150 -all"

Just noticed phabricator and benefactors have -all already.

lists and donate have ?all. Should we be changing them too at the same time?

Also T131930 and https://gerrit.wikimedia.org/r/#/c/283870/ should be done too

@Reedy could we tackle the lists and donate subdomains in their own issues to review in detail and spread out deployment?

Reedy added a comment.Jun 12 2017, 3:52 PM

@Reedy could we tackle the lists and donate subdomains in their own issues to review in detail and spread out deployment?

We can, yeah. Wasn't sure if it was worth doing the lot in one go or whatever

herron claimed this task.Jun 12 2017, 5:26 PM
herron moved this task from Backlog to In Progress on the Mail board.Jun 14 2017, 10:07 PM
herron added a comment.Aug 2 2017, 4:53 PM

https://gerrit.wikimedia.org/r/#/c/358132/ has been merged and SPF soft fail is now live for wikimedia.org

$ host -t txt wikimedia.org
wikimedia.org descriptive text "v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ~all"
Reedy closed this task as Resolved.Aug 2 2017, 5:01 PM
Reedy removed a project: WMF-NDA.
Reedy changed the visibility from "Custom Policy" to "Public (No Login Required)".
Reedy changed the edit policy from "Custom Policy" to "All Users".
herron moved this task from In Progress to Done on the Mail board.Aug 2 2017, 5:22 PM