Page MenuHomePhabricator

Ensure that gitlab.wikimedia.org adheres to Google's sender guidelines
Closed, ResolvedPublic

Description

Google has announced new requirements for sending email to gmail accounts effective 2024-02-01. This is a tracking task to review what (if anything) is required for GitLab to be compliant.

Requirements for all senders

  • Set up SPF or DKIM email authentication for your domain.
  • Ensure that sending domains or IPs have valid forward and reverse DNS records, also referred to as PTR records.
  • Use a TLS connection for transmitting email.
  • Keep spam rates reported in Postmaster Tools below 0.10% and avoid ever reaching a spam rate of 0.30% or higher.
  • Format messages according to the Internet Message Format standard (RFC 5322).
  • Don’t impersonate Gmail From: headers. Gmail will begin using a DMARC quarantine enforcement policy, and impersonating Gmail From: headers might impact your email delivery.
  • If you regularly forward email, including using mailing lists or inbound gateways, add ARC headers to outgoing email.

Requirements for high-volume senders

I don’t have data on this atm but I would not be surprised if we’re over the 5k emails per day threshold.

  • Set up DMARC email authentication for your sending domain. Your DMARC enforcement policy can be set to none.
  • For direct mail, the domain in the sender’s From: header must be aligned with either the SPF domain or the DKIM domain. This is required to pass DMARC alignment.
  • Marketing messages and subscribed messages must support one-click unsubscribe, and include a clearly visible unsubscribe link in the message body.

Event Timeline

  • notification emails are sent from gitlab@gitlab.wikimedia.org so this is a separate domain., DNS verified ipv4/ipv6 forward and reverse
  • SPF - Received-SPF: pass (google.com: domain of gitlab@gitlab.wikimedia.org designates 2620:0:860:1:208:80:153:7 as permitted sender

mail route:

Received: from git by gitlab2002.wikimedia.org with local (Exim
Received: from gitlab2002.wikimedia.org .. by mx1001.wikimedia.org
Received: from mx1001.wikimedia.org (mx1001.wikimedia.org....by mx.google.com

Change 994314 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/dns@master] add google-site-verification to gitlab.wikimedia.org

https://gerrit.wikimedia.org/r/994314

per comment on the related ticket for phabricator (T355691#9500163) "dkim signature should be okay, since our dmarc record is set to relaxed. Which allows subdomain matching, i.e. dkim: wikimedia.org aligns with from: no-reply@phabricator.wikimedia.org."

This should therefore also be true for gitlab.wikimedia.org.

To actually check the "spam rate" box we first have to add google-site-verification TXT record to DNS for gitlab.wikimedia.org

With that we can add it as a separate domain in https://postmaster.google.com

mail volume is all over the place per day:

1for eximlog in $(ls /var/log/exim4/mainlog*.gz); do zcat $eximlog | head -n1 | cut -d " " -f1; zgrep gitlab.wikimedia.org $eximlog | wc -l; done
22024-01-20
343
42024-01-19
5357
62024-01-18
7374
82024-01-17
9594
102024-01-16
11326
122024-01-15
13237
142024-01-14
1517
162024-01-13
1725
182024-01-12
19491
202024-01-11
21560
222024-01-10
23550
242024-01-09
25355
262024-01-08
27340
282024-01-07
2932
302024-01-06
3132
322024-01-05
33730
342024-01-04
354808
362024-01-03
374640
382024-01-02
394147
402024-01-01
411323
422024-01-28
4340
442023-12-31
4521
462023-12-30
475
482023-12-29
4945
502023-12-28
5117
522023-12-27
5322
542023-12-26
5516
562023-12-25
57103
582023-12-24
591447
602023-12-23
611450
622023-12-22
631611
642024-01-27
6530
662023-12-21
671520
682023-12-20
69162
702023-12-19
71225
722023-12-18
73217
742023-12-17
7560
762023-12-16
77142
782023-12-15
79q677
802023-12-14
81304
822023-12-13
83422
842023-12-12
85434
862024-01-26
87161
882023-12-11
89496
902023-12-10
91265
922023-12-09
935331
942023-12-08
951758
962023-12-07
971790
982023-12-06
991692
1002023-12-05
101482
1022023-12-04
103245
1042023-12-03
1058
1062023-12-02
10728
1082024-01-25
109330
1102023-12-01
111235
1122024-01-24
113471
1142024-01-23
115647
1162024-01-22
117296
1182024-01-21
1196

Change 994314 merged by Dzahn:

[operations/dns@master] add google-site-verification to gitlab.wikimedia.org

https://gerrit.wikimedia.org/r/994314

LSobanski moved this task from Incoming to Work in Progress on the collaboration-services board.

Change 994864 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/dns@master] add DMARC record for gitlab.wikimedia.org

https://gerrit.wikimedia.org/r/994864

LSobanski lowered the priority of this task from High to Medium.Feb 1 2024, 5:53 PM

Change 994864 abandoned by Dzahn:

[operations/dns@master] add DMARC record for gitlab.wikimedia.org

Reason:

https://gerrit.wikimedia.org/r/994864

I was about to create the upstream feature request to implement true "one-click unsubscribe" but then I noticed it actually exists.

I am not sure where I was looking before but now when I checked an email from gitlab@gitlab.wikimedia.org that notified me about some changes to a repo thread, there was an "unsubscribe" link and when I followed that I was immediately unsubscribed with one click.

I think last time I just saw "manage all notifications" which is more than one click.

So this box is checked.

There is still nothing at https://postmaster.google.com/u/0/dashboards#do=gitlab.wikimedia.org&st=userReportedSpamRate&dr=7

But I now think it just means no spam was ever reported. And this means the checkbox "keep spam rate below 0.10%" can be checked.

Why are some or all of my dashboards empty? Why do I not see any data?

Most of the Postmaster Tools dashboards will only display data when there’s a sizable daily volume of email traffic (up to the order of hundreds) coming from your Authentication Domains and/or certain other conditions, in place to prevent abuse.

You may see a "No data to display" error message on the IP and Domain reputation dashboards. This may happen when your reputation is too low to show a value. Learn how to improve your email delivery rate.

To show data, some of the dashboards, like Spam Rate and Feedback Loop, need your emails to be authenticated by DKIM.

It's "userReportedSpamRate" and you can also play around with the "dr=" parameter which I think means days but that changes nothing.