@jhathaway Sorry for not closing the loop on this one. It is resolved now.

Thanks @jhathaway . I just clicked the button on the Shopify admin console to test DKIM and SPF, but I'll confirm here once the test is complete. It says it may take up to 24 hours.

@jhathaway All good, thanks for the breakdown. I also CCd you in the support interaction with Shopify and they mentioned that the 4th record was for SPF as well. Is there any further action required to complete this on my end? Last I checked, we added the 4 record associated with the public instance, but not our internal instance.

@jhathaway Thanks for flagging this. I do not know the answer off the top of my head, but will try to figure out why this week. Should Gmail show dmarc auth results if we're not actually enforcing a dmarc policy on the mail server side?

@jhathaway I reached out to Sandra requesting that I be connected with our Shopify rep for clarification. Will update this task when I learn more.

Thanks @ssingh . The other Shopify instance still needs the CNAME records added it looks like, but we are good-to-go on this one.

Thanks @ssingh. All looks good on the Shopify end for this instance. It says are domain is authenticating now.

@jhathaway I do not know what CNAME record 4 is for. I can ask Sandra to connect me with Shopify Support if we need to confirm the purpose of the record before implementing the changes.

@ssingh Thank you, I just initiated the process, which Shopify says may take 24 hours to complete. I'll follow up here when the process finishes.

@jhathaway is this the Task to track requirements for high-volume third party senders who impersonate our domain? Such as Shopify, Qualtrics, Acoustic? Or should each of those software titles be its own task?

Hey @JAnstee_WMF, I've been working with Tanja more on this on our Zendesk ticket. The next step I proposed was to set up a meeting with us and a Qualtrics engineer to troubleshoot further, because our SMTP relay is seemingly set up correctly on the Google side. I think Tanja is out this week, so we will pick it back up when Tanja returns.

how about, surveys@wikimedia.org?

@jhathaway qualtrics@wikimedia.org exists as a Google Group, but not a Google user.

Thanks for the additional background. I was unaware that we had any SMTP relay rules set up for Qualtrics, but it looks like we do (screenshot). Should I add the additional IP ranges Qualtrics noted in their email to this SMTP relay rule?

I've sent test mail from a couple different addresses, one internal and one external, and both emails went through to Zendesk just fine. All looks good on that front.

Hey @Dzahn my apologies for the delay. I just completed the first two steps:

Sounds good, thanks @Dzahn. I'll follow up here tomorrow when the ITS tasks are done (around 15:30 UTC most likely).

Hey @Dzahn I'm just following up to confirm that Advancement approved the plan, so let's proceed tomorrow with the steps you outlined:

That makes sense @Dzahn. I'll check in again on this task on the 16th.

Hey @Dzahn I heard back from Advancement and they're ready to move on this. They have a maintenance window open next week, so I was planning on making the changes on Tuesday, May 17, if that works for you.

I heard back from SADA, our Google vendor.

Thank you @herron and @jbond I'll open a ticket with Google now and keep you updated.

Hey @Dzahn I heard back from Advancement and they'd like to hold off on adjusting their Zendesk intake (donate@) until their maintenance window on May 16. I'll comment on this Task in May when we're ready to make the changes.

Thanks again @Dzahn. I'll circle back with Advancement and keep you updated.

Hey @Dzahn my apologies, but I discovered there is one more issue that needs to be resolved with Advancement and that person is in a much different timezone for me, so it cannot be resolved right now. Can you please re-add donate@ on your side and I will work with this person to coordinate an asynchronous switchover? I'm sorry for the extra work.

Hey @Dzahn and @akosiaris I'm working Kristie Robinson from Advancement to also move over donate@ to LDAP on the ITS side. Can you please remove the following aliases from the SRE side?

Hey @jbond awesome news! Glad it's working again. I agree that it would be better to use a service account to prevent this from happening again. We have a super admin service account ldapadmin@wikimedia.org. Would that work for you?

Hey @brennen we currently use the Slack "email a channel" for a couple different hacky purposes. I don't think implementing for your team would necessarily require a Legal/Security review. Who would be able to access the Slack channel email address on the Phabricator side of things?

Hey @Dzahn can you please remove wikimania as an alias from the mail servers controlled by SRE? @elappen-WMF and the Wikimania team would like to use it as a Google Group alias managed by ITS. I know there are still some other aliases to shift over to ITS as well. Thanks for your help.

I can assist with this. I believe once SRE removes the aliases from their side, ITS can add on the Google side.

@Dzahn Got it, thank you for clarifying.

@Dzahn Thanks for sharing the doc, that's helpful. Are there any outstanding emails left in the queue?

Hey @herron thanks. I think I uploaded the eml file privately and added you as a subscriber, but let me know if you don't see it.

@Dsharpe I heard back from the vendor and they let me know that the developer confirmed that the encryption implementation is the same as the method described in the document. I'll now schedule a follow-up meeting with ITS, Security, and Privacy to discuss.

Hey @Dsharpe. The vendor just completed the burning keys after use feature and deployed it to our server. They told me that the document shared with us in July is essentially still accurate, so I followed up and requested that they revise the document so that it describes the exact solution implemented for us. I'll follow up with you here when I hear back.

Hey @Dsharpe! Can you please share the diagram you are referencing with me? Are you referring to the confidential Element recommendation memo we put together with the Privacy team?

@sbassett - Will do, understood.

@sbassett The vendor let me know that they will not be able to provide us with server logs, but they are willing to work with us to get access to the backend of a testing environment. They told me that because the testing environment they set up is running on their private infrastructure, they would prefer to move it to a separate instance for us to be able to freely access. They prefer that we use our own hardware, but offered to set something up for us on their side that we would be able to access.

Understood, thanks. I'll see what New Vector can provide us with and follow up here.

@Aklapper Got it, noted!

@BBlack Bingo, that worked. Our domain is now verified. Apple's documentation says you can remove the record now if you want.

Hey @BBlack all good, thanks for the help. Unfortunately, the verification random string resets itself every 14 calendar days, so the txt record in the original request is now invalid. Can we update with the new txt record below?

Thanks all. Is the patch live yet?

Thanks all. It's working. https://federationtester.matrix.org/#foundation.wikimedia.org

Hey folks. Unfortunately, we discovered that the SRV/DNS solution is blocking our ability to use the Integrations feature on Element. The Integrations feature is what allows us to do self-service bridging configuration and other things. Can we use the https://foundation.wikimedia.org/.well-known/matrix/server URL for the well known file? Here is what the person from Element says:

Thanks. I found it. Sorry about that. Resolved.

Vendor says it looks to be all correct now. They shared this link: https://federationtester.matrix.org/#foundation.wikimedia.org.

I heard back from the vendor regarding DNS and the rep said "I have not found the DNS way for delegation in our internal docs, but also no explicit "we don't support it". So lets try it. The required DNS entry is described in the Spec under step 4 here: https://matrix.org/docs/spec/server_server/latest#resolving-server-names".

Thanks @Aklapper and @Dzahn. You're right, it's just a Google Group with no LDAP entry behind it because invite-wmfall@ doesn't need to send or receive mail. This issue is resolved.

Thanks @Aklapper . Sorry about not adding the correct tag.

I successfully tested the matrix-appservice-slack dedicated slack integration (using the Events API) on the free trial of modular. It was a little buggy a couple weeks ago when I first tested, but they fixed the bugs after I reached out to them. You can't DM on the free trial using this type of bridge.

@Milimetric For the sake of transparency and not creating more work for others, hosting our own Matrix server is most likely not going to happen. I'm speaking to the person at Mozilla who spearheaded their transition to Matrix this Friday and I'm looking forward to learning about how they made it happen.

I'll set up a trial for modular.im, the Matrix SaaS product, in the next couple of months and update this task when I do so.

Thanks all. Just another quick reminder to check the Group settings if you haven't already. They're set to the default.

@Dzahn I created jenkins-bot@ as a Google Group and added @hashar as the sole owner and member (don't need a minimum of 2 anymore). The permissions are set to default, so @hashar, please navigate to groups.google.com to change settings, as necessary.

@Dzahn I just added pat@, gary@, and box6699@ as aliases to Google Group tsops@wikimedia.org. You should be able to delete on your side now.

@jrbs Okay, so I think I figured this out...

Hey folks. Not sure what to do for this task now that James is gone. Should I make the changes I suggested?

@Dzahn I see in our ticket history that a Google Group was not desired by @egalvezwmf, but rather an additional individual account and email address. The ticket has since been resolved, so I believe this task can be closed.

@Dzahn makes sense, thanks. I'll let you know.

We can set these aliases up on our end. Just let me know when to do so. I imagine you'll remove the aliases from your end and we'll add said aliases to our end?

@jijiki I'll email Legal today and follow up with an answer on this thread. Thanks for your help.

@jcrespo It looks like @Dzahn added @RStallman-legalteam to the thread for Legal's input, but I can follow-up with legal via email if necessary. This request stemmed from an OIT techsupport ticket from @Jalexander requesting that gary@ be changed to an alias for trustandsafety@.

The links should be fixed now. Thanks, @zeljkofilipin

New YouTube link with higher res slides: https://www.youtube.com/watch?v=GvowpttwdqQ
Commons upload: https://commons.wikimedia.org/wiki/File:Wikimedia_Continuous_Delivery_Pipeline-_Say_What%3F.webm

Sorry for all of the confusion and fractured messages, but I deleted the link I just pasted above and changed the start time of the original link for "Wikimedia Continuous Delivery Pipeline: Say What?" [1] to 2pm tomorrow. Please let me know if you have any questions.

Oh, after actually reading this thread it looks like it's "Wikimedia Continuous Delivery Pipeline: Say What?" Is that correct?

@dduvall here is the streaming/recording link: https://www.youtube.com/watch?v=Ejn2x6MXSh4

Public YouTube stream on MediaWiki channel: https://www.youtube.com/watch?v=IXSqbNxsyhg