Thanks to @Dzahn for the pointer on adding the relevant team and VTRS. @Arnoldokoth: do you think you can help with this one? Thanks!

@Seddon These are going to https://wikitech.wikimedia.org/wiki/VRT_System not mailing lists. So this would mean no more volunteers would answer them via tickets. Is that still what you want?

I wanted to add that since these contact addressess are in the Android and iOS stores, we should make sure that the dissociation from VTRS and moving to Google Groups happens almost at the same time, ideally. This should then require some coordination with ITS.

@Seddon: I think it might make sense to start the conversation at that end point them to this task. Thanks.

Yep, in that case the first step would be to open a ticket in Zendesk. Then once your new groups exists in Google (and have admins, you have access etc), then you should ask for them to be removed from VRTS.

Ending this has been requested by the VRTS team. These are old queues that the apps teams used to use a while back. They are unmanaged by either volunteers or staff.

The overall goal is to close the VRTS queues, and send the emails that go to the old addresses, to the newer Google-group addresses. I.e.

• Emails to mobile-ios-wikipedia@wikimedia.org (currently VRTS) should instead end up at iOS-support@wikimedia.org
  • already exists (I think, I cannot access details) https://groups.google.com/a/wikimedia.org/g/ios-support
• Emails to mobile-android-wikipedia@wikimedia.org (currently VRTS) should instead end up at android-support@wikimedia.org
  • Unclear if it exists (different error message for me) https://groups.google.com/a/wikimedia.org/g/android-support

There are emails already used by the apps teams that are set up as google groups:

ios-support@wikimedia.org and android-support@wikimedia.org

I want to point mobile-android-wikipedia@wikimedia.org AND mobile-ios-wikipedia@wikimedia.org at those addresses respectively.

You should probably use the same email addresses for the google groups instead of starting new ones. They just need to exist on the Google side first and then they can be removed from the config that routes them to VRTS.

If you want aliases for other addresses to be sent to your new google groups then that would also be a request for ITS to add them as aliases in your new Google groups.

android-support@wikimedia.org might be a user vs a google group. I cna double check if it makes a difference

I would highly recommend to use real Google groups and not single user inboxes with generic names.

We can't tell the difference from the outside, both just tell us that they get routed to a gsuite_account.

Regardless, SRE would only be involved in removing any addresses from VRTS, not in aliases or forwarding between different email addresses.

That would all be in ITS nowadays.

P.S. I don't think you need to NDA this ticket and that just meant your new subscriber can't read it yet.

@Arnoldokoth Could you take a look in VRTS and VRTS config and check if you see the email addresses mentioned in this ticket? They are configured somewhere in mail servers to point to VRTS but, unexpected to me, I don't see them in the puppet repo. But I also know there is that alias config file that gets generated.

Meanwhile they have been created in Google so the overall goal is to disable them on our side.

P.S. I had this in mind too that is maybe kind of related and I wanted to mention anyways:

23:56 < jinxer-wm> FIRING: [4x] SystemdUnitFailed: generate_vrts_aliases.service on mx-in1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - 
                   https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed

@Dzahn I think those are stored in the database and from the vrts_aliases script are the ones queried on L64.

I looked into it and I think the steps for disabling this would be to first disassociate the queue from the email or disabling the queue entirely by setting it to invalid then doing the same for the email address under 'System Email Address Management' on the admin page.

I think we can loop in one of the admins to see if there is a procedure for disabling/transitioning a queue. @Krd Is there any documented process for this?

P.S. I had this in mind too that is maybe kind of related and I wanted to mention anyways:

23:56 < jinxer-wm> FIRING: [4x] SystemdUnitFailed: generate_vrts_aliases.service on mx-in1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - 
                   https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed

It very likely is. A patch to remove these two addresses from the script is ready to be merged: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1070222

Hey all,

these 2 addresses are now shown as "routed to gmail" on our mail servers, as opposed to before:

[mx1001:~] $ sudo exim4 -bt mobile-android-wikipedia@wikimedia.org
mobile-android-wikipedia@wikimedia.org
  router = gsuite_account, transport = remote_smtp
  host aspmx.l.google.com [172.253.62.26]

[mx1001:~] $ sudo exim4 -bt mobile-ios-wikipedia@wikimedia.org
mobile-ios-wikipedia@wikimedia.org
  router = gsuite_account, transport = remote_smtp
  host aspmx.l.google.com [172.253.62.26]

@Seddon This means you should now be seeing emails in the new Google groups, if you wanna confirm it works. Cheers

In T373485#10112944, @LSobanski wrote:

It very likely is. A patch to remove these two addresses from the script is ready to be merged: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1070222

Thanks! I merged that, hoping it clears the alert. But I do have to add that the monitoring alert was already alerting before we did anything on this ticket. There might be more than one reason for that.

Why is this exemption hardcoded in a script while the addresses could have been just removed from VRTS? I'd appreciate if VRTS admins could be involved in such things, so that it can at least be considered to build as little legacy as possible.

@Krd this is based on an older comment found here: https://phabricator.wikimedia.org/T284145#7218511

Are you aware of a better way to handle removing addresses from VRTS?

Addresses can be disabled in VRTS.
I don't know how the exim integration works at all, but if somebody is willing to share some information, VRTS admins can perhaps add their perspective.

In T373485#10115801, @Krd wrote:

Why is this exemption hardcoded in a script while the addresses could have been just removed from VRTS? I'd appreciate if VRTS admins could be involved in such things, so that it can at least be considered to build as little legacy as possible.

Yes, tbh, this is how the whole process should have started. By contacting the existing addresses.

There are at least 2 different aspects to this.

One is "are WMF mail servers configured to route mails to this address to VRTS or not" (and then "if not" it means it falls back to Google).

The second is "is VRTS configured to expect mails coming to this address". I don't know that much about this but of course I agree we also also need this.

And then there is T368257 which needed https://gerrit.wikimedia.org/r/c/operations/puppet/+/1070222 as a fix but needs another and unrelated fix.

I _think_ the same gerrit patch above also did the switch in the mail routing. Unless somebody else did that. But I can say that until recently these were routed to VRTS and now they are routed to Google, where new groups have been created for this by ITS.

This could be checked with exim4 -bt <email address> on the WMF mx servers.

compare to previous comment T373485#10098016.

[mx1001:~] $ sudo exim4 -bt mobile-android-wikipedia@wikimedia.org
mobile-android-wikipedia@wikimedia.org
  router = gsuite_account, transport = remote_smtp
  host aspmx.l.google.com [142.251.167.26]

I would like to see the whole content of vrts_aliases.py and discuss (perhaps at a different venue) if things can be cleanup up or otherwise improved.

@Krd I created a separate task for this and linked to the full script in the description: T374090: Improve generating VRTS alias list

In T373485#10120180, @Krd wrote:

I would like to see the whole content of vrts_aliases.py and discuss (perhaps at a different venue) if things can be cleanup up or otherwise improved.

Here you go https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/profile/files/mail/vrts_aliases.py

@Seddon Could you confirm if the google groups work for you and you are receiving mails there?

I think if that's the case we can call this ticket resolved, given that there is a separate task for the other questions above.

Great! Thanks @Seddon :)