Page MenuHomePhabricator

Create cloud-admin and archive labs-admin mailing list
Closed, ResolvedPublic

Description

Work out any kinks in the mailing list replacement process by creating a new cloud-admin mailman list and archiving the current labs-admin mailman list.

The members and general configuration of the existing labs-admin list should be copied to the new cloud-admin list and verified before the older list is archived.

Details

Related Gerrit Patches:

Event Timeline

bd808 created this task.Jun 6 2017, 4:38 PM

We want to make sure that the existing list archives redirect transparently to the new list archives. This is probably the biggest deal in the renaming process. We really don't want to break all links to labs-l archives that are on wikis and in Phabricator.

bd808 moved this task from Triage to Backlog on the Cloud-Services board.Jun 17 2017, 8:02 PM
RobH added a subscriber: RobH.EditedJul 10 2017, 2:40 PM

Anytime an mbox file is touched, if anything was removed since the last re-compile, it can change the links. Also it will require downtime on the mailman server to accomplish. If the archives are private, moving the list name is better than my below alternative.

Alternative: the old list could be left in place, shut down, and just linked to the archives. (This is only a good idea when archives are public. If it is a private archive, then leaving them in place and abandoned tends to cause access issues in the future for those archives.) A new list with a new name could be created, and the membership list copied over. This would require no downtime on the host. This wouldn't seemlessly link them, the old url would still work normally for archives. Again though, if the archives are private, this is non ideal.

@RobH, you certainly have more experience with this sort of thing than I do. If creating new lists and archiving the old lists is the safe way to rename then lets do it that way. I think you are describing is basically option 1 from https://wiki.list.org/DOC/4.70%20How%20do%20I%20change%20the%20name%20of%20(rename)%20a%20list%3F

bd808 renamed this task from Rename labs-admin mailing list to cloud-admin to Create cloud-admin and archive labs-admin mailing list.Sep 6 2017, 5:38 PM
bd808 updated the task description. (Show Details)

Task description has been changed to describe a create and archive change rather than a rename.

RobH claimed this task.Sep 7 2017, 5:29 PM

Chatted with BD about this via irc, will attempt to get to this later today/tomorrow.

RobH closed this task as Resolved.Sep 19 2017, 7:40 PM
This comment was removed by RobH.
RobH reopened this task as Open.Sep 19 2017, 7:41 PM
RobH added a subscriber: Quiddity.

Ok, removed my old comment, it was incorrect.

The new list is in place with settings and Chase has the new admin email sent to him. I've left the old list in place and functioning for now, due to @Quiddity's email requesting that the new list receive the old lists emails.

Is that a requirement for this change? Does the new list have to process the old lists to address? (I saw no requirement for this to keep the old archives intact, not a requirement for redirection of new emails.)

RobH added a subscriber: herron.Sep 19 2017, 7:46 PM

Adding @herron as he has done quite a bit of mailman work recently and may have insight.

RobH removed RobH as the assignee of this task.Sep 19 2017, 7:46 PM
MarcoAurelio added a subscriber: MarcoAurelio.EditedSep 19 2017, 7:49 PM

Can't we just rename it? I think there was a command for that?

Edit: yep, I've found https://wikitech.wikimedia.org/wiki/Mailman#Rename_a_mailing_list

RobH added a comment.Sep 19 2017, 7:56 PM

Can't we just rename it? I think there was a command for that?

Anytime an mbox file is touched, if anything was removed since the last re-compile, it can change the links. Also it will require downtime on the mailman server to accomplish. If the archives are private, moving the list name is better than my below alternative.
Alternative: the old list could be left in place, shut down, and just linked to the archives. (This is only a good idea when archives are public. If it is a private archive, then leaving them in place and abandoned tends to cause access issues in the future for those archives.) A new list with a new name could be created, and the membership list copied over. This would require no downtime on the host. This wouldn't seemlessly link them, the old url would still work normally for archives. Again though, if the archives are private, this is non ideal.

Yes, but it requires scheduled downtime of the mailman server and opsen who are comfortable renaming a list. I'm happy to give it a shot, but I've not used the script, only done it manually awhile back. Renaming means the mbox file is touched, which also means it can reindex the entire archives and old links could break.

https://wikitech.wikimedia.org/wiki/Mailman#Rename_a_mailing_list has some details. The mbox reindexing is not always going to happen, but it tends to more often than not. Considering the comment below, it seems pretty important they not reindex, but I've almost always seen it do so. Renaming means downtime and possibly reindexing of archives.

We want to make sure that the existing list archives redirect transparently to the new list archives. This is probably the biggest deal in the renaming process. We really don't want to break all links to labs-l archives that are on wikis and in Phabricator.

RobH added a comment.Sep 19 2017, 7:57 PM

While I put the new list in place, it seems it may have been premature. I've pulled the new list off the server until we figure it out!

While I put the new list in place, it seems it may have been premature. I've pulled the new list off the server until we figure it out!

The need to halt all of mailman for in-place renaming seemed to be unnecessarily impactful to me. The create, copy, and archive approach seems much less likely to cause larger issues.

I did forget/neglect to document that it would be ideal for the old list alias to continue to deliver to the new list. In the case of the admin and announce lists that is really not a big deal, but for T175190: Create cloud mailman list and archive labs-l list it will be.

RobH added a comment.Sep 19 2017, 8:13 PM

The old list alias can continue to deliver to the new list alias by our inclusion of it in operations/puppet:/modules/profile/files/exim/listserver_aliases. That should put in the mail redirection for the old address to the new list. (That is what its seems used for in other cases.) I'd want to have us plan to test it when it is done though.

@bd808: Since you are the manager for cloud, I'll take your say as final on how to accomplish this. It seems the following:

  • Do NOT rename, as it requires downtime of the list server. It could also lead to re-indexing of the mbox file.
  • Create new cloud-admin mailing list, copy the settings and admin/list members from labs-admin.
  • Disable the old labs-admin mailing list so it no longer delivers mail, but leave its archives in place and untouched.
  • Merge change to operations/puppet:/modules/profile/files/exim/listserver_aliases to add in labs-admin: cloud-admin line.
  • test email to old labs-admin@lists.wikimedia.org to ensure it delivers to cloud-admin@lists.wikimedia.org.

This should accomplish what you want on cloud-admin, as well as hopefully what is needed for the conversion of the old labs-l list as well. (It seems best to try it out on this lower traffic list first!)

@bd808: Does this seem reasonable to you?

  • Do NOT rename, as it requires downtime of the list server. It could also lead to re-indexing of the mbox file.
  • Create new cloud-admin mailing list, copy the settings and admin/list members from labs-admin.
  • Disable the old labs-admin mailing list so it no longer delivers mail, but leave its archives in place and untouched.
  • Merge change to operations/puppet:/modules/profile/files/exim/listserver_aliases to add in labs-admin: cloud-admin line.
  • test email to old labs-admin@lists.wikimedia.org to ensure it delivers to cloud-admin@lists.wikimedia.org.

This should accomplish what you want on cloud-admin, as well as hopefully what is needed for the conversion of the old labs-l list as well. (It seems best to try it out on this lower traffic list first!)
@bd808: Does this seem reasonable to you?

+1 -- that seems like a good plan and test to me.

bd808 raised the priority of this task from Normal to High.Sep 21 2017, 8:14 PM

We would like to get this done by the end of the current Foundation quarter.

Change 381052 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] redirect labs-admin to cloud-admin mailing list

https://gerrit.wikimedia.org/r/381052

RobH added a comment.Sep 27 2017, 5:56 PM

Ok, I've setup an alias for the redirection, and disabled the old mailing list.

emails to the new list work, will merge the patch live and test old list redirection

Change 381052 merged by RobH:
[operations/puppet@production] redirect labs-admin to cloud-admin mailing list

https://gerrit.wikimedia.org/r/381052

RobH assigned this task to bd808.Sep 27 2017, 6:18 PM
RobH removed a project: Patch-For-Review.

We've gotten this working now. All emails sent to labs-admin@lists.wikimedia.org automatically forward into cloud-admin@lists.wikimedia.org without any special actions needed to be taken by any parties.

The old archives should remain intact. Please test everything out, since this will be how we move over the other lists.

Steps taken:

  • create new list
  • export listing of old lists members, mass subscribe to new list with welcome message pointing to this phab task
  • disable of the old mailing list via disable_list, leaving the archives intact!
  • alias added via https://gerrit.wikimedia.org/r/#/c/381052/ and merged live on lists server
  • added the old mailing list address to acceptable aliases, listed in new lists web admin panel under Privacy, Recipeint Filters, acceptable aliases
    • not doing the above means every email sent to the old address automatically hits moderation, this is non-optimal, so added for it to automatically accept those emails.
  • tested both cloud-admin@ and labs-admin@lists.wikimedia.org, both work.

I'm assigning this to @bd808 for his followup and approval. If this all tests and works as you want, we can handle the other labs => cloud list migrations in the same method.

Restricted Application added a project: User-bd808. · View Herald TranscriptSep 27 2017, 6:18 PM
bd808 closed this task as Resolved.Sep 27 2017, 7:49 PM
bd808 reassigned this task from bd808 to RobH.

Everything looks good to me.

I announced the list name change: https://lists.wikimedia.org/pipermail/cloud-admin/2017-September/000004.html