Page MenuHomePhabricator

Have a conversation about migrating from GNU Mailman 2.1 to GNU Mailman 3.0
Open, Stalled, NormalPublic

Subscribers
Tokens
"Like" token, awarded by Sjoerddebruin."Like" token, awarded by Ladsgroup."Like" token, awarded by Pcoombe."Like" token, awarded by MarcoAurelio."Like" token, awarded by He7d3r."Like" token, awarded by MGChecker."Mountain of Wealth" token, awarded by Man77."Love" token, awarded by Steinsplitter."Like" token, awarded by Slaporte."The World Burns" token, awarded by Vituzzu."Like" token, awarded by Addshore."Love" token, awarded by greg."Like" token, awarded by dr0ptp4kt.
Assigned To
None
Authored By
AzaToth, Jul 6 2013

Description

As a Wikimedian I want to be able to follow discussions (read, reply, create) per project or theme in a convenient way, whether through email client on a (mobile) device, webmail on a (mobile) device, or through a web interface of the discussion system itself, so I'm up to date informed about what is going on and can join the conversation anytime as I like.

Alternatives to consider:

  • Keep Mailman 2.1
  • Migrate to Mailman 3.0 which has a new Django-based web user interface for end users and list administrators named Postorius (not yet officilally sanctioned by GNU)
  • Consider Discourse as web interface for Mailman mailing list (requires development of synchronization)
  • Consider (flow enabled) talk pages on wiki - add support for reply by email to topics on a talk page for example

Not a user story (original task description)
We should update Mailman to version 3.

The new version, among others, stores hashed passwords, which could have minimized the impact of last weeks security incident

Details

Reference
bz50864

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Elitre added a subscriber: Elitre.Nov 13 2015, 12:44 PM

From @Legoktm on IRC: Fedora is switching to Mailman 3 next week: https://fedoraproject.org/wiki/Mailman3_Migration

That will satisfy the "actually to be used elsewhere" blocker.

We also had a security incident (http://blog.wikimedia.org/2015/11/12/mailman-security-incident/) which bit us especially hard because Mailman 2 stores mailing list passwords in cleartext.

KTC added a subscriber: KTC.Nov 13 2015, 7:01 PM
Man77 awarded a token.Nov 15 2015, 2:16 PM
Man77 added a subscriber: Man77.
MartinK raised the priority of this task from Lowest to High.Nov 15 2015, 3:36 PM
MartinK updated the task description. (Show Details)
Aklapper lowered the priority of this task from High to Normal.Nov 16 2015, 10:44 AM

This is blocked on external parameters (see T52864#1676829) hence setting priority to normal here.

He7d3r added a subscriber: He7d3r.
KuboF added a subscriber: KuboF.Nov 17 2015, 2:24 PM
-jem- added a subscriber: -jem-.Nov 19 2015, 10:03 AM

@Legoktm and I both replied to on meta:Talk:Discourse about this upgrade. Update?

Update?

Upstream created a new patch level release https://gitlab.com/mailman/mailman/blob/3.0.1/src/mailman/docs/NEWS.rst . No new features like upgrade support, but the next minor release will have some improvements for upgrades.

As before this is still lacking people working on it. With upstream on migration support (without further knowledge and testing I don't know to what extend actual software changes are needed here); on packaging the upgrade for Debian; and on testing and applying that on our installation. There are no current, real external blockers (, while this might look like I'm disagreeing with T52864#1676829 , I think it is just saying the same from a different perspective).

@RobLa-WMF: Are you willing to resource this?

jayvdb added a subscriber: jayvdb.Jan 20 2016, 12:43 PM

@RobLa-WMF: Are you willing to resource this?

@JanZerebecki - I don't have authority to resource this. I was hoping @mark or someone from Operations would respond, but I believe that so far, all of the momentum seems to be around T124690: Successful pilot of Discourse on https://discourse.wmflabs.org/ as an alternative to wikimedia-l mailinglist.

Tgr added a comment.Mar 21 2016, 1:19 AM

@JanZerebecki - I don't have authority to resource this. I was hoping @mark or someone from Operations would respond, but I believe that so far, all of the momentum seems to be around T124690: Successful pilot of Discourse on https://discourse.wmflabs.org/ as an alternative to wikimedia-l mailinglist.

If by "momentum" you mean WMF resourcing than so far neither of those have any.

ori added a subscriber: ori.Mar 21 2016, 1:27 AM

I was hoping @mark or someone from Operations would respond

@faidon did, in T52864#954874 above.

In T52864#2138089, @Tgr wrote:

@JanZerebecki - I don't have authority to resource this. I was hoping @mark or someone from Operations would respond, but I believe that so far, all of the momentum seems to be around T124690: Successful pilot of Discourse on https://discourse.wmflabs.org/ as an alternative to wikimedia-l mailinglist.

If by "momentum" you mean WMF resourcing than so far neither of those have any.

Nope, I mean interest in spite of explicit WMF resourcing. @AdHuikeshoven has been building interest in the pilot he's been running for the past few weeks. My understanding is that the goal of T124690 is to make a case for applying WMF funds toward a Wikimedia community discussion replacement for Mailman2 mailing lists. Discourse seems to be the only alternative that is being seriously considered right now, and that's mainly due to Ad's efforts.

I understand why Operations hasn't been eager to take this on. The codfw-rollout clearly takes precedent for Operations. This past quarter has been pretty tumultuous outside of the Ops team. I only took interest today because someone else brought up Discourse, and that caused me to look here and notice that I hadn't replied to Jan.

@RobLa-WMF , thanks for the kind words. The status of Discourse is a pilot a test and generates feedback about what people like and what people don't like. There are some strong preferences to receive updates or notifications by mail about discussions and to be able not only to reply by mail to a discussion thread but also to be able to create a new thread. Some people have been asking for synchronization of mailing lists and Discourse. As, if and when that materializes Discourse can or will coexist alongside Mailman mailing lists, just like gmane coexists alongside Mailman. Discourse provides an alternative web interface and some extras. GNU Mailman 3.0 also provides a web interface. After an updrade to Mailman 3 there might be a reduced level of interest in moving to Discourse.

We are all in the same Wikimedia movement in which there are over 900 wiki websites. Project related discussion can and should be primarily on wiki. For movement wide discussion there is meta, and those discussion can and should be primarily on wiki. (This discussion should be on meta, or on mediawiki.) The mailing lists exist because they all exist for too long.

@sumanah is on the Mailman team. I'll bet she has an insider perspective on what kind of challenges you'll face when migrating from Mailman 2.1 to Mailman 3.0, or how long it will take for Mailman 3.1 to arrive.

@brion came with a challenge to integrate core concepts of mediawiki in Discourse. In the long term our goal should be to integrate Mailman mailing list features or Discourse features in the (Flow enabled) talk pages of Wikimedia project wiki websites. One of those features is to follow a discussion by mail, not only reading, but also replying and creating.

(Going to rewrite the task description to a user story)

AdHuikeshoven renamed this task from Upgrade Mailman to version 3 to Have a conversation about migrating from GNU Mailman 2.1 to GNU Mailman 3.0.Mar 21 2016, 7:10 PM
AdHuikeshoven updated the task description. (Show Details)

@sumanah is on the Mailman team. I'll bet she has an insider perspective on what kind of challenges you'll face when migrating from Mailman 2.1 to Mailman 3.0, or how long it will take for Mailman 3.1 to arrive.

I was volunteering on Mailman and am not right now. But here's a July 12th post by Mailman maintainer Barry Warsaw on the mailman developers' list in which he recognizes that 3.1 really needs to be released soon. (Mailman 3.1 is the version where the maintainers advise 2.0 users to upgrade and believe the upgrade path is smooth.) I predict Mailman 3.1 will be out by the end of 2016, and my bet would be that it'll be before the end of September.

People who'd like to know when 3.1 is out can subscribe to the Mailman announcement mailing list which is very low-volume.

sumanah removed a subscriber: sumanah.Jul 24 2016, 2:40 AM
Man77 removed a subscriber: Man77.Nov 10 2016, 10:43 PM

Hello. What's the status of this one? It'd be interesting to have our mailing lists system updated. Regards.

Regarding Debian packages for GNU Mailman 3, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799292 for the issues.

Re https://lists.wikimedia.org/pipermail/wikimedia-l/2017-August/088350.html , I think adding Mailman 3 to translatewiki.net would a reasonable first step to increase awareness and help its development. Erik, are you willing to open this discussion in the relevant venues?

A second step could be switching some non-WMF Wikimedia mailman installations. https://intern.wikimedia.ch/lists/listinfo/ seems a better candidate than http://mailman.wikimedia.it/listinfo because they seem to run on more up to date software (and they have a more diverse userbase, including some international mailing lists). Both are run by volunteers with little time though, AFAIK.

JFTR, the first maiman 3 package has been uploaded to Debian now: https://packages.qa.debian.org/m/mailman3-core.html

@MoritzMuehlenhoff Does that help or ease the suggested migration? Thanks.

Elitre updated the task description. (Show Details)Oct 9 2017, 2:17 PM

@MoritzMuehlenhoff Does that help or ease the suggested migration? Thanks.

The mailman 3 packaging in Debian isn't complete yet (missing the web interface ATM), but when it's complete that would certainly ease a migration to mailman 3.

Legoktm added a subscriber: herron.Jan 26 2018, 9:42 PM

I briefly talked with @herron about this today. I think we are still blocked on the lack of Debian packaging. Looking at https://qa.debian.org/developer.php?email=pkg-mailman-hackers%40lists.alioth.debian.org, none of the packages were included in stretch (and some aren't in testing yet), so we'd either need stretch backports or wait until the buster release.

Restricted Application added a subscriber: alanajjar. · View Herald TranscriptJan 26 2018, 9:42 PM

mailman3-core, mailman3-hyperkitty, postorius and mailmanclient have been accepted into stretch-backports today.

Does that mean we can start considering our migration?

That's dependent on goal planning / road map considerations, I only meant to point out the availability in backports since it was mentioned earlier on this task.

mailman3-core, mailman3-hyperkitty, postorius and mailmanclient have been accepted into stretch-backports today.

@Legoktm @herron where does this put us now then? Thanks.

I think we need to lobby/convince/remind @faidon and other roadmap deciders to allocate resources for this :)

This comment was removed by MarcoAurelio.
Reedy updated the task description. (Show Details)May 11 2018, 6:24 PM
Reedy removed subscribers: wikibugs-l-list, JohnLewis.

I think we need to lobby/convince/remind @faidon and other roadmap deciders to allocate resources for this :)

Hello @faidon. Do you think we can move on this? Thanks.

Hi. Any status updates here? Thanks.

Hi. Any status updates here? Thanks.

If you are in a hurry to switch to mailman 3, maybe you could talk with the development team about increasing its translations as I suggested?

Re https://lists.wikimedia.org/pipermail/wikimedia-l/2017-August/088350.html , I think adding Mailman 3 to translatewiki.net would a reasonable first step to increase awareness and help its development. Erik, are you willing to open this discussion in the relevant venues?

I am not in any hurry. I am just requesting an status update.

chasemp added a subscriber: chasemp.Nov 6 2018, 6:37 PM
Meno25 removed a subscriber: Meno25.Nov 23 2018, 7:48 AM
RuyP added a subscriber: RuyP.Jan 17 2019, 7:45 PM
Stryn added a subscriber: Stryn.Jan 17 2019, 7:56 PM
Base added a subscriber: Base.Mar 6 2019, 8:22 PM
Masti added a subscriber: Masti.Mar 6 2019, 8:40 PM

@Legoktm @herron Could we have an update on this one, please? Thanks.

Presumably it’s the usual no activity == no update...

Tgr added a subscriber: Qgil.Mar 13 2019, 10:14 PM

There has been a lot of activity on Discourse, OTOH. @Qgil might be able to say more on that.

@Reedy As far as I could see, T52864#3241375 is resolved, and T52864#3940724 is as well. GNU seems happy with Postorius and HyperKitty as well (http://list.org/features.html). Therefore, is there any blockers that prevent this to go forward?

@Reedy As far as I could see, T52864#3241375 is resolved, and T52864#3940724 is as well. GNU seems happy with Postorius and HyperKitty as well (http://list.org/features.html). Therefore, is there any blockers that prevent this to go forward?

Potentially whatever host mailman is running on may need an os upgrade, or a reinstall... which may potentially mean new hardware depending on warranty status

Then someone to do test migrations, before upgrading the live versions. I don’t know if SRE has this in their goals/timelines

In T52864#5022889, @Tgr wrote:

There has been a lot of activity on Discourse, OTOH. @Qgil might be able to say more on that.

A few weeks ago I set up two scenarios to test:

So far so good, although more testing and feedback is definitely welcome. There seem to be some corner cases / bugs related to identifying signatures and attachments, but the Discourse team has been very responsive.

I am trying to build a case around Discourse that includes mailing lists / email users, but as you know or may imagine there are many pieces involved. I believe Discourse may provide a better end result for email and web users, integrating forum and mailing lists under a common umbrella (same usernames, common search...).

This is just a quick update. I need a bit more time to prepare a decent proposal for discussion.

Sorry, it took more time than I expected when I posted the comment above, but here it is: T226727: Integrate mailing lists in Wikimedia Space & https://discuss-space.wmflabs.org/t/integrating-mailing-lists-to-wikimedia-space/136

How would you prefer to have this conversation about migrating from GNU Mailman 2.1 to... ?

I am well aware that a plan for a potential migration from Mailman to Discourse would be complex and require many details defined that today are vague or unknown. I just want to say that we (the team working on Wikimedia Space) are ready to take part in this conversation and assume our share in the work required to proceed with this migration.

@Qgil Thanks for your comment. As a user of some mailing lists, I am still interesting in upgrading to Mailman 3+. At T52864#5022944 we have some potential issues. I guess that, to be totally sure, we could create a subtask and check what really needs to be done to move this forward?

To be clear, the proposal is that users of a Mailing mailing list about X could keep using the same email features subscribing in mailing list mode to a Discourse category about X (same topic, same email functionality). The hypothesis is that Discourse can roughly cover the functionality that Mailman 3+ provides.