Seems resolved looking around and with the added configuration variables, can be fine-tuned per list if necessary but defaults should suffice.

When I did them actively, it took about 30-60 minutes to prepare going through all the channels we use for communication (on wiki, mailing lists, twitter, email chat etc.). The only lessons I learn was keeping things simple and concise were best, especially in a weekly update feed. Updates grow quickly, especially in long sentences and explanations and people don't like have to read something the size of an entire singpost just for a weeks brief update.

Added to patch.

This should be left for @Nosy79 being the listadmin who did this.

Blockers:

• 2.1 upgrade support for Mailman 3 (serious and tested, right now it's 'partial' and patched in 2010)
• Debian upstream packaging (likely next Debian distro)
• Actually to be used elsewhere in a large deployment.

and with https://meta.wikimedia.org/wiki/Mailing_lists/Standardization I am going to call this closed! (a documentation ticket actually meeting an end, shocking).

Meta-Wiki is done.

Wikitech documentation seems done to me now!

Details sent to OTRS admins.

Information seems mixed in what I'm getting so:

Can I have an email to use for the administrator and to send the password to?

Looking over this a bit more. I'm going to try and get this done over the weekend starting Friday.

the shunt directory is a really bad messaure of this though. Files are shunted for hundreds of reasons, few of which actually equate to 'failed to process'.

Furthermore we shouldn't check the shunt directory. in, out, virgin and bounces should be the directories we check.

Looking into this, the check is running with insufficient privileges to be able to check the directories it is.

Searching via wikitech, I see:

Looks good to me. Don't see this becoming a problem if these lists received emails.

Just for an update on progress:

Done. Deleted all subscription requests for the list.

Looked into the cause of this quickly; it is a feature added in 2.1.16 as part of Debian's UTF-8 support. Where ISO encoded strings are no longer being converted automatically. Still a fatal and has further reaching effects but should be handled by list administrators (apart from these) alone in future.

It's subscription spam again (has happened in the past). Has anyone received any recently? My last bounce was September 18th and since then nothing else. Closing as invalid, re-open if this is still happening as looking at logs on fermium, this is not occurring.

It was running 2.1.13 which was released in 2008, so yes indeed.

3.0 is indeed the current release but it is not packaged and not advised to upgrade to from 2.1.x branches. 2.1.20 is also the current latest branch but not packaged in Jessie. 2.1.18 is the latest version in Jessie and thus what we are running. The change logs for us are already above in mine and Daniel's comments.

Setting to invalid as this is sort of not a bug really. Verifying the times, you made the rest post DNS change from sodium->fermium but prior to exim being started. The time you received the email does also align with a few minutes after exim was started. The few minutes delay is expected as fermium would have been handling 3 hours worth of mail.

Migration is over, things seem working (receiving emails, tailing logs on fermium show a bit of activity), so I feel confident saying we're on Jessie now, on .18 and have an unused Lucid box in the datacenter. Daniel's final call for this ticks my boxes.

In T112912#1653355, @Multichill wrote:

Hi John,

In T112912#1653302, @JohnLewis wrote:

First comment for the sake of everyone: please can people stop commenting things that add no value or enhance the ability for us to look into the issue. the 'xx list also' comments are useless and just ironically generate more emails for people.
Now my comment: I'm assigning this to myself and I will look at it in detail when I can.

Last time I checked you were not on the operations team or do you have shell access to the machine?

First comment for the sake of everyone: please can people stop commenting things that add no value or enhance the ability for us to look into the issue. the 'xx list also' comments are useless and just ironically generate more emails for people.

As Krenair said, enough lists is enough. This ticket is likely generating more spam than these bounces together. I'll take a look at this tomorrow, likely post migration (as I'll be able to actually look then).

Done. Password sent to emails listed in description.

Well, once a decision is made please assign the task to me and I'll create the list using what was decided. Thanks.

@Tfinc (list creation blocked on this now) You seem to mention a rename following @Dzahn's comment yet looking at the original request - there seems no need for a rename? The search list has the comment of "The public search mailing list could be kept for search-related discussions" which seems to be the only existing lists Discovery has. The rename comment Daniel made seems sort of unrelated in my opinion as it deviates completely from the original request.

Done. Sent password to @ori.

Will poke @Dzahn with this.

Regarding the mailing lists, no one has stepped forward to take on as list administrators. Though looking at the department and those CC'd, my mind says Wes and Tomasz? Obviously though since they're flexible to be changed - anyone does just for me to create them.

There is an SSL check (though not for this apparently) which reports warning 60 before expiry and then 30 days before. This really should be added in a meta way.

I'll handle the mailing lists tomorrow.

so we're seeing static values below 2 hours but not less than an hour and forty minutes? Still seems high but if we are directly rsyncing, it would fall under a well timed window.

Hi @Qgil, I'll be interested in this as I have the time to dedicate. You know my background so if you feel I'll be able to provide something, do ask :) otherwise other volunteers can take priority over me. Thanks.

@Aklapper nothing. seems he just forgot to resolve the task.

In T110949#1635456, @Jalexander wrote:

What's the planned timeline for the migration/change? Still planned for this coming week?

This is indeed upstream. It is not something we can change without getting into an area we're really wanting to avoid for technical reasons. Furthermore with mailman3 coming along with an entirely new archiver - I doubt this will be patched upstream. Closing as invalid for these reasons.

@jcrespo it seems and the manual swap increase form the default 1GB set in the lvm partman recipe to 8GB.

Sent to the emails listed.

Looks okay. Will create soon once I've looked over a few things.

It seems these are mailman bugs as heart which is really outside of the scope for us. If they can be fixed easily by us, document it and we'll do so otherwise if they're upstream issues - filing bugs here will do nothing and likely end in 'invalid' or 'declined' resolutions.

Closing as original.

90 minutes still seems like a concern to me but expected I guess. The length though can increase a bit which may be an issue on the day. So we really need to cut down the time between rsync 1 and rsync 2.

@Dzahn see T105229#1600235 for being able to send the commands as a contact :)

FYI doc wise: http://docs.icinga.org/latest/en/objectdefinitions.html#contact

In T105229#1555857, @Dzahn wrote:

just wondering, let's say the subtask was resolved, for which services is John requesting permissions? mailman?

Testing (on a 1.11 install on a separate project; though this seems consistent for 1.6) and docs show that if a contact has 'can_submit_commands' set to 1 in their contact definition - they are allowed to submit commands for their services. Which is what this ticket and the services ticket are about.

In T109279#1596750, @Nemo_bis wrote:

closed blocking task Restricted Task as "Resolved".

If it's resolved, please make it public. Thanks.

Operations

• DMARC improvements (requires python-dnspython)
• List names are now includes in vette.log entries for message management (debugging will be easier for specific lists)
• Script added to count amount of emails in lists. Good for informing on whether a list rebuild or move is a worthy risk.
• Mailman lock file can be configured.
• list_lists can display on lists with public archives now.
• qrunners don't cache lists.
• Cookies can be invalided after x seconds now (AUTHENTICATION_COOKIE_LIFETIME set to a >0 value).
• RESPONSE_INCLUDE_LEVEL can be set to 0 to prevent backscatter.

In T110140#1582771, @Dzahn wrote:

2.1.18 (03-May-2014)

Dependencies

  - There is a new dependency associated with the new Privacy options ->
    Sender filters -> dmarc_moderation_action feature discussed below.
    This requires that the dnspython <http://www.dnspython.org/> package
    be available in Python.  This package can be downloaded from the above
    site or from the CheeseShop <https://pypi.python.org/pypi/dnspython/>
    or installed with pip.

New Features

  - The from_is_list feature introduced in 2.1.16 is now unconditionally
    available to list owners.  There is also, a new Privacy options ->
    Sender filters -> dmarc_moderation_action feature which applies to list
    messages where the From: address is in a domain which publishes a DMARC
    policy of reject or possibly quarantine.  This is a list setting with
    values of Accept, Wrap Message, Munge From, Reject or Discard. There is
    a new DEFAULT_DMARC_MODERATION_ACTION configuration setting to set the
    default for this, and the list admin UI is not able to set an action
    which is 'less' than the default.  The prior ALLOW_FROM_IS_LIST setting
    has been removed and is effectively always Yes. There is a new
    dmarc_quarantine_moderation_action list setting with default set by a
    new DEFAULT_DMARC_QUARANTINE_MODERATION_ACTION configuration setting
    which in turn defaults to Yes.  The list setting can be set to No to
    exclude domains with DMARC policy of quarantine from
    dmarc_moderation_action.

    dmarc_moderation_action and from_is_list interact in the following way.
    If the message is From: a domain to which dmarc_moderation_action applies
    and if dmarc_moderation_action is other than Accept,
    dmarc_moderation_action applies to that message.  Otherwise the
    from_is_list action applies.

    Also associated with dmarc_moderation_action are configuration settings
    DMARC_RESOLVER_TIMEOUT and DMARC_RESOLVER_LIFETIME. These are described
    in more detail in Defaults.py.  There are also new vette log entries
    written when dmarc_moderation_action is found to apply to a post.

Bug fixes and other patches

  - Added the list name to the vette log "held message approved" entry.
    (LP: 1295875)

Miscellaneous

  - Added to the contrib directory, a script from Alain Williams to count
    posts in a list's archive.

2.1.16 (16-Oct-2013)

New Features

  - There is a new list attribute from_is_list to either rewrite the From:
    header of posts replacing the posters address with that of the list or
    wrap the message in an outer message From: the list for compatability
    with DMARC and or ADSP.  There is a new mm_cfg.py setting
    DEFAULT_FROM_IS_LIST to control the default for new lists, and the
    existing REMOVE_DKIM_HEADERS setting has been extended to allow removing
    those headers only for certain from_is_list lists.  This feature must
    be enabled by setting ALLOW_FROM_IS_LIST to Yes in mm_cfg.py.  See the 
    description of these settings in Defaults.py for more detail.  This
    feature is experimental in 2.1.16, and it is subject to change or to
    become just one of the two methods in a subsequent release. People 
    interested in this feature are encouraged to try it and report their
    experiences to the mailman-users@python.org list.

  - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET which will put
    a dynamically generated, hidden hash in the listinfo subscribe form and
    check it upon submission.  Setting this will prevent automated processes
    (bots) from successfully POSTing web subscribes without first retrieving
    and parsing the form from the listinfo page.  The form must also be
    submitted no later than FORM_LIFETIME nor no earlier than
    SUBSCRIBE_FORM_MIN_TIME after retrieval.  Note that enabling this will
    break any static subscribe forms on your site.  See the description in
    Defaults.py for more info.  (LP: #1082746)

  - The name of the mailmanctl master lock file is now congigurable via the
    mm_cfg.py setting MASTER_LOCK_FILE.  (LP: #1082308)

  - list_lists now has an option to list only lists with public archives.
    (LP: #1082711)

2.1.15 (13-Jun-2012)

Security

  - Strengthened the validation of email addresses.

New Features

  - Added a password reminder button to the private archive login page.
    Backported from the 2.2 branch.

  - Eliminated the list cache from the qrunners.  Indirect self-references
    caused lists to never be dropped from the cache which in turn caused
    the qrunners to grow very large in installations with many lists or
    multiple large lists.  Bug #862683.

  - A new list poster password has been implemented.  This password may only
    be used in Approved: or X-Approved: headers for pre-approving posts.
    Using this password for that purpose precludes compromise of a more
    valuable password sent in plain text email.  Bug #770581.

  - A new mm_cfg.py setting AUTHENTICATION_COOKIE_LIFETIME has been added.
    If this is set to a non-zero value, web authentication cookies will
    expire that many seconds following their last use.  Its default value is
    zero to preserve current behavior.

  - A new mm_cfg.py setting RESPONSE_INCLUDE_LEVEL has been added to control
    how much of the original message is included in automatic responses to
    email commands.  The default is 2 to preserve the prior behavior of
    including the full message.  Setting this to 1 in mm_cfg.py will include
    only the original headers, and 0 will include none of the original.  It
    is recommended to set this to 0 in mm_cfg.py to minimize the effects of
    backscatter.  Bug #265835.

Bug fixes and other patches
  - Mailman now sets the 'secure' flag in cookies set via https URLs.
    Bug #770377.

  - Added a logout link to the admindb interface and made both admin and
    admindb logout effective for a site admin cookie if allowed.
    Bug #769318.

3PM BST and adding usernotice.

Done.

It's used as the origination email for things like bounces, password reminders and so on. It exists mostly as a 'method users can use to communicate with site administrators'. Since @mark is the only one with it and he's said he has no intention of doing so, its fine to just disable it. A better case would be a reject notice telling people to file phab tickets if they have questions/issues however just a stop gate.

https://lists.wikimedia.org/mailman/listinfo/wikiia-l and it is listed on the main listinfo page. Archives exist https://lists.wikimedia.org/pipermail/wikiia-l/2009-April/000007.html.

I think I flipped the workboard quickly while going through.