after migration is done, send success email, announce user impacting changes
|Resolved||faidon||T84041 Replace all instances of lighttpd with nginx|
|Resolved||faidon||T84053 mailman - replace lighttpd|
|Open||None||T111653 Encrypt all the things|
|Resolved||faidon||T82576 Enable STARTTLS (both inbound and outbound) on lists|
|Resolved||JanZerebecki||T55259 Add Forward Secrecy to all HTTPS sites|
|Resolved||Dzahn||T90351 Improve SSL of lists.wikimedia.org|
|Resolved||Dzahn||T83541 Upgrade Exim to >=4.73|
|Duplicate||None||T97492 Upgrade to Mailman 3.0|
|Resolved||Dzahn||T110141 TTL back up to normal 1H|
|Resolved||MZMcBride||T27231 Mailman mailing list archiver truncates if a line begins with "From"|
|Resolved||None||T66818 Mitigate strict DMARC policy on the mailing lists|
|Resolved||Dzahn||T80945 Get rid of all Ubuntu Lucid (10.04) installs|
|Resolved||Dzahn||T82698 shutdown sodium after mailman has migrated to jessie VM|
|Resolved||Dzahn||T105756 Mailman Upgrade (Jessie & Mailman 2.x) and migration to a VM|
|Resolved||Dzahn||T110140 send follow-up email, announce changes with new mailman version if any that have user impact|
Dependencies - There is a new dependency associated with the new Privacy options -> Sender filters -> dmarc_moderation_action feature discussed below. This requires that the dnspython <http://www.dnspython.org/> package be available in Python. This package can be downloaded from the above site or from the CheeseShop <https://pypi.python.org/pypi/dnspython/> or installed with pip. New Features - The from_is_list feature introduced in 2.1.16 is now unconditionally available to list owners. There is also, a new Privacy options -> Sender filters -> dmarc_moderation_action feature which applies to list messages where the From: address is in a domain which publishes a DMARC policy of reject or possibly quarantine. This is a list setting with values of Accept, Wrap Message, Munge From, Reject or Discard. There is a new DEFAULT_DMARC_MODERATION_ACTION configuration setting to set the default for this, and the list admin UI is not able to set an action which is 'less' than the default. The prior ALLOW_FROM_IS_LIST setting has been removed and is effectively always Yes. There is a new dmarc_quarantine_moderation_action list setting with default set by a new DEFAULT_DMARC_QUARANTINE_MODERATION_ACTION configuration setting which in turn defaults to Yes. The list setting can be set to No to exclude domains with DMARC policy of quarantine from dmarc_moderation_action. dmarc_moderation_action and from_is_list interact in the following way. If the message is From: a domain to which dmarc_moderation_action applies and if dmarc_moderation_action is other than Accept, dmarc_moderation_action applies to that message. Otherwise the from_is_list action applies. Also associated with dmarc_moderation_action are configuration settings DMARC_RESOLVER_TIMEOUT and DMARC_RESOLVER_LIFETIME. These are described in more detail in Defaults.py. There are also new vette log entries written when dmarc_moderation_action is found to apply to a post. i18n - Added missing <mm-digest-question-start> tag to French listinfo template. (LP: #1275964) Bug Fixes and other patches - Removed HTML tags from the title of a couple of rmlist.py pages because browsers don't render tags in the title. (LP: #265848) - Most Mailman generated notices to list owners and moderators are now sent as Precedence: list instead of bulk. (LP: #1313146) - The Reply-To: munging options weren't honored if there was no from_is_list action. (LP: #1313010) - Changed from_is_list actions to insert the list address in Cc: if the list is fully personalized. Otherwise, the list address is only in From: and Reply-To: overrides it. (LP: #1312970) - Fixed the Munge From action to only Munge the From: and/or Reply-To: in the outgoing message and not in archives, digests and messages sent via the usenet gateway. (LP: #1311431) - Fixed a long standing issue in which a notice sent to a user whose language is other than that of the list can cause subsequent things which should be in the list's language to be in the user's language instead. (LP: #1308655) - Fixed the admin Membership List so a search string if any is not lost when visiting subsequent fragments of a chunked list. (LP: #1307454) - For from_is_list feature, use email address from original From: if original From: has no display name and strip domain part from resultant names that look like email addresses. (LP: #1304511) - Added the list name to the vette log "held message approved" entry. (LP: 1295875) - Added the CGI module name to various "No such list" error log entries. (LP: 1295875) - Modified contrib/mmdsr to report module name if present in "No such list error log entries. - Fixed a NameError exception in cron/nightly_gzip when it tries to print the usage message. (LP: #1291038) - Fixed a bug in ListAdmin._handlepost that would crash when trying to preserve a held message for the site admin if HOLD_MESSAGES_AS_PICKLES is False. (LP: #1282365) - The from_is_list header munging feature introduced in Mailman 2.1.16 is no longer erroneously applied to Mailman generated notices. (LP: #1279667) - Changed the message from the confirm CGI to not indicate approval is required for an acceptance of an invitation. (LP: #1277744) - Fixed POSTFIX_STYLE_VIRTUAL_DOMAINS to be case-insensitiive. (LP: #1267003) - Added recognition for another simple warning to bounce processing. (LP: #1263247) - Fixed a few failing tests in tests/test_handlers.py. (LP: #1262950) - Fixed bin/arch to not create scrubbed attachments for messages skipped when processing the --start= option. (LP: #1260883) - Fixed email address validation to do a bit better in obscure cases. (LP: #1258703) - Fixed a bug which caused some authentication cookies to expire too soon if AUTHENTICATION_COOKIE_LIFETIME is non-zero. (LP: #1257112) - Fixed a possible TypeError in bin/sync_members introduced in 2.1.17. (LP: #1243343) Miscellaneous - Added to the contrib directory, a script from Alain Williams to count posts in a list's archive.
New Features - Handling of posts gated from usenet to a list via the Mail <-> News gateway is changed. Formerly, no list membership, moderation or *_these_nonmembers checks were done. Now, if the sender of the usenet post is a moderated member or a nonmember matching a *_these_nonmembers filter, those checks will be done and actions applied. Nonmember posts from senders not matching a *_these_nonmembers filter are still accepted as before. (LP: #1252575) - There is a new mm_cfg.py setting ANONYMOUS_LIST_KEEP_HEADERS. Since it is not possible to know which non-standard headers in a message might reveal sender information, we now remove all headers from incoming posts to anonymous lists except those which match regular expressions in this list. The default setting keeps non X- headers except those known to reveal sender information, Mailman added X- headers and x-Spam- headers. See the description in Defaults.py for more information. (LP: #1246039) i18n - The Japanese message catalog has been updated by SATOH Fumiyasu. (LP: #1248855) Bug Fixes and other patches - Added a reopen command to the sample init.d script in misc/mailman.in. (LP: #1251917) - Fixed a misspelling in Tagger.py causing an "unexpected keyword argument 'Delete'" exception. (LP: #1251495) - Fixed contrib/qmail-to-mailman.py to work with a user other than 'mailman' and to recognize more listname-* addresses. (LP: #412293) - Fixed a possible UnicodeDecodeError in bin/sync_members. (LP: #1243343) - Fixed Makefile to not include $DESTDIR in paths compiled into .pyc files for traceback purposes. (LP: #1241770)
New Features - There is a new list attribute from_is_list to either rewrite the From: header of posts replacing the posters address with that of the list or wrap the message in an outer message From: the list for compatability with DMARC and or ADSP. There is a new mm_cfg.py setting DEFAULT_FROM_IS_LIST to control the default for new lists, and the existing REMOVE_DKIM_HEADERS setting has been extended to allow removing those headers only for certain from_is_list lists. This feature must be enabled by setting ALLOW_FROM_IS_LIST to Yes in mm_cfg.py. See the description of these settings in Defaults.py for more detail. This feature is experimental in 2.1.16, and it is subject to change or to become just one of the two methods in a subsequent release. People interested in this feature are encouraged to try it and report their experiences to the email@example.com list. - There is a new DISPLAY_HELD_SUMMARY_SORT_BUTTONS setting which if set in mm_cfg.py will display a set of radio buttons in the admindb held message summary to select how the held messages are sorted and grouped for display. The exact setting determines the default grouping and sorting. See the description in Defaults.py for details. - Setting digest_size_threshhold to zero now means no digests will be sent based on size instead of a digest being sent with every post. (LP: #558274) - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET which will put a dynamically generated, hidden hash in the listinfo subscribe form and check it upon submission. Setting this will prevent automated processes (bots) from successfully POSTing web subscribes without first retrieving and parsing the form from the listinfo page. The form must also be submitted no later than FORM_LIFETIME nor no earlier than SUBSCRIBE_FORM_MIN_TIME after retrieval. Note that enabling this will break any static subscribe forms on your site. See the description in Defaults.py for more info. (LP: #1082746) - add_members now has an option to add members with mail delivery disabled by admin. (LP: #1070574) - IncomingRunner now logs rejected messages to the vette log. (LP: #1068837) - The name of the mailmanctl master lock file is now congigurable via the mm_cfg.py setting MASTER_LOCK_FILE. (LP: #1082308) - list_lists now has an option to list only lists with public archives. (LP: #1082711) Contributed programs - A new import_majordomo_into_mailman.pl script has been contributed by Geoff Mayes. (LP: #1129742) - A new "sitemap" bash script has been contributed by Tomasz Chmielewski <firstname.lastname@example.org> to generate a sitemap.xml file of an installation's public archives for submission to search engines. i18n - The Danish translation has been updated thanks to Tom Christensen. - Fixed a string in the Czech message catalog. (LP: #1234567) - A Farsi (Persian) translation has been added thanks to Javad Hoseini and Mahyar Moghimi. - Fixed several misspelled or garbled string replacements in the Spanish message catalog. (LP: #1160138) - pt_BR message catalog has two new and an updated message per Hugo Koji Kobayashi. (LP: #1138578) - German message catalog has been updated per Ralf Hildebrandt. - Corrected typo in templates/it/private.html. Bug Fixes and other patches - Fixed a crash in SpamDetect.py which caused messages with unparseable RFC 2047 encoded headers to be shunted. (LP: #1235101) - Fixed cron/disabled to send a fresh cookie when notifying disabled members. (LP: #1203200) - Added "message_id" to the interpolation dictionary for the Article.html template. (LP: #725498) - Changed the admin GUI to report only the bad entries in a list of email addresses if any are bad. (LP: #558253) - Added logging for template errors in HyperArch.py. (LP: #558254) - Added more explanation to the bad owner address message from bin/newlist. (LP: #1200763) - Fixed a bug causing the admin web interface to fail CSRF checking if the list name contains a '+' character. (LP: #1190802) - Fixed bin/mailmanctl -s to not remove the master lock if it can't be determined to be truly stale. (LP: #1189558) - It is no longer possible to add 'invalid' addresses to the ban_list and the *_these_nonmembers filters from the check boxes on the admindb interface. (LP: #1187201) - Backported recognition for mail.ru DSNs and minor bug fixes from lp:flufl.bounce. (LP: #1074592, LP: #1079249 and #1079254) - Defended against buggy web servers that don't include an empty QUERY_STRING in the CGI environment. (LP: #1160647) - The Switchboard.finish() method now logs the text of the exception when it fails to unlink/preserve a .bak file. (LP: #1165589) - The pending (un)subscriptions waiting approval are now sorted by email address in the admindb interface as intended. (LP: #1164160) - The subscribe log entry for a bin/add_members subscribe now identifies bin/add_members as the source. (LP: #1161642) - Fixed a bug where the Subject: of the user notification of a bin/remove_members unsubscribe was not in the user's language. (LP: #1161445) - Fixed a bug where BounceRunner could create and leave behind zero length bounce-events files. (LP: #1161610) - Added recognition for another Yahoo bounce format. (LP: #1157961) - Changed configure's method for getting Python's include directory from distutils.sysconfig.get_config_var('CONFINCLUDEPY') to distutils.sysconfig.get_python_inc(). (LP: #1098162) - Added an Auto-Generated: header to password reminders. (LP: #558240) - Fixed a bug where non-ascii characters in the real name in a subscription request could throw a UnicodeEncodeError upon subscription approval and perhaps in other situations too. (LP: #1047100) - The query fragments send_unsub_notifications_to_list_owner and send_unsub_ack_to_this_batch will now assume default values if not set in mass unsubscribe URLs. (LP: #1032378) - Replaced utf-8 encoded characters in newly added German templates with HTML entities. (LP: #1018208)
Security - Two potential XSS vulnerabilities have been identified and fixed. New Features - A new feature for controlling the addition/replacement of the Sender: header in outgoing mail has been implemented. This allows a list owner to set include_sender_header on the list's General Options page in the admin GUI. The default for this setting is Yes which preserves the prior behavior of removing any pre-existing Sender: and setting it to the list's -bounces address. Setting this to No stops Mailman from adding or modifying the Sender: at all. Additionally, there is a new Defaults.py/mm_cfg.py setting ALLOW_SENDER_OVERRIDES which defaults to Yes but which can be set to No to remove the include_sender_header setting from General Options, and thus preserve the prior behavior completely. - Bounce processing has been enhanced so that if a bounce is returned to a list from a non-member who is a member of a regular_include_list, the bounce will be processed as a bounce for the included list. i18n - Fixed a missing format character in the German bin/mailmanctl docstring. - Updated Dutch translation from Jan Veuger. - Updated Japanese Translation from Tokio Kikuchi. - Updated Finnish translation from Joni T�yryl�. - Made a few corrections to some Polish templates. Bug #566731. - Made a minor change to the Chinese (China) message catalog. Bug #545772. - Changed a few DOCTYPE directives in templates for compliance. Bug #500952 and Bug #500955. Bug Fixes and other patches - Made minor wording improvements and typo corrections in some messages. Bug #426979. - Fixed i18n._() to catch exceptions due to bad formats. Bug #632660. - Fixed admindb interface to decode base64 and quoted-printable encoded message body excerpts for display. Bug #629738. - Fixed web CGI tracebacks to properly report sys.path. Bug #615114. - Changed the member options login page unsubscribe request to include the requesters IP address in the confirmation request. Bug #610527. - Changed fix_url to lock the list if not locked. Bug #610364. - Made a minor change to the English subscribeack.txt (welcome message) template to emphasize that a password is only required to unsubscribe *without confirmation*. - Fixed an issue in admindb that could result in a KeyError and "we hit a bug" response when a moderator acts on a post that had been handled by someone else after the first moderator had retrieved it. Bug #598671. - Fixed a bug which would fail to show a list on the admin and listinfo overview pages if its web_page_url contained a :port. Bug # 597741. - Fixed bin/genaliases to not throw TypeError when MTA = None. Bug #587657. - Provided the ability to specify in mm_cfg.py a local domain (e.g. 'localhost') for the local addresses in the generated virtual-mailman when MTA = 'Postfix'. See VIRTUAL_MAILMAN_LOCAL_DOMAIN in Defaults.py. Bug #328907. - Made a minor change to the removal of an Approved: pseudo-header from a text/html alternative to allow for an inserted '\xA0' before the password. - Fixed Content Filtering collapse_alternatives to work on deeply nested multipart/alternative parts. Bug #576675. - We now accept/remove X-Approved: and X-Approve: headers in addition to Approved: and Approve: for pre-approving posts. Bug #557750. - Reordered the 'cancel' and 'subscribe' buttons on the subscription confirmation web page so the default action upon 'enter' will be the subscribe button in browsers that pick the first button. Bug #530654. - Fixed a bug in the admindb interface that could apply a moderator action to a message not displayed. Bug #533468. - Added a traceback to the log message produced when processing the digest.mbox throws an exception. - Added a urlhost argument to the MailList.MailList.Create() method to allow bin/newlist and the the create CGI to pass urlhost so the host will be correct in the listinfo link on the emptyarchive page. Bug #529100. - Added the List-Post header to the default list of headers retained in messages in the MIME digest. Bug #526143. - When daemonizing mailmanctl, we now ensure terminal files are closed. - Fixed a bug in pipermail archiving that caused fallback threading by subject to fail. Bug #266572. - We now give an HTTP 401 status for authentication failures from admin, admindb, private, options and roster CGIs, and an HTTP 404 status from all the CGIs for an invalid list name. - Backported the listinfo template change from the 2.2 branch to fix Bug #514050. - Fixed a bug where going to an archives/private/list.mbox/list.mbox URL would result in a munged URL if authentication was required. Bug #266164. - Fixed a bug where check_perms would throw an OSError if an entry in Mailman's lists/ directory was not a directory. Bug #265613. - Fixed a bug where a message with an Approved: header held by a handler that precedes Approve (SpamDetect by default) would not have the Approved: header removed if the held message was approved. Bug #501739.
- DMARC improvements (requires python-dnspython)
- List names are now includes in vette.log entries for message management (debugging will be easier for specific lists)
- Script added to count amount of emails in lists. Good for informing on whether a list rebuild or move is a worthy risk.
- Mailman lock file can be configured.
- list_lists can display on lists with public archives now.
- qrunners don't cache lists.
- Cookies can be invalided after x seconds now (AUTHENTICATION_COOKIE_LIFETIME set to a >0 value).
- RESPONSE_INCLUDE_LEVEL can be set to 0 to prevent backscatter.
Administrators / Moderators
- Emails are validated more accurately. Spam prevention.
- Poster password added. Allows users who may meet moderation standards to automatically have their emails accepted.
- Cookies now set the secure flag when over HTTPS (default).
- Sessions can be invalidated through logouts on administrator and moderation interfaces.
- DMARC improvements solve issues users have with Yahoo and Outlook regarding mail.
- Password reminder link is now on the roster page (private archives).
- Emails can be automatically accepted to private lists if a poster password is used provided by list administrators.
Nothing changes for users directly. Administrators have a few new settings for DMARC management and a new poster password setting. The changes seem to have focused on Operations mostly.
I recommend the follow be set:
AUTHENTICATION_COOKIE_LIFETIME = 3600
RESPONSE_INCLUDE_LEVEL = 0
3.0 is indeed the current release but it is not packaged and not advised to upgrade to from 2.1.x branches. 2.1.20 is also the current latest branch but not packaged in Jessie. 2.1.18 is the latest version in Jessie and thus what we are running. The change logs for us are already above in mine and Daniel's comments.