Page MenuHomePhabricator

Legoktm (Legoktm)
UserAdministrator

Projects (103)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Sep 19 2014, 7:30 PM (346 w, 3 d)
Roles
Administrator
Availability
Available
IRC Nick
legoktm
LDAP User
Legoktm
MediaWiki User
Unknown

Recent Activity

Yesterday

Legoktm created T282501: Make sure bounce_score_threshold isn't set too low.
Mon, May 10, 10:47 PM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T250963: Replace use of deprecated hook InternalParseBeforeSanitize.

I believe there are two issues that need solving:

  1. How to implement Variables as a Parsoid extension, and if new Parsoid functionality is needed for that
  2. What to do in the meantime for the legacy Parser, as the InternalParseBeforeSanitize hook is hard deprecated but still in use
Mon, May 10, 10:30 PM · ci-test-error, MediaWiki-extensions-Variables
Legoktm reopened T250963: Replace use of deprecated hook InternalParseBeforeSanitize as "Open".

I'm re-opening this based on discussion in T236809: Refactor Parser.php to allow alternate parser (Parsoid), where it seems that the declining of this task was based on a misunderstanding.

Mon, May 10, 10:26 PM · ci-test-error, MediaWiki-extensions-Variables
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

Current status:

  • I upgraded flufl.bounce which I thought might be better at parsing bounce messages, hopefully causing less crashes and then it unsubscribed me and Amir from listadmins@ for bouncing too much, which I'm pretty sure isn't the case.
    • Also listadmins@ bounce threshold was 1(!) which I bumped to 5.
  • The bounce runner is also crashing because of lock wait timeouts.
  • And while writing this it crashed again.
Mon, May 10, 9:47 PM · SRE, Wikimedia-Mailing-lists
Legoktm triaged T282490: acme-chief is down: ValueError: OCSP response status is not successful so the property has no value as Unbreak Now! priority.
Mon, May 10, 8:18 PM · Patch-For-Review, SRE, Acme-chief
Legoktm added a comment to T236809: Refactor Parser.php to allow alternate parser (Parsoid).

...and they offered to do the same for Variables except it was the Variables maintainer who said they didn't want to do anything: T250963#6169439.

I would like to rebut this statement. In this issue and in the followi, we came to the conclusion that it is fundamentally impossible to port the main Variables features to Parsoid, since it depends on pages being parsed in linear order. @ssastry has confirmed this just a few posts later, in T250963#6934680.

Mon, May 10, 7:59 PM · MW-1.36-notes (1.36.0-wmf.8; 2020-09-08), MediaWiki-extensions-Variables, MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), Parsing-Active-Work, Patch-For-Review, Parsoid, MediaWiki-Parser
Legoktm added a comment to T236809: Refactor Parser.php to allow alternate parser (Parsoid).

I'm not angry, really, so much as I see this sort of WMF-centric thinking from the developers often, and I think there needs to be some better feedback mechanism than simply trusting wfDeprecated() and the like to tell the developers what's in use and what's not. The reality outside of WMF wikis is that most lag several versions behind the current. Just browsing around, I easily found wikis between 1.25 and 1.33; I found none at 1.34 or above. So, deprecating something in 1.34 and then removing it in 1.35 or 1.36 because nobody complained or was logged as using the feature is not really a good plan for wikis like these. I think, if nothing else, there needs to be some kind of communication of planned deprecations/removals that allows extension developers who may not be at the current version to be made aware of breaking changes in advance and be able to say "Hey, we're still using this. We need a path forward."

Mon, May 10, 7:57 PM · MW-1.36-notes (1.36.0-wmf.8; 2020-09-08), MediaWiki-extensions-Variables, MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), Parsing-Active-Work, Patch-For-Review, Parsoid, MediaWiki-Parser
Legoktm added a comment to T281548: Delete lists-next.wikimedia.org.

We don't need a backup, it's all just emails of people saying "Test" etc. :P, I'll delete the VM tomorrow (Pacific Time) so it should be ready for DBA deletion on Wednesday :)

Mon, May 10, 7:00 PM · Patch-For-Review, DBA, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T236809: Refactor Parser.php to allow alternate parser (Parsoid).

@RobinHood70 I see that you are angry about changes in the parser that will break something your organization has developed, and that causes you extra work and uncertainty about the future. Is that right?

I am not part of the team that develops the parser, but I develop the Translate extension, which is also incompatible with the upcoming parser changes. The parser developers have been really helpful for redesigning it to work with the new version.

Mon, May 10, 6:53 PM · MW-1.36-notes (1.36.0-wmf.8; 2020-09-08), MediaWiki-extensions-Variables, MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), Parsing-Active-Work, Patch-For-Review, Parsoid, MediaWiki-Parser
Legoktm added a comment to T281711: Add approvals on Github for all the ORES-related repositories.

I took some extra steps:

  • added only ML-team members to the list of users able to push to the master branches of repos that are deployed to Production.
  • modified the Machine Learning team's Maintainers on github to reflect the structure of the actual team at Wikimedia.

We value and trust the community support, and we'll keep supporting them as much as possible. We need to be able to review all the changes that are landing to ORES and its models, so we'll get no surprises when deploying in production.

Mon, May 10, 6:22 AM · Machine-Learning-Team, ORES
Legoktm added a comment to T278857: "build: Updating y18n to 3.2.2" commits didn't all work properly.

What is npm installing on those repos? y18n doesn't show up as vulnerable on https://libraryupgrader2.wmcloud.org/vulns/npm for those repos...?

Mon, May 10, 6:14 AM · User-DannyS712, LibUp
Legoktm closed T278857: "build: Updating y18n to 3.2.2" commits didn't all work properly as Invalid.

Seems to have been fine? Please reopen if something is actually wrong.

Mon, May 10, 5:59 AM · User-DannyS712, LibUp

Sun, May 9

Legoktm added a project to T282341: Mailman 3: Months with no emails are still listed in the archive: Upstream.

I think this is https://gitlab.com/mailman/hyperkitty/-/issues/255

Sun, May 9, 9:58 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm closed T282366: Monitor mailman3 runner processes as Resolved.
14:45:23 <+icinga-wm> PROBLEM - mailman3_runners on lists1001 is CRITICAL: PROCS CRITICAL: 13 processes with UID = 38 (list), regex args /usr/lib/mailman3/bin/runner https://wikitech.wikimedia.org/wiki/Mailman/Monitoring
14:48:47 <+icinga-wm> RECOVERY - mailman3_runners on lists1001 is OK: PROCS OK: 14 processes with UID = 38 (list), regex args /usr/lib/mailman3/bin/runner https://wikitech.wikimedia.org/wiki/Mailman/Monitoring
Sun, May 9, 9:49 PM · SRE, observability, Wikimedia-Mailing-lists
Legoktm created T282393: LibUp: replace celery queues with MariaDB.
Sun, May 9, 8:25 PM · LibUp
Legoktm created P15868 (An Untitled Masterwork).
Sun, May 9, 7:19 PM
Legoktm added a comment to T282390: Dienstleisterberlinde's contributions on mediawikiwiki disappeared.

Dupe of T235687: Edits of deleted topic not shown in special:deletedcontribs/user neither in topic history?

Sun, May 9, 7:15 PM · StructuredDiscussions, Security
Legoktm added a comment to T282390: Dienstleisterberlinde's contributions on mediawikiwiki disappeared.

https://www.mediawiki.org/w/index.php?title=Special:AbuseLog&wpSearchUser=Dienstleisterberlinde shows spam attempts in Flow.

Sun, May 9, 7:14 PM · StructuredDiscussions, Security
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

I hack applied https://gitlab.com/mailman/mailman/-/merge_requests/811/, I think it'll fix the crash issue.

Sun, May 9, 6:33 PM · SRE, Wikimedia-Mailing-lists
Aklapper awarded T282348: Mailman3 bounce runner is running very slowly a Yellow Medal token.
Sun, May 9, 11:14 AM · SRE, Wikimedia-Mailing-lists
Nemo_bis awarded T282348: Mailman3 bounce runner is running very slowly a Doubloon token.
Sun, May 9, 11:00 AM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

Email validation: https://gitlab.com/mailman/mailman/-/issues/892

Sun, May 9, 10:30 AM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

The bounce runner died again because it tried to insert a too-big email into the bounceevent table (VARCHAR(255)):

Sun, May 9, 9:57 AM · SRE, Wikimedia-Mailing-lists
Legoktm created T282366: Monitor mailman3 runner processes.
Sun, May 9, 9:37 AM · SRE, observability, Wikimedia-Mailing-lists
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

I live hacked in the following patch:

diff --git a/src/mailman/model/member.py b/src/mailman/model/member.py
index 1adeb6e41..6f70f990f 100644
--- a/src/mailman/model/member.py
+++ b/src/mailman/model/member.py
@@ -240,7 +240,7 @@ class MembershipManager:
         # could have been reset due to bounce info getting stale. We will send
         # warnings to people who have been disabled already, regardless of
         # their bounce score. Same is true below for removal.
-        query = store.query(Member).join(
+        query = store.query(Member, MailingList.bounce_you_are_disabled_warnings_interval).join(
             MailingList, Member.list_id == MailingList._list_id).join(
             Member.preferences).filter(and_(
                 MailingList.process_bounces == True,       # noqa: E712
@@ -259,9 +259,8 @@ class MembershipManager:
         # func.DATETIME(MailingList.bounce_you_are_disabled_warnings_interval))
         # < func.DATETIME(now())))
Sun, May 9, 9:16 AM · SRE, Wikimedia-Mailing-lists
Legoktm renamed T282348: Mailman3 bounce runner is running very slowly from Mailman3 bounce runner not working to Mailman3 bounce runner is running very slowly.
Sun, May 9, 8:49 AM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.
>>> from mailman.interfaces.member import DeliveryStatus, IMembershipManager
>>> from zope.component import getUtility
>>> manager = getUtility(IMembershipManager)
>>> p=manager.memberships_pending_warning()
>>> import time
>>> a=time.time();x=list(p);b=time.time()
>>> b-a
26.814743757247925
Sun, May 9, 8:49 AM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

I tried to apply the patch from https://gitlab.com/mailman/mailman/-/merge_requests/811 but it made no difference.

Sun, May 9, 8:01 AM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

So the bounce runner is running! It's just super slow

Sun, May 9, 7:33 AM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282348: Mailman3 bounce runner is running very slowly.

https://gitlab.com/mailman/mailman/-/issues/755 suggests upgrading flufl.bounce to 3.0.1 might help, we're on 3.0.0

Sun, May 9, 6:52 AM · SRE, Wikimedia-Mailing-lists
Legoktm triaged T282348: Mailman3 bounce runner is running very slowly as Unbreak Now! priority.
Sun, May 9, 6:49 AM · SRE, Wikimedia-Mailing-lists
Legoktm created P15865 (An Untitled Masterwork).
Sun, May 9, 6:49 AM

Sat, May 8

Legoktm triaged T282308: Mailman3 templates with colons in filename made operations/puppet not cloneable on Windows as High priority.
Sat, May 8, 9:01 AM · SRE
Legoktm added a comment to T280731: Implement static redirects from pipermail archives to hyperkitty archives.

rOPUPfff57796d2a9: mailman3: Fix generating redirects for non UTF-8 messages avoids the unicode problem. Re-running it in a screen now.

Sat, May 8, 8:53 AM · Patch-For-Review, SRE, Wikimedia-Mailing-lists
Legoktm triaged T282307: Review and harden LibUp against CVE-2020-5252 style attacks as Medium priority.
Sat, May 8, 8:18 AM · LibUp
Ladsgroup awarded T280731: Implement static redirects from pipermail archives to hyperkitty archives a Love token.
Sat, May 8, 8:15 AM · Patch-For-Review, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282301: Move icinga-wm from #wikidata to #wikidata-feed.

For reference https://gerrit.wikimedia.org/g/operations/puppet/+/7bbfebf8e5c99a90923fc200f270a8654e384ca9/modules/profile/manifests/icinga/ircbot.pp#11 is what needs to be changed.

Sat, May 8, 8:09 AM · Wikidata, wdwb-tech, observability, SRE
Legoktm added a comment to T280731: Implement static redirects from pipermail archives to hyperkitty archives.

I enabled redirects for everything that didn't cause unicode errors, currently 102,189 individual emails. 10% there!

Sat, May 8, 8:08 AM · Patch-For-Review, SRE, Wikimedia-Mailing-lists
Legoktm committed rOSMTc3ca871539c8: Switch to Architecture: all (authored by Legoktm).
Switch to Architecture: all
Sat, May 8, 7:38 AM
Legoktm changed the status of T281548: Delete lists-next.wikimedia.org from Stalled to Open.

We can do this next week.

Sat, May 8, 7:10 AM · Patch-For-Review, DBA, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T280731: Implement static redirects from pipermail archives to hyperkitty archives.
legoktm@lists1001:/var/lib/mailman3/redirects$ sudo pipermail_redirects deutschschweiz --no-rebuild
Going through 2020-August...
Traceback (most recent call last):
  File "/usr/local/sbin/pipermail_redirects", line 141, in <module>
    sys.exit(main())
  File "/usr/local/sbin/pipermail_redirects", line 129, in main
    line = handle_email(listname, email)
  File "/usr/local/sbin/pipermail_redirects", line 76, in handle_email
    message_id = extract_in_reply_to(path)
  File "/usr/local/sbin/pipermail_redirects", line 52, in extract_in_reply_to
    text = path.read_text()
  File "/usr/lib/python3.7/pathlib.py", line 1200, in read_text
    return f.read()
  File "/usr/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 118: invalid start byte
Sat, May 8, 6:38 AM · Patch-For-Review, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T280731: Implement static redirects from pipermail archives to hyperkitty archives.

I enabled redirects for mediawiki-debian and mediawiki-distributors. So far these only work for individual messages for now. New lists will automatically be done by the migration script, I'll back fill the redirects later.

Sat, May 8, 1:44 AM · Patch-For-Review, SRE, Wikimedia-Mailing-lists
Legoktm created P15864 generate_mbox.py.
Sat, May 8, 1:36 AM
Legoktm added a project to T282271: daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name: Upstream.
Sat, May 8, 1:28 AM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282271: daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name.

I filed https://gitlab.com/mailman/mailman/-/issues/891 upstream

Sat, May 8, 1:25 AM · Upstream, SRE, Wikimedia-Mailing-lists

Fri, May 7

Legoktm added a comment to T282278: libup possible has issues with npm audit to get all vulnerability removed.

Is npm audit giving an return code when there is still something to fix?

Fri, May 7, 11:23 PM · Patch-For-Review, LibUp
Legoktm reopened T244241: Allow list admins to train spam filters as "Open".

There's some new interest in this, see https://lists.wikimedia.org/hyperkitty/list/listadmins@lists.wikimedia.org/message/I4NRRS23N3KMR7XH4GWYTKF72FI3Q4RB/

Fri, May 7, 11:07 PM · serviceops, Wikimedia-Mailing-lists, SRE
Legoktm added a comment to T282278: libup possible has issues with npm audit to get all vulnerability removed.

@Umherirrender Should we just run npm audit fix --only=dev in a loop like 2 or 3 times? I don't fully understand what's going wrong here.

Fri, May 7, 10:03 PM · Patch-For-Review, LibUp
Legoktm added a comment to T282278: libup possible has issues with npm audit to get all vulnerability removed.

Sigh, now we have https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GlobalWatchlist/+/686779 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/AbuseFilter/+/686777

Fri, May 7, 9:52 PM · Patch-For-Review, LibUp
Legoktm renamed T282271: daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name from daily-article-l import to Mailman3 failed to daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name.
Fri, May 7, 9:44 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282271: daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name.

This also affected the education@ list.

Fri, May 7, 9:44 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T281779: Find list owners for lists without them.

Saving this query for later:

mysql:mailman3@m5-master.eqiad.wmnet [mailman3]> select mailinglist.list_id from mailinglist where not exists(select * from member where member.list_id=mailinglist.list_id and (role=2));
+-------------------------------------------+
| list_id                                   |
+-------------------------------------------+
| ac-temp.lists.wikimedia.org               |
| fundraiser.lists.wikimedia.org            |
| helpdesk-l.lists.wikimedia.org            |
| iegcom.lists.wikimedia.org                |
| pressemeldungen.lists.wikimedia.org       |
| spcommittee-l.lists.wikimedia.org         |
| spcommittee-private-l.lists.wikimedia.org |
| test-second.lists.wikimedia.org           |
| wikifr-colloque-l.lists.wikimedia.org     |
| wikiid-l.lists.wikimedia.org              |
| wikimedia-ci.lists.wikimedia.org          |
| wikimediase-l.lists.wikimedia.org         |
| wikisv-l.lists.wikimedia.org              |
| wmf_community_dept.lists.wikimedia.org    |
+-------------------------------------------+
14 rows in set (0.009 sec)
Fri, May 7, 9:41 PM · Patch-For-Review, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282278: libup possible has issues with npm audit to get all vulnerability removed.

After checking out https://gerrit.wikimedia.org/r/c/mediawiki/extensions/AbuseFilter/+/686753 npm audit says I still have an issue:

Fri, May 7, 8:51 PM · Patch-For-Review, LibUp
Legoktm added a comment to T282279: Mailman 3: per-list language preferences don't work.

Poked in the logs a bit:

Fri, May 7, 8:38 PM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282278: libup possible has issues with npm audit to get all vulnerability removed.

The only thing I can think of is what we're running a different npm version? We're currently on 6.14.5. Latest 6.x is 6.14.13.

Fri, May 7, 8:23 PM · Patch-For-Review, LibUp
Legoktm added a comment to T282279: Mailman 3: per-list language preferences don't work.

Works for me, which list are you trying to do it for?

Fri, May 7, 8:22 PM · SRE, Wikimedia-Mailing-lists
Legoktm committed rLLICaaadedeaa263: Upgrade grunt to 1.4.0 on all branches (authored by Legoktm).
Upgrade grunt to 1.4.0 on all branches
Fri, May 7, 8:20 PM
Legoktm updated subscribers of T282271: daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name.

@jcrespo and I discussed this on #wikimedia-databases. In summary,

Fri, May 7, 7:36 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282271: daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name.
sqlalchemy.exc.InternalError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (pymysql.err.InternalError) (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\x8Dku...' for column `mailman3`.`user`.`display_name` at row 1") [SQL: 'UPDATE user SET display_name=%(display_name)s WHERE user.id = %(user_id)s'] [parameters: {'display_name': 'Priya😍kushwaha', 'user_id': 85452}] (Background on this error at: http://sqlalche.me/e/2j85)
Fri, May 7, 7:05 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm created T282271: daily-article-l@, education@ import to Mailman3 failed because of unicode characters in display name.
Fri, May 7, 6:58 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282264: Monitor certificate validity for Cloud VPS.

+1 to monitoring/alerting, besides the known acme-chief issue, there are other reasons getting a cert from LE could fail, so having an alert in case we have say, less than a week left would be good.

Fri, May 7, 6:21 PM · cloud-services-team (Kanban), Cloud-VPS
Legoktm closed T280744: Mail to root@lists1001.wikimedia.org from noreply@lists1001.wikimedia.org doesn't work as Resolved.

Triggered the unit and it successfully sent the email, yay!

Fri, May 7, 5:31 PM · Mail, SRE
Legoktm committed rOSMT5804e24d77bc: Add Debian packaging (authored by Legoktm).
Add Debian packaging
Fri, May 7, 5:13 PM
Legoktm committed rLLICdeb657810f0c: Add pywikibot/i18n (authored by Legoktm).
Add pywikibot/i18n
Fri, May 7, 5:10 PM
Legoktm created T282217: LibUp hits 30 minute timeout when processing Wikibase.
Fri, May 7, 8:29 AM · wdwb-tech, LibUp
Legoktm added a comment to T52864: Upgrade GNU Mailman from 2.1 to Mailman3.

Let's move the redirect conversation to T280731: Implement static redirects from pipermail archives to hyperkitty archives. I replied there.

Fri, May 7, 8:07 AM · Security-Team, SRE, Wikimedia-Mailing-lists
Legoktm updated subscribers of T280731: Implement static redirects from pipermail archives to hyperkitty archives.

the plan is to serve both mailman2 and mailman3 from the same server [...] FWIW the mailman mailing lists preserved their old pipermail archives and just left a note on top saying that the archives are no longer update

So the public HTML and txt.gz archives will keep being served indefinitely from the same URLs as before? That will reduce the pain considerably.

I mostly figured out how to redirect public HTML URLs, see T280731: Implement static redirects from pipermail archives to hyperkitty archives. Should get deployed tomorrow or early next week.

I don't think hyperkitty has an equivalent for the txt.gz files, so we can keep them around (by not deleting them) if there's value.

Fri, May 7, 8:07 AM · Patch-For-Review, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T52864: Upgrade GNU Mailman from 2.1 to Mailman3.

the plan is to serve both mailman2 and mailman3 from the same server [...] FWIW the mailman mailing lists preserved their old pipermail archives and just left a note on top saying that the archives are no longer update

So the public HTML and txt.gz archives will keep being served indefinitely from the same URLs as before? That will reduce the pain considerably.

Fri, May 7, 7:25 AM · Security-Team, SRE, Wikimedia-Mailing-lists
Legoktm committed rLLIC5cb73d6cffc7: Add wikidata/query-builder (authored by Michael).
Add wikidata/query-builder
Fri, May 7, 7:15 AM
Legoktm added a comment to T280744: Mail to root@lists1001.wikimedia.org from noreply@lists1001.wikimedia.org doesn't work.

I copied the code out of systemd-timer-mail-wrapper and ran it interactively to see if I could get it to work.

Fri, May 7, 12:48 AM · Mail, SRE
Legoktm renamed T280744: Mail to root@lists1001.wikimedia.org from noreply@lists1001.wikimedia.org doesn't work from Mail to root@lists1001.wikimedia.org doesn't work because of /etc/aliases file permissions to Mail to root@lists1001.wikimedia.org from noreply@lists1001.wikimedia.org doesn't work.
Fri, May 7, 12:44 AM · Mail, SRE
Legoktm added a reverting change for rLLICeac20dba4845: Temporarily disable LibUp, pushing dupes: rLLICcf6864f4f135: Revert "Temporarily disable LibUp, pushing dupes".
Fri, May 7, 12:06 AM
Legoktm committed rLLICcf6864f4f135: Revert "Temporarily disable LibUp, pushing dupes" (authored by Legoktm).
Revert "Temporarily disable LibUp, pushing dupes"
Fri, May 7, 12:06 AM
Legoktm added a comment to T280438: libup committed and uploaded an upgrade twice for the same package on the same repo.

I tried to figure out how to introduce locks into the queue and eventually gave up. Instead I think we can avoid duplicate pushes by

Fri, May 7, 12:01 AM · Patch-For-Review, LibUp

Thu, May 6

Legoktm closed T282092: Upload new mailman3 and hyperkitty packages as Resolved.

Upgraded both Cloud VPS instances too.

Thu, May 6, 11:50 PM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T281987: improve new mailing list admin notifications.

And because of rOPUP9b8147775d1e: mailman3: Change owner email from root@ to listadmins-owner the emails will now come from listadmins-owner@lists.wikimedia.org, which should be a bit better. I'll file an upstream task for allowing us to put a name on the sender and maybe have them come from noreply@

Thu, May 6, 10:04 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T281987: improve new mailing list admin notifications.

Here's the patch I've hacked in for now:

Thu, May 6, 9:49 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm updated the task description for T282092: Upload new mailman3 and hyperkitty packages.
Thu, May 6, 6:42 PM · SRE, Wikimedia-Mailing-lists
Legoktm committed rOSMT355809ae9ecc: Fix URLs getting line wrapped (authored by Legoktm).
Fix URLs getting line wrapped
Thu, May 6, 6:34 PM
Legoktm closed T282044: Mailman3 "New subscription request to" template line wraps, breaking long links as Resolved.
Thu, May 6, 6:34 PM · SRE, Upstream, Wikimedia-Mailing-lists
Legoktm added a comment to T282044: Mailman3 "New subscription request to" template line wraps, breaking long links.

Thu, May 6, 6:27 PM · SRE, Upstream, Wikimedia-Mailing-lists
Legoktm added a parent task for T282022: Add mailman-templates to translatewiki.net: T282018: Make customized Mailman3 templates translatable.
Thu, May 6, 8:49 AM · SRE, Wikimedia-Mailing-lists, translatewiki.net
Legoktm added a subtask for T282018: Make customized Mailman3 templates translatable: T282022: Add mailman-templates to translatewiki.net.
Thu, May 6, 8:49 AM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282044: Mailman3 "New subscription request to" template line wraps, breaking long links.

Apparently if the URL is on a line that starts with whitespace, it won't get wrapped. I didn't test this though.

Thu, May 6, 8:48 AM · SRE, Upstream, Wikimedia-Mailing-lists
Legoktm created T282092: Upload new mailman3 and hyperkitty packages.
Thu, May 6, 8:46 AM · SRE, Wikimedia-Mailing-lists
Legoktm closed T281753: In Mailman3 if a list has no owners, mail goes to root@ as Resolved.

Aaaand now it's going to listadmins-owner@ because of rOPUP9b8147775d1e: mailman3: Change owner email from root@ to listadmins-owner? I guess this works out. Why Mailman2 emails are being sent elsewhere because of a change to the Mailman3 configuration I'll never understand.

Thu, May 6, 8:25 AM · User-Ladsgroup, Mail, SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T282071: Error in qcluster.

Looks like we need to cherry-pick https://gitlab.com/mailman/hyperkitty/-/commit/2712722da7608c42e54ac73a392edb8673de9c4f ?

Thu, May 6, 6:03 AM · SRE, Wikimedia-Mailing-lists

Wed, May 5

Legoktm committed rLCSHd1e88036b97b: Add wvui to codesearch under libraries (authored by DannyS712).
Add wvui to codesearch under libraries
Wed, May 5, 9:32 PM
Legoktm created P15789 markBlocked.
Wed, May 5, 9:08 PM
Legoktm added a project to T282044: Mailman3 "New subscription request to" template line wraps, breaking long links: Upstream.

This is https://gitlab.com/mailman/mailman/-/issues/697

Wed, May 5, 7:34 PM · SRE, Upstream, Wikimedia-Mailing-lists
Legoktm added a comment to T281987: improve new mailing list admin notifications.

I filed https://gitlab.com/mailman/mailman/-/issues/890

Wed, May 5, 7:17 PM · Upstream, SRE, Wikimedia-Mailing-lists
Legoktm created T282044: Mailman3 "New subscription request to" template line wraps, breaking long links.
Wed, May 5, 7:09 PM · SRE, Upstream, Wikimedia-Mailing-lists
Legoktm closed T281933: Mailman3 import script is unnecessarily truncating list descriptions as Resolved.
Wed, May 5, 7:08 PM · SRE, Wikimedia-Mailing-lists
Legoktm added a comment to T280438: libup committed and uploaded an upgrade twice for the same package on the same repo.

Yeah, xSavitar pinged me on IRC so I disabled pushing for now :( Will try to get it re-enabled tonight.

Wed, May 5, 6:41 PM · Patch-For-Review, LibUp
Legoktm added a comment to T282022: Add mailman-templates to translatewiki.net.

I proposed some qqq in https://gerrit.wikimedia.org/r/c/operations/software/mailman-templates/+/685533/ - let me know if that's sufficient or needs improvement.

Wed, May 5, 5:30 PM · SRE, Wikimedia-Mailing-lists, translatewiki.net
Legoktm updated the task description for T282022: Add mailman-templates to translatewiki.net.
Wed, May 5, 4:47 PM · SRE, Wikimedia-Mailing-lists, translatewiki.net
Legoktm created T282022: Add mailman-templates to translatewiki.net.
Wed, May 5, 4:37 PM · SRE, Wikimedia-Mailing-lists, translatewiki.net
Legoktm committed rOSMTb97373deda10: Initial commit (authored by Legoktm).
Initial commit
Wed, May 5, 4:36 PM
Legoktm created T282018: Make customized Mailman3 templates translatable.
Wed, May 5, 4:18 PM · SRE, Wikimedia-Mailing-lists
Legoktm added a project to T281987: improve new mailing list admin notifications: Upstream.

Thanks, this is https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/commands/cli_notify.py#L134 which is not customizable via the template system (there's a TODO for it), so it'll need an upstream fix. We can probably patch it in manually in the meantime.

Wed, May 5, 4:10 PM · Upstream, SRE, Wikimedia-Mailing-lists