Change #1206894 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] fix(MigrateMentorStatusAway): ensure migration respects date format

https://gerrit.wikimedia.org/r/1206894

Change #1207118 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@wmf/1.46.0-wmf.3] fix(MigrateMentorStatusAway): ensure migration respects date format

https://gerrit.wikimedia.org/r/1207118

Change #1207118 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.46.0-wmf.3] fix(MigrateMentorStatusAway): ensure migration respects date format

https://gerrit.wikimedia.org/r/1207118

Mentioned in SAL (#wikimedia-operations) [2025-11-19T14:30:50Z] <sgimeno@deploy2002> Started scap sync-world: Backport for [[gerrit:1206936|fix(ReviseToneExperimentInteractionLogger): prevent breaking homepage for unsampled users (T405177)]], [[gerrit:1207118|fix(MigrateMentorStatusAway): ensure migration respects date format (T409170)]]

Mentioned in SAL (#wikimedia-operations) [2025-11-19T14:35:28Z] <sgimeno@deploy2002> sgimeno: Backport for [[gerrit:1206936|fix(ReviseToneExperimentInteractionLogger): prevent breaking homepage for unsampled users (T405177)]], [[gerrit:1207118|fix(MigrateMentorStatusAway): ensure migration respects date format (T409170)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-11-19T14:40:00Z] <sgimeno@deploy2002> Finished scap sync-world: Backport for [[gerrit:1206936|fix(ReviseToneExperimentInteractionLogger): prevent breaking homepage for unsampled users (T405177)]], [[gerrit:1207118|fix(MigrateMentorStatusAway): ensure migration respects date format (T409170)]] (duration: 09m 09s)

Not sure why mwscript-k8s is stopping at afwiki when I run this:

sgimeno@deploy2002:~$ mwscript-k8s -f --comment="T409170" --dblist="growthexperiments" -- GrowthExperiments:migrateMentorStatusAway --dry-run
GrowthExperiments:migrateMentorStatusAway: Running on growthexperiments
-----------------------------------------------------------------
abwiki
-----------------------------------------------------------------
abwiki No mentors found in config, skipping migration.
-----------------------------------------------------------------
acewiki
-----------------------------------------------------------------
acewiki No mentors found in config, skipping migration.
-----------------------------------------------------------------
adywiki
-----------------------------------------------------------------
adywiki No mentors found in config, skipping migration.
-----------------------------------------------------------------
afwiki
-----------------------------------------------------------------
afwiki Nothing new to save, skipping

@Urbanecm_WMF is helping to figure out why.

In T409170#11406385, @Sgs wrote:

@Urbanecm_WMF is helping to figure out why.

The problem is described in T411104: mwscript-k8s makes it challenging to run LoggedUpdateMaintenance across all wikis, Daniel Kinzler uploaded fixes to Core and GrowthExperiments, which need to be reviewed.

@Sgs also noticed that on kaawiki, the script errors out with:

kaawiki [9360699038247e008416e081] [no req]   TypeError: array_key_exists(): Argument #2 ($array) must be of type array, null given

This is happening, because an administrator manually edited the list of mentors to add themselves, and they used a non-existing user ID. Fortunately, that wiki does not have mentorship enabled in CC. I went ahead and fixed the mentor list, and I filled T412187: Help kaa.wikipedia to enable mentorship to check on the wiki (as it seems they attempted to enable mentorship, but unsuccessfully).

The change in T411104 has been merged, so this should be ready to be worked on again.

Mentioned in SAL (#wikimedia-operations) [2026-06-09T15:22:45Z] <urbanecm> Remove migrateMentorStatusAwayToCommunityConfiguration from updatelog on all wikis (T409170; the script was only ever run as a dry-run)

Change #1299540 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] migrateMentorStatusAway: ensure validateStrictly receives objects

https://gerrit.wikimedia.org/r/1299540

Change #1299540 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] migrateMentorStatusAway: ensure validateStrictly receives objects

https://gerrit.wikimedia.org/r/1299540

I believe this now needs to be actually executed (once wmf.7 is on all wikis, or we need to backport it).

Change #1302100 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@wmf/1.47.0-wmf.6] migrateMentorStatusAway: ensure validateStrictly receives objects

https://gerrit.wikimedia.org/r/1302100

Change #1302100 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.47.0-wmf.6] migrateMentorStatusAway: ensure validateStrictly receives objects

https://gerrit.wikimedia.org/r/1302100

Mentioned in SAL (#wikimedia-operations) [2026-06-15T13:35:26Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1300835|Remove no longer used product_metrics.homepage_module_interaction (T365889 T426742)]], [[gerrit:1302153|TaskSuggester: avoid nullable logger in setLogger call]], [[gerrit:1302100|migrateMentorStatusAway: ensure validateStrictly receives objects (T409170)]], [[gerrit:1301451|Store nowiki source in StripState::extra to support subst-nowiki (T3

Mentioned in SAL (#wikimedia-operations) [2026-06-15T13:37:13Z] <jforrester@deploy1003> arlolra, sgimeno, jforrester: Backport for [[gerrit:1300835|Remove no longer used product_metrics.homepage_module_interaction (T365889 T426742)]], [[gerrit:1302153|TaskSuggester: avoid nullable logger in setLogger call]], [[gerrit:1302100|migrateMentorStatusAway: ensure validateStrictly receives objects (T409170)]], [[gerrit:1301451|Store nowiki source in StripState::extra to support subst-nowik

Mentioned in SAL (#wikimedia-operations) [2026-06-15T13:43:22Z] <jforrester@deploy1003> Finished scap sync-world: Backport for [[gerrit:1300835|Remove no longer used product_metrics.homepage_module_interaction (T365889 T426742)]], [[gerrit:1302153|TaskSuggester: avoid nullable logger in setLogger call]], [[gerrit:1302100|migrateMentorStatusAway: ensure validateStrictly receives objects (T409170)]], [[gerrit:1301451|Store nowiki source in StripState::extra to support subst-nowiki (T

I run the script again today with dry-mode and the output (P94151) was unexpected in two ways:

1. Many wikis showed as already logged, but @Urbanecm_WMF had removed it from updatelog in https://sal.toolforge.org/log/LXj6rJ4BffdvpiTrm2ev, I may have run the script again after that, but only with dry-mode
2. The script didn't finalize but end up hanging

I haven't reviewed the details from T411104 but I was expecting to not have the script marked as logged in (1) and finalize normally (2). This is not blocking to perform the migration per-se I can use the force flag.

Regarding the changes to be stored I haven't reviewed it's correctness from it in detail, the values to save are correct in value and format but I was expecting less timestamps with the old format (Unix timestamp) vs the new (ISO format). I will review that and post and update here.

Mentioned in SAL (#wikimedia-operations) [2026-06-16T15:38:45Z] <urbanecm> Remove migrateMentorStatusAwayToCommunityConfiguration from updatelog on all wikis in growthexperiments.dblist (T409170)

Mentioned in SAL (#wikimedia-operations) [2026-06-16T15:42:44Z] <urbanecm@deploy1003> mwscript-k8s job started: GrowthExperiments:migrateMentorStatusAway --wiki=abwiki --dry-run # T409170

Change #1302902 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] migrateMentorStatusAway: Return SIMULATED for all dry-run executions

https://gerrit.wikimedia.org/r/1302902

@Sgs and I discussed this during our 1:1. The problem was that the script was returning LoggedUpdateOutcome::COMPLETE even on dry-run, meaning it got logged as finished in updatelog. I removed all records of its execution again (logs are available at P94199). I also uploaded a patch that changes the script's behavior to always return SIMULATED on dry-runs.

Regarding viwiki, this seems to be expected. The mentor set their away date for a year in advance on August 15, 2025 (diff), which was likely before we deployed the format change. @Sgs will investigate whether this is the case.

Change #1302902 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] migrateMentorStatusAway: Return SIMULATED for all dry-run executions

https://gerrit.wikimedia.org/r/1302902

Change #1303364 had a related patch set uploaded (by Sergio Gimeno; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.47.0-wmf.7] migrateMentorStatusAway: Return SIMULATED for all dry-run executions

https://gerrit.wikimedia.org/r/1303364

Change #1303365 had a related patch set uploaded (by Sergio Gimeno; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.47.0-wmf.6] migrateMentorStatusAway: Return SIMULATED for all dry-run executions

https://gerrit.wikimedia.org/r/1303365

Change #1303365 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.47.0-wmf.6] migrateMentorStatusAway: Return SIMULATED for all dry-run executions

https://gerrit.wikimedia.org/r/1303365

Change #1303364 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.47.0-wmf.7] migrateMentorStatusAway: Return SIMULATED for all dry-run executions

https://gerrit.wikimedia.org/r/1303364

Mentioned in SAL (#wikimedia-operations) [2026-06-17T20:10:58Z] <sgimeno@deploy1003> Started scap sync-world: Backport for [[gerrit:1303365|migrateMentorStatusAway: Return SIMULATED for all dry-run executions (T409170)]], [[gerrit:1303364|migrateMentorStatusAway: Return SIMULATED for all dry-run executions (T409170)]]

Mentioned in SAL (#wikimedia-operations) [2026-06-17T20:12:57Z] <sgimeno@deploy1003> sgimeno: Backport for [[gerrit:1303365|migrateMentorStatusAway: Return SIMULATED for all dry-run executions (T409170)]], [[gerrit:1303364|migrateMentorStatusAway: Return SIMULATED for all dry-run executions (T409170)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-06-17T20:17:53Z] <sgimeno@deploy1003> Finished scap sync-world: Backport for [[gerrit:1303365|migrateMentorStatusAway: Return SIMULATED for all dry-run executions (T409170)]], [[gerrit:1303364|migrateMentorStatusAway: Return SIMULATED for all dry-run executions (T409170)]] (duration: 06m 55s)

In T409170#12024716, @Urbanecm_WMF wrote:

@Sgs and I discussed this during our 1:1. The problem was that the script was returning LoggedUpdateOutcome::COMPLETE even on dry-run, meaning it got logged as finished in updatelog. I removed all records of its execution again (logs are available at P94199). I also uploaded a patch that changes the script's behavior to always return SIMULATED on dry-runs.

I run the script again in dry-mode (logs P94276) and it did run for all wikis this time, none was marked as already logged. Thank you for helping with this operation. The hang suspicion was also my bad, looking at non-terminating tail -f, so nothing to worry on script termination.

Regarding viwiki, this seems to be expected. The mentor set their away date for a year in advance on August 15, 2025 (diff), which was likely before we deployed the format change. @Sgs will investigate whether this is the case.

The change that started writing in config rEGRE8bfc6b50e7e823396f42c304a3632065bc69af01 made it to all wikis by Aug 14th 2025 as shown in MediaWiki_1.45/wmf.14 so edits on the 15th should already carry the new format. I've also checked some more edits which show the same behavior (diff, diff, diff, diff). Unless I'm missing some later change that would amend a a flaw with format it's seems something did not work as expected for some time. Also strange that I cannot find flawed recent timestamps. Maybe something else changed, eg, the recent addition of the schema. On the bright side of things both formats are ConvertibleTimestamp compatible so we could proceed and file a separate task for identifying the format discrepancy. Does this make sense? Should we try to identify the source of the discrepancy before running the script for real or move on and fix this later? cc @Urbanecm_WMF @Michael

Another flaw I've identified is for an itwiki mentor https://phabricator.wikimedia.org/P94276#L2795 where the system is trying to copy longer period than one year ahead. Wouldn't yield validation error but maybe we want to clean it before or after. The user property exists:

MariaDB [itwiki]> select * from user_properties where up_user = 2048534 and up_property like '%mentor%';
+---------+------------------------------------------------+----------------+
| up_user | up_property                                    | up_value       |
+---------+------------------------------------------------+----------------+
| 2048534 | growthexperiments-mentor-away-timestamp        | 20300607130641 |
| 2048534 | growthexperiments-mentor-dashboard-last-update | 20260618091803 |
| 2048534 | growthexperiments-mentor-dashboard-seen        | 1              |
| 2048534 | growthexperiments-mentorship-weight            | 1              |
+---------+------------------------------------------------+----------------+
4 rows in set (0.001 sec)

In T409170#12033264, @Sgs wrote:

In T409170#12024716, @Urbanecm_WMF wrote:

Regarding viwiki, this seems to be expected. The mentor set their away date for a year in advance on August 15, 2025 (diff), which was likely before we deployed the format change. @Sgs will investigate whether this is the case.

The change that started writing in config rEGRE8bfc6b50e7e823396f42c304a3632065bc69af01 made it to all wikis by Aug 14th 2025 as shown in MediaWiki_1.45/wmf.14 so edits on the 15th should already carry the new format. I've also checked some more edits which show the same behavior (diff, diff, diff, diff). Unless I'm missing some later change that would amend a a flaw with format it's seems something did not work as expected for some time. Also strange that I cannot find flawed recent timestamps. Maybe something else changed, eg, the recent addition of the schema. On the bright side of things both formats are ConvertibleTimestamp compatible so we could proceed and file a separate task for identifying the format discrepancy. Does this make sense? Should we try to identify the source of the discrepancy before running the script for real or move on and fix this later? cc @Urbanecm_WMF @Michael

Hmm... Indeed, T396375 says it got deployed on the 14th. Unless we're getting unexpected values even now, I don't think we need to spend time figuring this out.

Another flaw I've identified is for an itwiki mentor https://phabricator.wikimedia.org/P94276#L2795 where the system is trying to copy longer period than one year ahead. Wouldn't yield validation error but maybe we want to clean it before or after.

I would suggest to manually set it to 1 year from now, and alerting that community. CC @Trizek-WMF for any potential comms implications here and @KStoller-WMF for awareness. I'm happy to perform that action if we decide on it.

Mentioned in SAL (#wikimedia-operations) [2026-06-19T10:29:12Z] <sergi0> Run MigrateMentorStatusAway script for all wikis in growthexperiments dblist - T409170

I run the script in normal mode (P94280, P94281), the execution went well until viwiki which has a couple of mentors to migrate and returned the error:

viwiki Error: This action has been automatically identified as harmful, and therefore disallowed.
viwiki If you believe your action was constructive, please inform an administrator of what you were trying to do.
viwiki A brief description of the abuse rule which your action matched is: Spam

It seems we're triggering an AbuseFilter rule, see AbuseLog. I'm not sure though what is the offending content, I haven't seen anything obvious like a URL but I'm not sure where to check the triggering rules. I guess we need to coordinate with viwiki admin to perform this operation (?), be it we find the offending content, or temporarily set the filter to warn only, or exempt Maintenance script account? What would you recommend @Urbanecm_WMF ?

Mentioned in SAL (#wikimedia-operations) [2026-06-19T12:04:27Z] <urbanecm@deploy1003> mwscript-k8s job started: GrowthExperiments:MigrateMentorStatusAway --wiki=viwiki # T409170

Mentioned in SAL (#wikimedia-operations) [2026-06-19T12:05:13Z] <urbanecm@deploy1003> mwscript-k8s job started: GrowthExperiments:migrateMentorStatusAway.php --wiki=viwiki # T409170

The rules are hidden from public view, so you'd need someone with sufficient on-wiki permissions to view them (or using direct database access). I checked them, and this rule exempts users with at least 20 edits (I've copied the rule to P94285 as a WMF-NDA paste for easier access). Maintenance script had 19 viwiki edits, so I used edit.php to make an edit to its sandbox (and immediately deleted it), which should be enough to make the script pass.

After this, I've run the script again, and it passed:

[urbanecm@deploy1003 ~]$ mwscript-k8s -f --sal --comment=T409170 -- GrowthExperiments:migrateMentorStatusAway.php --wiki=viwiki
⏳ Starting GrowthExperiments:migrateMentorStatusAway.php on Kubernetes as job mw-script.eqiad.7ikejfad ...
🚀 Job is running.
📜 Streaming logs:
Saved!
[urbanecm@deploy1003 ~]$

It might make sense to coordinate with viwiki to exempt bots, but given Maintenance script is now resolved for once and for good, maybe it's not that important anymore.