Mentioned in SAL (#wikimedia-operations) [2024-10-24T17:04:52Z] <urbanecm> mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=nowiki (running as mw-script.codfw.ui7285yu; T376749)

I executed the script for nowiki. Logs:

This should be all done for nowiki!

Started checking the first 120 pages on the log. @EAkinloose will check the other 120 pages.
Progress can be found here: https://docs.google.com/spreadsheets/d/1PSusVBqaKtjiox9nsmQhj8y-cFLI9o2M2vx8JzzFkcQ/edit?gid=0#gid=0

There are some Flow pages that are moved to a sub-page that do not seem to have any Move notice and also the pages from which these Flow boards were moved do not have DiscussionTools enabled on them.

In T376749#10264513, @Ryasmeen wrote:

Progress can be found here: https://docs.google.com/spreadsheets/d/1PSusVBqaKtjiox9nsmQhj8y-cFLI9o2M2vx8JzzFkcQ/edit?gid=0#gid=0

Does this have to be an access restricted Google Doc, or could community members take a look?

In T376749#10264772, @Aklapper wrote:

In T376749#10264513, @Ryasmeen wrote:

Progress can be found here: https://docs.google.com/spreadsheets/d/1PSusVBqaKtjiox9nsmQhj8y-cFLI9o2M2vx8JzzFkcQ/edit?gid=0#gid=0

Does this have to be an access restricted Google Doc, or could community members take a look?

Thank you for raising this question, @Aklapper. For now, we're going to keep this doc private simply because we're working to meet a deadline and would like to defer the effort of reviewing the doc to ensure all the info. within it is fit for public view.

With the above said, if/when someone expresses a need to see the doc, we're happy to prioritize the clean-up necessary to make it visible.

Mentioned in SAL (#wikimedia-operations) [2024-10-31T23:41:15Z] <urbanecm> Run extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php for several wikis (T376749; wiki list is on task)

I'm running the script now per Editing request. Progress below:

• commonswiki (NO OUTPUT)
• cswiki (OK)
• enwiki (ERRORED OUT)
• eowiki (NO MOVES)
• frwiktionary
• hewiki
• kkwiki
• metawiki (ERRORED OUT)
• pawiki
• ruwiki (ERRORED OUT)
• specieswiki
• svwiki
• test2wiki
• tewiki
• outreachwiki

Highlights

I noticed some weird things while running the script. I note them below, full logs are linked near the bottom of my comment.

enwiki run errored out:

[urbanecm@deploy2002 ~/flow-logs]$ ./run.sh enwiki --dry-run
⏳ Starting extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php on Kubernetes as job mw-script.codfw.jnq84brc ...
⏳ Waiting for the container to start...
🚀 Job is running.
📜 Streaming logs:
The "Flow" extension must be installed for this script to run. Please enable it and then try again.
[urbanecm@deploy2002 ~/flow-logs]$

Flow is also not present at metawiki or ruwiki.

For commons, the script produced no output at all:

[urbanecm@deploy2002 ~/flow-logs]$ ./run.sh commonswiki --dry-run
⏳ Starting extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php on Kubernetes as job mw-script.codfw.k0wn6hmu ...
⏳ Waiting for the container to start...
🚀 Job is running.
📜 Streaming logs:


[urbanecm@deploy2002 ~/flow-logs]$

Although the dry run didn't print any results, it did take non-trivial time to finish:

[urbanecm@deploy2002 ~/flow-logs]$ time mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=commonswiki
⏳ Starting extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php on Kubernetes as job mw-script.codfw.eaq566fp ...
⏳ Waiting for the container to start...
🚀 Job is running.
📜 Streaming logs:

real    0m59.116s
user    0m5.823s
sys     0m0.656s
[urbanecm@deploy2002 ~/flow-logs]$

Is this OK?

Logs

I published complete logs of the script runs as Phabricator pastes, see links below:

commonswiki.log https://phabricator.wikimedia.org/P70811
cswiki.log https://phabricator.wikimedia.org/P70812
enwiki.log https://phabricator.wikimedia.org/P70813
eowiki.log https://phabricator.wikimedia.org/P70814
frwiktionary.log https://phabricator.wikimedia.org/P70815
hewiki.log https://phabricator.wikimedia.org/P70816
kkwiki.log https://phabricator.wikimedia.org/P70817
metawiki.log https://phabricator.wikimedia.org/P70818
outreachwiki.log https://phabricator.wikimedia.org/P70819
pawiki.log https://phabricator.wikimedia.org/P70820
ruwiki.log https://phabricator.wikimedia.org/P70821
specieswiki.log https://phabricator.wikimedia.org/P70822
svwiki.log https://phabricator.wikimedia.org/P70823
svwik.log https://phabricator.wikimedia.org/P70824
test2wiki.log https://phabricator.wikimedia.org/P70825
tewiki.log https://phabricator.wikimedia.org/P70826

run.sh

For transparency, here is run.sh:

[urbanecm@deploy2002 ~/flow-logs]$ cat run.sh 
#!/bin/bash

if [ "$#" -eq 0 ]; then
        echo "ERROR: First argument needs to be the wiki name"
        exit 1
fi

WIKI=$1
shift

echo "$ mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=$WIKI $@" > $WIKI.log
mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=$WIKI $@ 2>&1 | tee -a $WIKI.log
[urbanecm@deploy2002 ~/flow-logs]$

So, checking in to this...

These wikis were disabled by removing them from flow.dblist in our config repo on the mentioned dates. The effect of this is that the Flow extension isn't even loaded on those wikis, and so they've been completely unable to display any hypothetical existing Flow boards remaining since then. (But I'd assume that manual cleanup happened before we turned them off.)

commonswiki... the initial queries for number-of-flow-boards thinks that there should be 2 boards on that wiki, but they could potentially not be attached to existing pages which would produce that result. As for it taking so long... we're doing a query on page where page_content_model is CONTENT_MODEL_FLOW_BOARD. Since I don't think page_content_model has any index, I'd assume it's just that commons has a large page table to scan through. If enwiki hadn't errored out early because of Flow not being enabled, I'd assume it'd have taken a fair while as well -- though commons has twice as many pages, so it's probably the single worst-case wiki for that query.