Page MenuHomePhabricator

Convert dumps repo python scripts to python3
Closed, ResolvedPublic

Description

Sad but true, it needs to get done.

See also T210980 which is related.

Event Timeline

ArielGlenn triaged this task as Medium priority.Dec 3 2018, 9:37 AM
ArielGlenn created this task.

Change 477227 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/dumps@python3] start conversion to python3

https://gerrit.wikimedia.org/r/477227

Change 478702 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/dumps@master] convert dump scripts to python3

https://gerrit.wikimedia.org/r/478702

These have been tested: all the shell scripts in the repo, all python scripts with an entry point, with both checkpoint and regular dump setups. I plan to deploy this weekend; it will have to go at the same time as some puppet changes to call at least one of these scripts using python3.

Change 479985 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] switch all invocations of scripts in dumps rep to use python3

https://gerrit.wikimedia.org/r/479985

Change 477227 abandoned by ArielGlenn:
start conversion to python3

Reason:
squashed into later commit

https://gerrit.wikimedia.org/r/477227

I plan to push these to commits out on all snapshot hosts Monday in the middle of the day, when the only thing running will be the wikidata json weeklies script, and its invocation of python will have long since been done and the results used in the remainder of the script. This assumes that the wikidata xml/sql dumps currently running will have completed by then; the last four 7z files are being produced now, so we're on track for that.

The window of opportunity is from about 09:00 to 15:00 UTC time.

Change 478702 merged by ArielGlenn:
[operations/dumps@master] convert dump scripts to python3

https://gerrit.wikimedia.org/r/478702

Change 479985 merged by ArielGlenn:
[operations/puppet@production] switch all invocations of scripts in dumps rep to use python3

https://gerrit.wikimedia.org/r/479985

Leaving this ticket open until one of the big wikis finishes its dump and we see that everything is good.

Change 481260 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/dumps@master] no itervalues in python3

https://gerrit.wikimedia.org/r/481260

Change 481260 merged by ArielGlenn:
[operations/dumps@master] no itervalues in python3

https://gerrit.wikimedia.org/r/481260

Dumps have run with the converted code just fine, so I'm closing this.