Page MenuHomePhabricator

Use Python 3 for all performance team projects
Closed, ResolvedPublic

Description

Python 2 is end-of-life; we should be using Python 3 instead.

Most services with CI are already being tested using Python 3. The bulk of our projects simply need /usr/bin/python3 as the interpreter and Python 3 versions of libraries installed (python3-xxx instead of python-xxx).

This is already in progress as a KR for this quarter; opening a task for it as well to make it easier to group together the related patches.

Event Timeline

dpifke created this task.Wed, Nov 4, 8:58 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptWed, Nov 4, 8:58 PM
dpifke moved this task from Inbox to Doing on the Performance-Team board.Wed, Nov 4, 8:58 PM

Change 639216 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[operations/puppet@production] [WIP] webperf: convert statsv to use Python 3

https://gerrit.wikimedia.org/r/639216

Change 639197 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[operations/puppet@production] [WIP] webperf: change navtiming to use Python 3

https://gerrit.wikimedia.org/r/639197

Change 639223 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[analytics/statsv@master] Make compatible with Python 3

https://gerrit.wikimedia.org/r/639223

Change 639885 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[operations/puppet@production] arclamp: Use Python 3

https://gerrit.wikimedia.org/r/639885

Change 639886 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/arc-lamp@master] Use Python 3 interpreter

https://gerrit.wikimedia.org/r/639886

Change 640226 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[operations/puppet@production] [WIP] coal: Use Python 3

https://gerrit.wikimedia.org/r/640226

Change 640227 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/coal@master] [WIP] Use Python 3

https://gerrit.wikimedia.org/r/640227

Change 640227 merged by jenkins-bot:
[performance/coal@master] Use Python 3

https://gerrit.wikimedia.org/r/640227

Change 639197 merged by Jbond:
[operations/puppet@production] webperf: change navtiming to use Python 3

https://gerrit.wikimedia.org/r/639197

Change 639885 merged by Jbond:
[operations/puppet@production] arclamp: Use Python 3

https://gerrit.wikimedia.org/r/639885

Change 640226 merged by Jbond:
[operations/puppet@production] coal: use Python 3

https://gerrit.wikimedia.org/r/640226

Change 639216 merged by Jbond:
[operations/puppet@production] webperf: convert statsv to use Python 3

https://gerrit.wikimedia.org/r/639216

Change 639886 merged by jenkins-bot:
[performance/arc-lamp@master] Use Python 3 interpreter

https://gerrit.wikimedia.org/r/639886

Change 639223 merged by Ottomata:
[analytics/statsv@master] Make compatible with Python 3

https://gerrit.wikimedia.org/r/639223

Change 641464 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/navtiming@master] Convert message from bytes to str

https://gerrit.wikimedia.org/r/641464

Change 641464 merged by jenkins-bot:
[performance/navtiming@master] Convert message from bytes to str

https://gerrit.wikimedia.org/r/641464

dpifke closed this task as Resolved.Tue, Nov 17, 6:05 PM

All Python processes on webperf[12]00[12] are now /usr/bin/python3, and appear to be healthy.

dpifke reopened this task as Open.Tue, Nov 17, 6:11 PM

Scratch that, looks like I missed another bytes/str issue with ArcLamp. Fixing now.

Change 641474 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/arc-lamp@master] Ignore encoding when writing logs

https://gerrit.wikimedia.org/r/641474

Change 641474 merged by jenkins-bot:
[performance/arc-lamp@master] Ignore encoding when writing logs

https://gerrit.wikimedia.org/r/641474

Mentioned in SAL (#wikimedia-operations) [2020-11-18T22:56:01Z] <dpifke@deploy1001> Started deploy [performance/arc-lamp@6bbac6d]: Fix for bytes/str issue after T267269

Mentioned in SAL (#wikimedia-operations) [2020-11-18T22:56:08Z] <dpifke@deploy1001> Finished deploy [performance/arc-lamp@6bbac6d]: Fix for bytes/str issue after T267269 (duration: 00m 04s)

dpifke closed this task as Resolved.Wed, Nov 18, 11:15 PM

The ArcLamp fix is deployed, and the bogus lines were fixed via:

$ find /srv/arclamp -name '*.log' | xargs egrep -l "^b'" | xargs sudo -u xenon sed -e "s/^b'//" -e "s/'$//" -i

I reviewed logs again and don't see any other issues at the moment; calling this done.