Change 537267 had a related patch set uploaded (by Elukey; owner: Elukey):
[analytics/reportupdater@master] Move reportupdater_test5 to python3

https://gerrit.wikimedia.org/r/537267

Change 537268 had a related patch set uploaded (by Elukey; owner: Elukey):
[analytics/reportupdater@master] Move codebase to python3

https://gerrit.wikimedia.org/r/537268

Change 537267 abandoned by Elukey:
Move reportupdater_test5 to python3

https://gerrit.wikimedia.org/r/537267

This is a bit harder than expected, since tests will need some adjustment to:

1. run tox with python 3.7 in jenkins
2. encode/decode properly strings to avoid failures

https://gerrit.wikimedia.org/r/537267 could be used as baseline, I have ran the python 2to3 tool on all files.

https://gerrit.wikimedia.org/r/#/c/analytics/reportupdater/+/537268/ is ready for a first review, testing with real data has not been done yet.

Jenkins already supports python3, it is just a matter of changing the tox.ini file!

Change 539021 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] reportupdater::job: use python3 in the systemd timer

https://gerrit.wikimedia.org/r/539021

Change 537268 merged by Elukey:
[analytics/reportupdater@master] Move codebase to python3

https://gerrit.wikimedia.org/r/537268

Change 539021 merged by Elukey:
[operations/puppet@production] reportupdater::job: use python3 in the systemd timer

https://gerrit.wikimedia.org/r/539021

Marcel verified that the code runs correctly with python3, and today I deployed it and updated the systemd timers on stat1006/7. Everything seems working as expected!

This must be my fault, but I thought I would mention it anyway, at least until I understand what's wrong. I get this python2-backcompat-looking error when running reportupdater on a stat machine:

2019-10-29 14:39:17,772 - ERROR - Report "baseline" could not be written because of error: a bytes-like object is required, not 'str'
Traceback (most recent call last):
  File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 46, in run
    self.record_to_graphite(report, new_dates)
  File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 185, in record_to_graphite
    self.graphite.record_row(row, report)
  File "/srv/reportupdater/reportupdater/reportupdater/graphite.py", line 64, in record_row
    self.record(metric, value, timestamp)
  File "/srv/reportupdater/reportupdater/reportupdater/graphite.py", line 104, in record
    sock.send('{} {} {}\n'.format(metric, value, timestamp))
TypeError: a bytes-like object is required, not 'str'

Steps to reproduce:

• Log in to stat1007
• virtualenv -p python3 ~/venv
• ~/venv/bin/pip install -r /srv/reportupdater/reportupdater/requirements.txt
• Check out https://gerrit.wikimedia.org/r/#/c/analytics/reportupdater-queries/+/542419/
• ~/venv/bin/python /srv/reportupdater/reportupdater/update_reports.py --log-level=debug reference-previews reference-previews/output

+1 it does look like a bug. The solution would be as simple as .encode('utf-8') (I ran that locally and it works, at least) however I haven't been able to verify which encoding the Graphite plaintext protocol expects, so a non-ascii character might cause meltdown.

In T204736#5615474, @awight wrote:

+1 it does look like a bug. The solution would be as simple as .encode('utf-8') (I ran that locally and it works, at least) however I haven't been able to verify which encoding the Graphite plaintext protocol expects, so a non-ascii character might cause meltdown.

Yep the fix looks good, if you want to file a code review I'll review/merge it! Please also be aware that graphite is in the process of being deprecated (in a couple of years probably, but still best to know it).

Change 546971 had a related patch set uploaded (by Awight; owner: Awight):
[analytics/reportupdater@master] Leftover py2-3 glitch: encode a string to bytes

https://gerrit.wikimedia.org/r/546971

Change 546971 merged by jenkins-bot:
[analytics/reportupdater@master] graphite.py: encode a text string before socket.send

https://gerrit.wikimedia.org/r/546971