Page MenuHomePhabricator
Create Task
Maniphest T192698

Upgrade Quarry to python 3
Closed, ResolvedPublic
Actions

Description

Python 2 will die... in less than 2 years. We get lots of good stuffs like map() result being a lazy generator rather than a list in python 3.

Details

SubjectRepoBranchLines +/-
killer.py: fix import since py3analytics/quarry/webmaster+1 -1
worker.py: fix indexing of sql errorsanalytics/quarry/webmaster+7 -7
Port to Python3analytics/quarry/webmaster+66 -83
quarry: Move the install into a venv and upgrade to Python 3operations/puppetproduction+33 -411
Update Vagrant files to py3analytics/quarry/webmaster+2 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

zhuyifei1999 created this task.Apr 21 2018, 12:16 AM
zhuyifei1999 added a subscriber: Halfak.Apr 21 2018, 1:17 AM

@Halfak is ORES running on python 3?

zhuyifei1999 mentioned this in T194691: Upgrade Quarry main server from Trusty.May 14 2018, 9:48 PM
zhuyifei1999 mentioned this in T196153: Quarry cannot save queries with emojies.Jun 1 2018, 11:06 AM
gerritbot subscribed.Jun 12 2018, 7:34 PM

Change 440007 had a related patch set uploaded (by Framawiki; owner: Framawiki):
[analytics/quarry/web@master] [WIP, DON'T MERGE] Port to Python3

https://gerrit.wikimedia.org/r/440007

gerritbot added a project: Patch-For-Review.Jun 12 2018, 7:34 PM
zhuyifei1999 added a comment.Aug 9 2018, 4:16 PM

@Framawiki Can I play with your quarry-frama-test-py3-* instances? (ssh in, install packages, git fetch & checkout, restart services) Kinda lazy to rebuild all these instances.

Framawiki added a comment.EditedAug 9 2018, 4:48 PM

Hello @zhuyifei1999, these old instances have really been draft :) you'd better create them again.
We still have to work on the dependencies (upgrade to Trusty at the same time) and update puppet with this. Doc for standalone puppetmaster.

zhuyifei1999 added a comment.Aug 9 2018, 5:54 PM
In T192698#4492188, @Framawiki wrote:

Hello @zhuyifei1999, these old instances have really been draft :) you'd better create them again.

Ok sure.

In T192698#4492188, @Framawiki wrote:

Doc for standalone puppetmaster.

There some annoying things (like if you rebuild an instance puppet break). But thanks for linking

zhuyifei1999 added a comment.Aug 9 2018, 5:56 PM

Umm. the quota on the # of instances can't let me create more instances... Can I delete them?

gerritbot added a comment.Aug 9 2018, 7:47 PM

Change 451698 had a related patch set uploaded (by Zhuyifei1999; owner: Zhuyifei1999):
[operations/puppet@production] [WIP] Quarry: Move the install into a venv and upgrade to Python 3

https://gerrit.wikimedia.org/r/451698

gerritbot added a comment.Aug 20 2018, 7:15 PM

Change 454079 had a related patch set uploaded (by Framawiki; owner: Framawiki):
[analytics/quarry/web@master] Update Vagrant files to py3

https://gerrit.wikimedia.org/r/454079

Framawiki added a parent task: T202588: Do the big Quarry migration.Aug 22 2018, 10:14 PM
Framawiki removed a subtask: T194691: Upgrade Quarry main server from Trusty.
gerritbot added a comment.Sep 6 2018, 2:59 AM

Change 454079 abandoned by Zhuyifei1999:
Update Vagrant files to py3

Reason:
Squashed into https://gerrit.wikimedia.org/r/#/c/analytics/quarry/web/ /440007/

https://gerrit.wikimedia.org/r/454079

Framawiki mentioned this in T202588: Do the big Quarry migration.Sep 7 2018, 8:15 PM
gerritbot added a comment.Sep 12 2018, 5:18 PM

Change 451698 merged by Bstorm:
[operations/puppet@production] quarry: Move the install into a venv and upgrade to Python 3

https://gerrit.wikimedia.org/r/451698

gerritbot added a comment.Sep 12 2018, 7:07 PM

Change 440007 merged by jenkins-bot:
[analytics/quarry/web@master] Port to Python3

https://gerrit.wikimedia.org/r/440007

Framawiki mentioned this in T204161: "Recent queries" page badly shows time.Sep 12 2018, 8:38 PM
gerritbot added a comment.Sep 12 2018, 8:56 PM

Change 460104 had a related patch set uploaded (by Framawiki; owner: Framawiki):
[analytics/quarry/web@master] worker.py: fix indexing of sql errors

https://gerrit.wikimedia.org/r/460104

gerritbot added a comment.Sep 12 2018, 8:58 PM

Change 460104 merged by Zhuyifei1999:
[analytics/quarry/web@master] worker.py: fix indexing of sql errors

https://gerrit.wikimedia.org/r/460104

zhuyifei1999 closed this task as Resolved.Sep 12 2018, 11:17 PM
zhuyifei1999 claimed this task.
zhuyifei1999 reassigned this task from zhuyifei1999 to Framawiki.
gerritbot added a comment.Sep 23 2018, 2:29 PM

Change 462295 had a related patch set uploaded (by Framawiki; owner: Framawiki):
[analytics/quarry/web@master] killer.py: fix import since py3

https://gerrit.wikimedia.org/r/462295

gerritbot added a comment.Sep 23 2018, 4:24 PM

Change 462295 merged by jenkins-bot:
[analytics/quarry/web@master] killer.py: fix import since py3

https://gerrit.wikimedia.org/r/462295

zhuyifei1999 mentioned this in T209226: Quarry can be affected by CSV Injection.Dec 3 2018, 7:10 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL