Page MenuHomePhabricator

E901 SyntaxError: invalid syntax is wrongly raised on using python's abc by jenkins python CI linter
Closed, ResolvedPublic

Description

https://gerrit.wikimedia.org/r/#/c/326155/6
https://integration.wikimedia.org/ci/job/operations-puppet-tox-jessie/10611/console

This is probably because a python3 files is being linted with a python2.6 linter:
https://bugs.launchpad.net/pyflakes/+bug/1362617

Can we stop making the linters vote until they work as intended? This is the second time I run into a problem with them being outdated.

Event Timeline

hashar subscribed.

Jenkins just runs tox which has the env/commands to run defined in tox.ini at the root of the repo. The relevant section for the linter is:

[testenv:pep8]
deps = flake8==2.5.5
commands = flake8

It is run on Jessie which has python defaulting to python2.7 hence flake8 is run under python2.7 and use its AST parser. Thus when one does:

#!/usr/bin/python3
import abc

class RemoteExecution(metaclass=abc.ABCMeta):
    pass

That indeed fails because metaclass=abc.ABCMeta is invalid python2.7 syntax.

flake8 could probably be made smarter by detecting the inspected file has a python3 shebang and have it shell out to something like python3 -m flake8 whatever.py. But I don't think it support that.

A lame workaround is to setup a linter environment for each python version. And exclude/include files as needed. It is not so nice and going to be annoying long term, but on a short term perspective that will unlock that patch.

Change 327746 had a related patch set uploaded (by Hashar):
Support flake8 with python3

https://gerrit.wikimedia.org/r/327746

https://gerrit.wikimedia.org/r/327746 is a terrible hack but should solve the issue for now :}

Change 327746 abandoned by Hashar:
Support flake8 with python3

Reason:
On https://gerrit.wikimedia.org/r/#/c/326155 , Volans recomends to move the file transferring system to a standalone repository. Would indeed be easier to finely tune the test and maybe even package it :D

https://gerrit.wikimedia.org/r/327746

hashar changed the task status from Open to Stalled.Jan 23 2017, 2:04 PM
hashar moved this task from Repo setup to In progress on the Continuous-Integration-Config board.

Seems the easiest is to move the code out of puppet.git to a standalone repo. Not much I can do on that front.

Three years later, is this still a problem? If yes, is move the code out of puppet.git to a standalone repo the preferred solution and should that be a subtask which blocks this task? If yes, would someone create that? If no, how to proceed? Asking as I'd like to not have tickets with "stalled" status forever. Thanks! :)

hashar assigned this task to jbond.
hashar added a subscriber: jbond.

The patch https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/326155/ introduced a python 3 based software to puppet but the linter used python 2. The change eventually got abandoned in favor of a standalone repo operations/software/wmfmariadbpy.

Eventually @jbond fixed it properly by having the python stuff to support both python 2 and 3: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/510613/