mw 1.14-1.18 is still supported with the current release. I think you can Always use "python2" release for older mw versions.

Change 598976 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [doc] Use python2 release for older mw versions

https://gerrit.wikimedia.org/r/598976

Change 409823 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [IMPR] Use one central point for framework version

https://gerrit.wikimedia.org/r/409823

In T253719#6168009, @Xqt wrote:

mw 1.14-1.18 is still supported with the current release. I think you can Always use "python2" release for older mw versions.

Is the "python2" release available on PyPI? I found it by navigating from https://doc.wikimedia.org/pywikibot/master/ to https://doc.wikimedia.org/pywikibot/master/installation.html to https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation, which is a bit annoying, compared to just specifying a version number in setup.py.

In T253719#6168009, @Xqt wrote:

mw 1.14-1.18 is still supported with the current release. I think you can Always use "python2" release for older mw versions.

I'm connecting to mw 1.16.

The 'pywikibot==3.0.20200111' distribution was not found and is required by pwbtest

This bug was fixed in 3.0.20200508.

Is the "python2" release available on PyPI?

Any 3.0.x release is automatically a "python2" release

This indicates that pywikibot version 3.0.20200111 should be used.

3.0.20200508 still supports old MW versions. And the next 1 or 2 releases will also support old MW versions. Could you test the latest release if any of your described issues still occur?

In T253719#6168454, @Dvorapa wrote:

Is the "python2" release available on PyPI?

Any 3.0.x release is automatically a "python2" release

Thanks for clarifying & asking for the message change in https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/598976/ to make this clear.

This indicates that pywikibot version 3.0.20200111 should be used.

3.0.20200508 still supports old MW versions. And the next 1 or 2 releases will also support old MW versions. Could you test the latest release if any of your described issues still occur?

Same thing with 3.0.20200508:

Processing dependencies for pwbtest==0.1.dev2+g9c44c35.d20200527
Searching for pywikibot==3.0.20200508
Reading https://pypi.org/simple/pywikibot/
Downloading https://files.pythonhosted.org/packages/42/3d/ecfce167ec6b29726d043ec324daf29e48b094c37319a7b193bbd9e40610/pywikibot-3.0.20200508.tar.gz#sha256=c61095a8d56c02ebc73f5c23f7e75248f11e0272d88cbe6b5d258b0438cbb0b6
Best match: pywikibot 3.0.20200508
Processing pywikibot-3.0.20200508.tar.gz
Writing /var/folders/pr/6ffcygy56cj18vcwp0njbtyh0000gn/T/easy_install-0wm8hase/pywikibot-3.0.20200508/setup.cfg
Running pywikibot-3.0.20200508/setup.py -q bdist_egg --dist-dir /var/folders/pr/6ffcygy56cj18vcwp0njbtyh0000gn/T/easy_install-0wm8hase/pywikibot-3.0.20200508/egg-dist-tmp-81utx7nq
fatal: Kein Git-Repository (oder irgendeines der Elternverzeichnisse): .git
Command '['git', 'log', '-1', '--format=%ci']' returned non-zero exit status 128.
The 'pywikibot' distribution was not found and is required by the application
Cannot include pywikibot/DIRECTORIES.rst; file not found
Cannot include ROADMAP.rst; file not found
Cannot include CODE_OF_CONDUCT.rst; file not found
zip_safe flag not set; analyzing archive contents...
pywikibot.__pycache__.config2.cpython-37: module references __file__
pywikibot.__pycache__.i18n.cpython-37: module references __file__
pywikibot.__pycache__.i18n.cpython-37: module references __path__
pywikibot.__pycache__.version.cpython-37: module references __file__
creating /Users/wieland/dev/pwbtest/venv/lib/python3.7/site-packages/pywikibot-3.0.dev0-py3.7.egg
Extracting pywikibot-3.0.dev0-py3.7.egg to /Users/wieland/dev/pwbtest/venv/lib/python3.7/site-packages
Adding pywikibot 3.0.dev0 to easy-install.pth file

Installed /Users/wieland/dev/pwbtest/venv/lib/python3.7/site-packages/pywikibot-3.0.dev0-py3.7.egg
error: The 'pywikibot==3.0.20200508' distribution was not found and is required by pwbtest
Run "source venv/bin/activate" and then pwbtestcli. It will not work

If I'm reading https://github.com/wikimedia/pywikibot/blob/fac57cf2773186669b96b20548876b6a43a7e0fc/setup.py#L171-L187 correctly, the calls to git and get_distribution both fail:

fatal: Kein Git-Repository (oder irgendeines der Elternverzeichnisse): .git
Command '['git', 'log', '-1', '--format=%ci']' returned non-zero exit status 128.
The 'pywikibot' distribution was not found and is required by the application

(The first line is german for "This is not a git repository because '.git' does not exist").

Hmmm, we should use PKGINFO in that case (@Xqt). But we do use it since January. This is weird, why it does not work? What version of setuptools do you use?

The 'pywikibot' distribution was not found and is required by the application

The fallback to PKGINFO seems not working in your case. But why?

Please describe the actual steps and commands you do to make this error occur, I was unable to reproduce the issue so far.

I've no glue what is going wrong, probably a wrong setuptools version?

Maybe we should switch to a fixed versioning like it was proposed by that patch given above.

In T253719#6168842, @Dvorapa wrote:

The 'pywikibot' distribution was not found and is required by the application

The fallback to PKGINFO seems not working in your case. But why?

From a quick look at the setuptools documentation, get_distribution will only work if pywikibot is already installed. That's not the case when installing pywikibot itself.

In T253719#6168844, @Dvorapa wrote:

Please describe the actual steps and commands you do to make this error occur, I was unable to reproduce the issue so far.

See my first post: clone https://github.com/mineo/pwbtest, run repro.sh. Optionally, bump the version number of pywikibot that's required in setup.py.

In T253719#6169283, @Xqt wrote:

I've no glue what is going wrong, probably a wrong setuptools version?

I've added pip install --upgrade setuptools to repro.sh before the call to setup.py develop. That installs setuptools 46.4.0, the most recent one. That didn't help.

Okay, so I've tried your repo from https://github.com/mineo/pwbtest . I got the same error, but I noticed you use setup.py develop which is not a recommended practice afaik. So I've tried pip install -e . instead and everything worked like a charm.

Using setup.py develop these days is literally like asking for a trouble from my point of view, it is also not recommended by Python developers. These days PEP 517 and 518 make the use of setup.py install and develop effectively deprecated. Similarly setuptools project makes setup.py test deprecated and test frameworks like pytest, unittest, nosetest etc. also recommend using their way instead. So generally using setup.py as an executable script instead of using it just as a source of package information for package manager (like pip) is not a good idea (except setup.py sdist of course).

I still don't know why exactly pip install -e . works while setup.py develop doesn't. To me those two should work the same, only setup.py develop should first determine package manager in use (we use pip & setuptools) and then call its install function. Anyway, the fix might be easy to do. From your log it seems the distribution for pkg_resources is somewhat incorrect. I would prefer to close this as declined as setup.py develop is known for such bugs like installing the dependencies in an improper way (I found the same issue has been put on hold by googleapis python interface).

In T253719#6172489, @Dvorapa wrote:

Okay, so I've tried your repo from https://github.com/mineo/pwbtest . I got the same error, but I noticed you use setup.py develop which is not a recommended practice afaik. So I've tried pip install -e . instead and everything worked like a charm.

Using setup.py develop these days is literally like asking for a trouble from my point of view, it is also not recommended by Python developers. These days PEP 517 and 518 make the use of setup.py install and develop effectively deprecated.

I can't find any documentation about setup.py develop being deprecated in the setuptools documentation: https://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode. The test command in comparison has a big red deprecation banner.

pip install -e and/or using flit install -s as possible workarounds work. Thanks for the pointers.

See also https://github.com/googleapis/python-api-core/issues/27

Hmm, pip install -e . and flit install -s work, python setup.py install does not work. It ends with the same error ("error: The 'pywikibot==3.0.20200508' distribution was not found and is required by pwbtest").

Yeah, that's expected, setup.py install/develop/build will all have the same problem I think. I honestly don't know, how to fix this as I'm currently outside my PC, unable to create the in-the-middle-of setup.py develop conditions to find out, what should setup.py ask if not git or pkg_resources. In dist tarball, the only place with version number is PKGINFO and I don't know, how to instruct setup.py to parse this value from PKGINFO. I'm afraid single-sourced version will either have the same problem, or it would create a new one, but It would definitely be better than the current versioning madness.

I hate this Python packaging and versioning so much, such a good language with such a lame folder structure, packaging and versioning. In comparison to Dart or Go, this is still a madness even with PEP 517 and 518.

Python should totally inspire from them. Simple and clearly defined folder structure, single-sourced all package information and dependencies using a JSON/YAML serialized formats, simple tools to lint, format, build and publish...

I've been using setuptools_scm for years now and have never had any problems with it (I don't maintain any high-profile packages like pywikibot, though). Maybe a switch to it, or something similar, instead of reimplementing versioning in pywikibot itself, could be a solution?

If I got it right, either we would have to get rid of pip install -e option to enable setuptools_scm through pyproject.toml, or we would have to tell people to install setuptools_scm before installing Pywikibot. Both things are quite impractical for larger set-up projects. The issue with pyproject.toml and pip install -e is quite a flaw of PEP 517/518 :/ Although it looks like a good way to solve this.

Change 598976 merged by jenkins-bot:
[pywikibot/core@master] [doc] Use python2 release for older mw versions

https://gerrit.wikimedia.org/r/598976

Change 409823 merged by jenkins-bot:
[pywikibot/core@master] [IMPR] Use one central point for framework version

https://gerrit.wikimedia.org/r/409823