On pypi, pywikibot is published as version 3.0.*, but when installed, pywikibot identifies as 3.0.dev0. This breaks pip freeze, since you can't install pywikibot==3.0.dev0 from pypi.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| [bugfix] Fix broken get_version() in setup.py | pywikibot/core | master | +9 -4 |
Related Objects
Related Objects
- Mentioned In
- rPWBCd92464fa6308: [bugfix] Fix broken get_version() in setup.py
T236405: pywikibot.page.BasePage.imagelinks() crashes
T197936: [RfC] Figure out the version format - Mentioned Here
- rPWBCc67e06966aab: Allow date-versioned pypi releases from setup.py
T197936: [RfC] Figure out the version format
Event Timeline
Comment Actions
/tmp$ virtualenv venv
Running virtualenv with interpreter /usr/bin/python2
New python executable in /tmp/venv/bin/python2
Also creating executable in /tmp/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel.../bin/activate
done.
:/tmp$ source venv/bin/activate
(venv) :/tmp$ pip install pywikibot
Collecting pywikibot
Downloading https://files.pythonhosted.org/packages/47/a0/7c0264144f9b0af6c344507b46bff0523d64ff1f62d5990fa98c1c758909/pywikibot-3.0.20180603.tar.gz (493kB)
100% |████████████████████████████████| 501kB 2.4MB/s
Collecting requests!=2.18.2,>=2.9 (from pywikibot)
Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB)
100% |████████████████████████████████| 92kB 2.3MB/s
Collecting ipaddr>=2.1.10 (from pywikibot)
Downloading https://files.pythonhosted.org/packages/9d/a7/1b39a16cb90dfe491f57e1cab3103a15d4e8dd9a150872744f531b1106c1/ipaddr-2.2.0.tar.gz
Collecting idna<2.8,>=2.5 (from requests!=2.18.2,>=2.9->pywikibot)
Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)
100% |████████████████████████████████| 61kB 2.4MB/s
Collecting certifi>=2017.4.17 (from requests!=2.18.2,>=2.9->pywikibot)
Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)
100% |████████████████████████████████| 153kB 705kB/s
Collecting chardet<3.1.0,>=3.0.2 (from requests!=2.18.2,>=2.9->pywikibot)
Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
100% |████████████████████████████████| 143kB 2.1MB/s
Collecting urllib3<1.24,>=1.21.1 (from requests!=2.18.2,>=2.9->pywikibot)
Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB)
100% |████████████████████████████████| 143kB 2.4MB/s
Building wheels for collected packages: pywikibot, ipaddr
Running setup.py bdist_wheel for pywikibot ... done
Stored in directory: .cache/pip/wheels/7d/dc/44/d003350fab03f1cebbdf4a452c1df27674fab34ebc3676a209
Running setup.py bdist_wheel for ipaddr ... done
Stored in directory: .cache/pip/wheels/c8/46/0b/5220490c49f714459a27e1eaafe3eef4c7a5e9bb4e9baf10ca
Successfully built pywikibot ipaddr
Installing collected packages: idna, certifi, chardet, urllib3, requests, ipaddr, pywikibot
Successfully installed certifi-2018.4.16 chardet-3.0.4 idna-2.7 ipaddr-2.2.0 pywikibot-3.0.dev0 requests-2.19.1 urllib3-1.23
(venv) :/tmp$ pip freeze
certifi==2018.4.16
chardet==3.0.4
idna==2.7
ipaddr==2.2.0
pkg-resources==0.0.0
pywikibot==3.0.dev0
requests==2.19.1
urllib3==1.23Note that the file downloaded is pywikibot-3.0.20180603.tar.gz but it says Successfully installed pywikibot-3.0.dev0
If I try to install pwb using a requirement file containing pywikibot==3.0.dev0
(venv2) :/tmp$ pip install -r t Collecting pywikibot==3.0.dev0 (from -r t (line 1)) Could not find a version that satisfies the requirement pywikibot==3.0.dev0 (from -r t (line 1)) (from versions: 2.0rc1.post1, 2.0rc1.post2, 2.0rc3, 2.0rc4, 2.0rc5, 3.0.20170403, 3.0.20170521, 3.0.20170713, 3.0.20170801, 3.0.20171212, 3.0.20180108, 3.0.20180204, 3.0.20180302, 3.0.20180304, 3.0.20180403, 3.0.20180505, 3.0.20180603) No matching distribution found for pywikibot==3.0.dev0 (from -r t (line 1))
Comment Actions
I already proposed to switch to server versioning in https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/409823/
Comment Actions
This seems more like a bug in setup.py or in some other file responsible for package version when building Pip package. Switch to SemVer or any other versioning system should be discussed in T197936
Comment Actions
There are two possibilities to get 3.0.dev0: an exception when getting a date from git command or an exception when importing subprocess (https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/setup.py$188):
def get_version(): """Get a valid pywikibot module version string.""" version = '3.0' try: import subprocess date = subprocess.check_output( ['git', 'log', '-1', '--format=%ci']).strip() date = date.decode().split(' ')[0].replace('-', '') version += '.' + date if 'sdist' not in sys.argv: version += '.dev0' except Exception as e: print(e) version += '.dev0' return version
When running pip install pywikibot -v I found these suspicious lines:
$ pip install pywikibot -v ... Running setup.py (path:/tmp/pip-install-ce3eh3dh/pywikibot/setup.py) egg_info for package pywikibot Running command python setup.py egg_info fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). Command '['git', 'log', '-1', '--format=%ci']' returned non-zero exit status 128. Cannot include pywikibot/DIRECTORIES.rst; file not found Cannot include HISTORY.rst; file not found Cannot include CODE_OF_CONDUCT.rst; file not found ... Running setup.py install for pywikibot ... Running command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ce3eh3dh/pywikibot/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-3qobqmry/install-record.txt --single-version-externally-managed --compile --user --prefix= fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). Command '['git', 'log', '-1', '--format=%ci']' returned non-zero exit status 128. Cannot include pywikibot/DIRECTORIES.rst; file not found Cannot include HISTORY.rst; file not found Cannot include CODE_OF_CONDUCT.rst; file not found ...
Anyway we should never run into 3.0.dev0 so definitely this should be fixed in the first place, but there is a second problem: Why Pip fails to make proper version?
Comment Actions
Pip uses the source distribution published on pypi, which is basically the repository contents without the .git folder (but with some extra metadata) so the setup.py can't use git to determine the version. Most projects either set the version in setup.py explicitly or as a constant in the source code which they import at build time. It would IMO also be very good to also publish wheels to pypi, which do not need to be build to be installed. (It's not more work than installing wheel and running bdist_wheel before uploading)
Comment Actions
I see, but we need to call git to find out last master commit date. How could we achieve that? Or perhaps get it from GitHub somehow in setup.py?
Comment Actions
The source distribution on pypi is created with setuptools and therefore has access to the repository, so you could use e.g. setuptools_scm to write the current version to a .gitignore'd file. Even though this is possible, I'd recommend tracking the version explicitly in the code and/or build configuration. Modern build tools such as flit and poetry do even mandate this.
Querying github in setup.py fails when the building machine has no internet connection and will otherwise lead to really nasty non-deterministic bugs, so I'd definitely refrain from that. __init__.py currently states __version__ = __release__ = '3.1.dev0', which should also be corrected.
`
Comment Actions
__init__.py's value is completely fine according to the current versioning scheme of Pywikibot following the corresponding PEP rule . As I mentioned earlier, any versioning scheme proposals and discussions should be kept on one place, in T197936. For the current versioning scheme we just need to find out latest commit date anyhow. We load some details from remote git in version.py and I believe most users installing package from pypi will do that using pip (using internet connection). I'm not sure, why this was working until recently and now isn't (@Xqt was making changes in this stuff recently).
Comment Actions
Change 563431 had a related patch set uploaded (by Dvorapa; owner: Dvorapa):
[pywikibot/core@master] [bugfix] Fix broken get_version() in setup.py
Comment Actions
I guess this never worked with pwb 3.
See rPWBCc67e06966aabe83ade94d1b08b67c0f168e2665d
Comment Actions
Change 563431 merged by jenkins-bot:
[pywikibot/core@master] [bugfix] Fix broken get_version() in setup.py