Some repository name ideas (see https://gerrit.wikimedia.org/r/#/admin/projects/?filter=pywikibot):

• pywikibot/active (pywikibot-active) others will become inactive over the time
• pywikibot/advance (pywikibot-advance) why not?
• pywikibot/4 (pywikibot-4) after releases 1 = trunc/compat, 2 = core/stable/2.0, 3 = core/master
• pywikibot/main (pywikibot-main) the main framework library, probably we could have others
• pywikibot/pwb (pywikibot-pwb) the new shortener derived from pywikibot and pwb.py wrapper
• pywikibot/pwbd (pywikibot-pwbd) the current phab repository is rPWBC using the next char
• pywikibot/next (pywikibot-next) the next pywikibot main release - or the next python release which doesn't support Python 2 anymore - or just the next step

First I thought we would rename old core/master. But that is not possible without breaking stuff, right?

I would add:

• pywikibot/bleeding (pywikibot-bleeding) bleeding edge
• pywikibot/edge (pywikibot-edge) same as above

I don't like:

• active – is this the main branch? or what is the main branch? is the main branch inactive? is pywikibot development inactive?
• advance – is this just additional to some other main branch?
• pwb – is this some pywikibot submodule?

Special:

• pwbd – I like this for Phabricator repository, but for Gerrit repository we should use something else

pwbc... pywikibot core; pwbd... pywikibot... daemon?

Based on the experience we had with moving from compat to core, I would advice against a new repository for the continued development. I'm afraid it will lead to a slow death of v3 (with lots of people not moving, and then once they move it becomes very painful due to the large difference between the versions).

My suggestion would be:

• Release a final v3 version on PyPI and tag this in the repository
• Bump version to 4 in master
• Add a warning when pywikibot is imported on Python 2 that Py2 is no longer supported, and to use the final pwb 3.x release
• After some time, make this an error
• Start removing Py2 support

@valhallasw We are trying to break as little as possible. So if anyone's cron automatically updates git and it still runs under py2 (per my experience and per our Wikimedia Hackathon meetup a dominant group of users), this would break their cron scripts. That's why we are trying to freeze the current master (I like the idea of final v3 release anyway) and start a new branch from it.

In T226507#5283657, @zhuyifei1999 wrote:

pwbc... pywikibot core; pwbd... pywikibot... daemon?

pywikibot devil 😈 :D

In T226507#5283832, @Dvorapa wrote:

@valhallasw We are trying to break as little as possible. So if anyone's cron automatically updates git and it still runs under py2 (per my experience and per our Wikimedia Hackathon meetup a dominant group of users), this would break their cron scripts. That's why we are trying to freeze the current master (I like the idea of final v3 release anyway) and start a new branch from it.

Ok, if this has been discussed in depth during that meetup and this option was discussed then there's no reason to go back on the choices made.

In T226507#5284014, @valhallasw wrote:

In T226507#5283832, @Dvorapa wrote:

@valhallasw We are trying to break as little as possible. So if anyone's cron automatically updates git and it still runs under py2 (per my experience and per our Wikimedia Hackathon meetup a dominant group of users), this would break their cron scripts. That's why we are trying to freeze the current master (I like the idea of final v3 release anyway) and start a new branch from it.

Ok, if this has been discussed in depth during that meetup and this option was discussed then there's no reason to go back on the choices made.

Pywikibot meetup group only agreed that we should not break user scripts and cronjobs running on py2 by deprecation steps. Could we find a better (and still not breaking) solution?

I am just a lurker but still have some kind of interest in pywikibot from time to time. Here some feedback based on a couple decades of IT

I would highly recommend to avoid creating a new repository (such as what has been done for compat). That loose all history and confuses people as to which repo should really be used.

Instead I would propose you tag a last version of pywikibot that still support python 2.7 and make it extremely clear that is the last ever release to support python 2.7. Then have master branch to only be tested with python3. Done?

If your concern is people not switching/upgrading to python3, that surely can be made easier by having some tutorials (eg use python3 not python, how to install python3 etc). Regardless they can still use the last version still tested with python 2.7.

Pywikibot meetup group only agreed that we should not break user scripts and cronjobs running on py2 by deprecation steps. Could we find a better (and still not breaking) solution?

The way to achieve that is for users to keep the version they are currently running. If they want the latest bugfix and features, they would have to switch to python3. In all honesty it is not that complicated to handle.

In both case (1) creating a new repo like 'compat' (2) keeping compatibility with 2.7, you are going to have troubles having users to actively migrate and would still drag the python2.7 support for months and months.

At some point, you should force users to migrate and break the back compatibility. And again, the migration is not that complicated, just install python3?

Anyway T226507#5283686 sounds like a sane and safe plan.

So everyone on meetup: please do not break py2 users scripts/cronjobs, better to create new branch
Everyone in this task: let's break py2 compatibility

Mamma mia what shall we do?

Let's vote?

In T226507#5284014, @valhallasw wrote:

In T226507#5283832, @Dvorapa wrote:

@valhallasw We are trying to break as little as possible. So if anyone's cron automatically updates git and it still runs under py2 (per my experience and per our Wikimedia Hackathon meetup a dominant group of users), this would break their cron scripts. That's why we are trying to freeze the current master (I like the idea of final v3 release anyway) and start a new branch from it.

Ok, if this has been discussed in depth during that meetup and this option was discussed then there's no reason to go back on the choices made.

I was at that meeting, we didn't. I complete agree with valhallasw. Bad plan, waste of effort, don't do it.

I don't remember what was the proposed solution at the meetup (maybe branch or tags, but no memory of creating a new repo at all), but the problem was discussed AFAICR. You also have to account for that at the meetup there are many of those who are just 'using pywikibot', but here, pretty much everyone is a pywikibot dev, who may have different priorities than end users.

From my 'pywikibot user / out-of-tree scripts developer' perspective, yes, breaking compatibility is gonna generate a huge amount of work for me (almost all of my pywikibot scripts have only been tested with py2 because, well, I have been procrastinating to rebuild my toolforge virtualenv -- it's not just a virtualenv but a 'prefix' for a few compiled binaries as well), but this work is coming sooner or later anyways, with the pending EOL of py2. As long as I want to keep my bots alive it has to be done, whether it's Pywikibot killing py2 first or Toolforge does.

From my 'pywikibot dev' perspective, really, if you are auto updating from git, have you not have enough breakages already from randomly pulling, that this py3 breakage isn't a big deal? If not, then that's our bad; we aren't enforcing deprecations properly.

So yeah, I prefer breaking py2 soonish, a few weeks after an announcement, rather than creating a new repo and make migrations harder than git pull + rebuild virtualenv.

People from the Czech community are using py2 mainly because of procrastinating with transition. Also I like the idea to push them a little bit and from the meetup, you all know I hate maintaining old crappy py2-py3 compatibility workarounds (not only in Pywikibot). And finally I like the quote: "People still using Python 2 need to understand this is not main branch anymore and should accept the fact - if they are going to use it further - that it will require extra work from them instead of devs to keep their code running."

I also want to point out that with setuptools everyone using good old SemVer recommended practice of specifying dependencies like:

pywikibot >= 3.0.20180108, <= 4

should be also fine without any branch fiddling (why don't Pywikibot use this format for dependency specifications and we are surprised by breakage every time a dependency pushes some breaking release?)

In T226507#5285056, @zhuyifei1999 wrote:

From my 'pywikibot dev' perspective, really, if you are auto updating from git, have you not have enough breakages already from randomly pulling, that this py3 breakage isn't a big deal?

Thats the reason of introducing the "stable" tag 4 months ago which always should be used for productions systems instead of the master branch to keep breakage as small as possible

If not, then that's our bad; we aren't enforcing deprecations properly.

There is an hint in the HIYSTORY.rst and there are deprecation warnings too. Unfortunately the deprecation warning aren't visible by default except if you are using -debug option or setting the debug_log. Therefore I proposed to enable a FutureWarning with deprecating methods in https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/510722/ to enforce deprecated methods to be replaced. See also https://www.mediawiki.org/wiki/Manual:Pywikibot/2.0/Conversion.

In T226507#5285133, @Dvorapa wrote:

People from the Czech community are using py2

There are probably some peoble using compat and I am fine with Merlijns recommendation not to make this mistake twice to have years of parallel development (compat to core process needed more than 10 years btw.) and the difficulty to enforce people to migrate.

I hate maintaining old crappy py2-py3 compatibility workarounds (not only in Pywikibot).

The main problem are some syntax changes which cannot be backported which needs one library for py2 and one for py3 which is unnecessarily difficult to combine if it ever works and it does not work for code which resides in scripts. This hinders further development and maintaining keeping compatible needs a lot of work which could be better invested in new features or bug fixes.

if they are going to use it further - that it will require extra work from them instead of devs to keep their code running."

+1

I also want to point out that with setuptools everyone using good old SemVer recommended practice of specifying dependencies like:

pywikibot >= 3.0.20180108, <= 4

should be also fine without any branch fiddling (why don't Pywikibot use this format for dependency specifications and we are surprised by breakage every time a dependency pushes some breaking release?)

You can also use a tag if you are cloning the repository like https://github.com/yuvipanda/paws/pull/33/commits/5658d2f41834ae28c7b9d6e27fc688afbd5c254e or you can use the stable release from https://tools.wmflabs.org/pywikibot/. I think we can provide a final "python2" tag in the same way.

In T226507#5285444, @Xqt wrote:

I think we can provide a final "python2" tag in the same way.

Good idea. So are we going the way of a breakage? Somewhat like:

• T225591: Make Travis and Appveyor pass again
• push last 3.x version, tag it with stable & py2 tags
• commit version identifiers to 4.0.x all over rPWBC
• commit py2+3.4 deprecations during the summer (code deprecation, dependency version update, updated dependency issues fix, code cleanup, etc.)
• T225591: Make Travis and Appveyor pass again again
• push first 4.x version, tag it with stable

In T226507#5285475, @Dvorapa wrote:

In T226507#5285444, @Xqt wrote:

I think we can provide a final "python2" tag in the same way.

Good idea. So are we going the way of a breakage? Somewhat like:

• T225591: Make Travis and Appveyor pass again
• push last 3.x version, tag it with stable & py2 tags
• commit version identifiers to 4.0.x all over rPWBC
• commit py2+3.4 deprecations during the summer (code deprecation, dependency version update, updated dependency issues fix, code cleanup, etc.)
• T225591: Make Travis and Appveyor pass again again
• push first 4.x version, tag it with stable

Something like this, yes. But more documentation stuff is needed too and probably inform users via mailing list or so. Deprecated parts which are going to be removed should be anounced e.g. via FutureWarning as proposed to be visible by bot owners. Thy have mostly one month or more to fix their code if necesary.

Maybe we can have some migration service for private code. There are some code parts which has to be migrated from py2 to py3. Most of them can be done by the 2to3.py fixer but that script fixes more than necessary to be sure all problems are fixed. I migrated all my scripts few days ago and found only one problem.

inform users via mailing list

yes, of course

Maybe we can have some migration service for private code. I migrated all my scripts few days ago and found only one problem.

I migrated my py2 scripts to py3 cca in 2016 (I don't remember), so with this I can not help as much :D

I think we have an approach now how to proceed with supporting Python 3 only.
I guess we can close this here as declined?
Thanks all for the valuable notes in this task.

+1 and I have enjoyed following this civil conversation :-] Well done pywikibot devs!