Page MenuHomePhabricator

Drop support of MW <1.19
Open, MediumPublic

Description

MW 1.14 fails with so many errors

With @Urbanecm we established a MW 1.14.1 wiki (https://dvorapovawiki.urbanec.cz/mw/index.php/Main_Page) and created Pywikibot-test account there

We are aware of its configuration issues (mainly too new MySQL, too new PHP)

But Travis shows some interesting issues to be solved:
https://travis-ci.org/dvorapa/pywikibot/builds/650908994

Mainly these kind of errors are thrown:

  • KeyError: 'query+xyz' in pywikibot.data.api.ParamInfo
  • AssertionError self._action_modules == set(main_modules_param['type']) in pywikibot.data.api.ParamInfo
  • some tests fail due to wiki's emptyness (these should be skipped using SkipTest or if, so this is definitely an issue of our tests)

Event Timeline

Dvorapa created this task.Feb 15 2020, 8:56 PM
Restricted Application added a project: Pywikibot. · View Herald TranscriptFeb 15 2020, 8:56 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald Transcript

I am not sure this is worth the time for any fixes since 1.14's lifecycle has ended 10 years ago.

For older MediaWiki Pywikibot compat is more usable but it is no longer official maintained. It may be still an idea to (unofficially) port the compat to newer version of Python (though no new feature will be added).

Anyway per https://www.mediawiki.org/wiki/Manual:Pywikibot/Overview the core should theorically support MediaWiki 1.14, so it may be a real bug.

Please also note older install of MediaWiki may be highly customized (see http://www.wikitree.com/, it even disabled all special pages) so it is possible that neither core nor compat will work. Wikia already use a highly different editor.

For older MediaWiki Pywikibot compat is more usable but it is no longer official maintained. It may be still an idea to (unofficially) port the compat to newer version of Python (though no new feature will be added).

Pywikibot Compat is no longer officially maintained, yes. But as an unmaintained project, anyone can adopt it and do whatever they want to.

Anyway per https://www.mediawiki.org/wiki/Manual:Pywikibot/Overview the core should theorically support MediaWiki 1.14, so it may be a real bug.

This is what this task is about. Pywikibot tests don't test older MediaWiki wiki than 1.31.1, but Pywikibot docs still claim support for 1.14+. Pywikibot tests should definitely test lowest supported MediaWiki version (and perhaps even some other versions as well). We can establish test set of empty wikis (using WMCS?) or perhaps establish an empty wiki during tests (can increase testing time though). Also there are surely some issues with older MW version wikis (as noticed in the experiment described in the description), which should be solved on Pywikiobt's side or those MW versions should be dropped.

I believe we should support only MWs being supported now, and one or two unsupported versions. Actively using anything that's ten years old isnt wise, and no one should ever do it.

Dvorapa added a subscriber: Xqt.
Bugreporter added a comment.EditedFeb 16 2020, 5:55 AM

I instead think (only in my opinion) we should support we should continue to make it work in Fandom (1.19.24) for an unspecified time (probably drop it in 2025, 10 years after EOL of 1.19).

Xqt added a comment.Feb 16 2020, 6:52 AM

I instead think (only in my opinion) we should support we should continue to make it work in Fandom (1.19.24) for an unspecified time (probably drop it in 2025, 10 years after EOL of 1.19).

I agree because fandom is a most popular cluster. But their 1.19 release is also customised and has some deviations. We should be aware that we have only a couple of active developers and we spend a lot of time for actual problems with actual mw releases as well as Python versions already. Let's officially drop mw releases older than 1.19 with one of the next stable releases

Dvorapa added a comment.EditedFeb 16 2020, 1:33 PM

I instead think (only in my opinion) we should support we should continue to make it work in Fandom (1.19.24) for an unspecified time (probably drop it in 2025, 10 years after EOL of 1.19).

I agree because fandom is a most popular cluster. But their 1.19 release is also customised and has some deviations. We should be aware that we have only a couple of active developers and we spend a lot of time for actual problems with actual mw releases as well as Python versions already. Let's officially drop mw releases older than 1.19 with one of the next stable releases

That's a good idea. I think we should support mainly clean 1.19 and do not much specify to Fandom specialties. On the other hand we could add one Fandom wiki to our tests as we used to with Wikia in the past.

Xqt added a comment.Feb 16 2020, 4:34 PM

On the other hand we could add one Fandom wiki to our tests as we used to with Wikia in the past.

wowwiki could be a candidate for it

Change 572493 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [tests] Add wowwiki to travis test matrix

https://gerrit.wikimedia.org/r/572493

Xqt triaged this task as Low priority.Feb 18 2020, 5:54 PM

Change 577589 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [cleanup] Drop support of MediaWiki 1.14-1.18

https://gerrit.wikimedia.org/r/577589

Change 572493 merged by jenkins-bot:
[pywikibot/core@master] [tests] Add wowwiki to travis test matrix

https://gerrit.wikimedia.org/r/572493

19 errors with MW 1.19. One with family file, others with API usually

Dvorapa renamed this task from MW 1.14 fails with so many errors to Drop support of MW <1.19.Mar 9 2020, 4:03 PM
Dvorapa raised the priority of this task from Low to Medium.
Dvorapa updated the task description. (Show Details)

We should consider the next step once Fandom have completely moved to 1.33 (See T249403: login.py crashes on 1.33 Fandom wikis).

Dvorapa added a comment.EditedApr 4 2020, 11:20 AM

Umm, not completely, afaics it is currently in phase two where only new Fandom wikis are 1.33, but old are still 1.19 (with some parts from cca 1.11), otherwise our Fandom tests would be green already :D (8 hours ago they weren't)

Yes, not now, but what should the minimum version be to support once all old Fandom wikis are migrated.

As for now Fandom only enabled 1.33 platform for new wikis. They'll start moving old 1.19 wikis to the new platform soon and continue the process within following months. They're also about to update new platform regularly to stay up-to-date with WMF's version of MW.

More info: https://community.fandom.com/wiki/User_blog:MisterWoodhouse/Unified_Community_Platform:_The_Vision,_Purpose,_and_Process

Yeah, that's a good question. We dropped genealogy wikis recently (MW 1.14), Fandom wikis will move soon. What next major wiki farm with old MediaWiki should we support? Any idea? What Wikiapiary says?

Xqt added a comment.EditedApr 7 2020, 9:30 AM

Wikiapiary says that nearly 2/3 of all wikis use versions from obsolete branches but this analysis is not upto date because currently only MW 1.31, 1.33 and 1.34 hasn't reached its sunset. Currently we support 92 % of wikis up to mw1.14. With dropping 1.18 and below the supported wikis decrease to 64 %. The line of 50 % is with mw 1.21 and above. The most used mw version are 1.26 with 16 % and 1.16 with 10 %. LTS are mw 1.19, 1.23, 1.27, 1.31 and 1.35.

Supporting older mw versions has following aspects:

  • We should support important mw versions if pywikibot is used there; the last aspect cannot be validated I guess. I think the supporting line should not decrease 50 % then.
  • On the other hand supported mw versions must be tested. Can we find a test cluster for all of them? Does it increase CI tests a lot?
  • Older mw may make it more difficult for further developing. We are only a few developers and we cannot do both: Keeping pace with ongoing development of current mw versions and support older mw. We cannot spend a lot of time for older versions with a handful people.
  • Probably we can solve this problem above with a better versioning of our framework and a support matrix published somewere which shows the Python versions and MediaWiki versions supported. Maybe drop support of Python 2.7 and Python 3.4 and MediaWiki below 1.19 with next say 4.0 framework version. The next step can be dropping Python 3.5 and mw below 1.23 in 2021 with 5.0 Framework or sth like this.
Dvorapa added a comment.EditedApr 7 2020, 9:37 AM

Maybe drop support of Python 2.7 and Python 3.4 and MediaWiki below 1.19 with next say 4.0 framework version. The next step can be dropping Python 3.5 and mw below 1.23 in 2021 with 5.0 Framework or sth like this.

Seems reasonable, I support this

Xqt added a comment.Apr 7 2020, 10:22 AM

Seems reasonable, I support this

Should we start a new task to collect major changes pushed with next major pwb release?

Looking at Gerrit and pip/setuptools:

https://www.gerritcodereview.com/roadmap.html
https://setuptools.readthedocs.io/en/latest/history.html

What about this: PyPI description is so long currently. We could move history to a different place, but leave a link here. Then we could split current decomissions from HISTORY.rst into a separate file ROADMAP.rst (as HISTORY.rst is now even longer) and add these future release features into it. The structure would then be like this:

ROADMAP.rst
Roadmap
=======

Next release
------------

* End support of Python 2 and Python 3.4
* End support of MW < 1.19
* Move to SemVer
* Remove 10 years old desupports

Future releases
---------------

* **decomissions
HISTORY.rst
History
=======

3.0.20200326
------------

* 
* 
*
PyPI
Roadmap
=======

_include: ROADMAP.rst

History
=======

See <https://doc.wikimedia.og/pywikibot/master/history>
doc.wikimedia.org/pywikibot/master/history
History
=======

_include: HISTORY.rst

or just

PyPI
Roadmap
=======

_include: ROADMAP.rst

History
=======

See <https://gerrit.wikimedia.org/r/plugins/gitiles/pywikibot/core/+/refs/heads/master/HISTORY.rst>

I asked for an account on kernel.org family wikis, which seem to be active and have MW 1.19. I hope T249114 will not break their response to my request.

Mentioned in SAL (#wikimedia-cloud) [2020-05-10T08:22:25Z] <Dvorapa> regenerating credentials to kernel.org wikis - T245350

Change 598976 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [doc] Use python2 release for older mw versions

https://gerrit.wikimedia.org/r/598976

Change 598976 merged by jenkins-bot:
[pywikibot/core@master] [doc] Use python2 release for older mw versions

https://gerrit.wikimedia.org/r/598976