Mandatory dependency on mwparserfromhell
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	jayvdb
	Jul 23 2015, 11:23 PM

Description

mwparserfromhell includes a tokenizer of wikitext, with a cython version for efficiency.
A mandatory dependency will allow extract_templates_and_params_regex and _ETP_REGEX to be removed, and will mean the result from extract_templates_and_params will be consistent on all platforms.

There was a mandatory dependency on mwparserfromhell for unix, however it was made completely optional in T78378: Depend on mwparserfromhell consistently across operating systems due to v0.4 (2015-05-24) failing to install on Python 3.5 as was discussed at T68010#1306226. I believe this is still not fixed.

We hope to re-enable Python 3.5 builds soon in T106224: Python 3.5/3.6 compatibility, but they might be allowed-failures.

Details

Subject	Repo	Branch	Lines +/-
[parser] Make mwparserfromhell or wikitextparser mandatory	pywikibot/core	master	+20 -60
[parser] Deprecate textlib usage without MediaWiki markup parser	pywikibot/core	master	+111 -137
[parser] Make mwparserfromhell or wikitextparser mandatory	pywikibot/core	master	+100 -165

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T151110 RFC: Regressions of core should be rethought or solved
Resolved	Dvorapa	T186394 solve_disambiguation uses slice operator for PageGenerator
Open	None	T118719 solve_disambiguation.py gives possibilities from {{disambiguation}}
Resolved	JeanFred	T240312 Error with curly braces in selector (eg when using subquery or UNION)
Open	None	T229723 Insufficient wikitext regex parser functions in textlib (tracking)
Duplicate	None	T227386 template.py fails removing a template
Resolved	Xqt	T110529 template.py does not recognize nested templates
Open	None	T106765 migrate textlib parser functionality to mwparserfromhell
Resolved	None	T278454 templatesWithParams misses template where a parameter value has a curly brace
Resolved	None	T207582 templatesWithParams() doesn't seem to get all templates in Hebrew
Resolved	Xqt	T106763 Mandatory dependency on mwparserfromhell
Resolved	Ladsgroup	T88069 Add pure python mwparserfromhell to nightlies

Event Timeline

jayvdb created this task.Jul 23 2015, 11:23 PM

jayvdb raised the priority of this task from to High.

jayvdb updated the task description. (Show Details)

jayvdb added a project: Pywikibot.

jayvdb added subscribers: jayvdb, Ricordisamoa, Earwig, Xqt.

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJul 23 2015, 11:23 PM

jayvdb added a project: Pywikibot-textlib.py.Jul 23 2015, 11:24 PM

This task is what I actually meant T78378: Depend on mwparserfromhell consistently across operating systems to be.

jayvdb moved this task from Backlog to Improvements on the Pywikibot-textlib.py board.Jul 23 2015, 11:55 PM

jayvdb added a parent task: T106765: migrate textlib parser functionality to mwparserfromhell.Jul 23 2015, 11:57 PM

Sorry, I forgot I had an unreleased fix for the Python 3.5 issue that you've been waiting on. I'm back to working on the parser after a little break so it should come soon.

Okay! I released mwparserfromhell 0.4.1 (and 0.4.2, because I made a mistake...) just an hour ago, which fixes the Python 3.5 issue. I also have Windows binary releases working properly thanks to Appveyor.

Thanks @Earwig !
mwpfh 0.4.2 is passing our tests on 2.6-3.4. We'll need to do a manual check against Python 3.5/3.6, as our automated tests are not running on Python 3.5/3.6 yet due to what we think is a bug in Python (T106212), but as your Python 3.5-dev tests are passing (https://travis-ci.org/earwig/mwparserfromhell/builds/73326035), ours should also pass.

We can now seriously consider enabling mwparserfromhell as a mandatory dependency for Win32 users.

jayvdb added a subtask: T88069: Add pure python mwparserfromhell to nightlies.Aug 5 2015, 4:47 PM

Ladsgroup closed subtask T88069: Add pure python mwparserfromhell to nightlies as Resolved.Aug 30 2015, 2:53 PM

jayvdb mentioned this in T113914: Perform automated testing of nightlies.Sep 27 2015, 11:05 PM

Dalba subscribed.Apr 11 2016, 11:53 PM

Dalba added a subtask: T118719: solve_disambiguation.py gives possibilities from {{disambiguation}}.Jul 15 2016, 2:04 PM

Dalba removed a subtask: T118719: solve_disambiguation.py gives possibilities from {{disambiguation}}.Jul 15 2016, 2:08 PM

Dalba added a parent task: T118719: solve_disambiguation.py gives possibilities from {{disambiguation}}.

Xqt lowered the priority of this task from High to Lowest.Feb 5 2018, 4:09 AM

Is this being blocked by T225591: Make Travis and Appveyor pass again?

Xqt added a parent task: T229723: Insufficient wikitext regex parser functions in textlib (tracking).Aug 3 2019, 9:50 AM

Change 675170 had a related patch set uploaded (by Xqt; author: Xqt):
[pywikibot/core@master] [mwpfh] Make mwparserfromhell mandatory