- Name of tool/project: Extension:Linter
- Project home page: https://www.mediawiki.org/wiki/Extension:Linter
- Name of team requesting review: Parsing
- Primary contact: @Legoktm and @Arlolra
- Target date for deployment: Before end of 2016
- Link to code repository / patchset: mediawiki/extensions/Linter in Gerrit, additionally the linter code in Parsoid is mostly in lib/wt2html/pp/handlers/linter.js and lib/logger/linter.js
Description of the tool/project
T48705: Parsoid-based wikitext "linting" tool for "buggy" / "deprecated" wikitext usage; keywords: broken wikitext information
Parsoid has a linter that can identify common errors in wikitext (deprecated elements, fostered content, bad image options, etc.). The extension collects them in a database table, and surfaces them to users. It additionally has some small client-side JS that highlights the section of wikitext with the error to make it easier for editors to fix.
Description of how the tool will be used at Wikimedia
- Random user saves a page
- change-prop asks restbase to re-render the page
- note that the originally edited page might have been a template, and change-prop is re-rendering dependent pages
- restbase asks parsoid to re-render the page
- parsoid does so and spots a lint error
- parsoid sends an API request to MW with a JSON encoded set of warnings, and some metadata about them
- even if there are no lint errors, parsoid will still send an API response to tell MW that any previous warnings may have been fixed
- Parsoid (technically it's not a dependency, but the extension won't do anything without Parsoid sending it data)
Has this project been reviewed before?
please link to tasks or wiki pages of previous reviews
Working test environment
please link or describe setup process for setting up a test environment
You'll need to set up parsoid, and configure it to use your local wiki. Also set the two lint settings as specified on https://www.mediawiki.org/wiki/Extension:Linter in parsoid's config.yaml.
Install the MW extension like normal (wfLoadExtension). If Parsoid is running on another host, you'll need to configure $wgLinterSubmitterWhitelist, as by default it only allows requests from localhost.
I don't know how to set up the whole changeprop/restbase pipeline, so I would usually go to http://localhost:8000/localhost/v3/page/html/Using_big_template in my browser which has parsoid parse the page and submit lint errors. An easy lint error to test is the usage of <big> tags.
name of team responsible for tool/project after deployment and primary contact
The API module action=record-lint uses an IP whitelist ($wgLinterSubmitterWhitelist) before accepting any data. Any client that passes the whitelist is considered trusted. If it's not whitelisted, the data is discarded.
The other main concern is accidentally DoSing ourselves during the initial rollout, as we expect that there are a large number of errors that already exist, and would want to do a lot of DB writes. There is also a linterAPISampling setting which can be set in parsoid's config.yaml to only submit lint errors for some pages. We could use this to gradually ramp up the number of requests that are going to the MW API. Additionally, the linting variable can be a list of lint warning categories to enable (e.g. "obsolete-tag"), so we can reduce it that way.