Page MenuHomePhabricator

Provide additional Linter statistics about template-generated issues
Open, MediumPublic

Description

Sometimes, a small number of templates are responsible for a large number of individual linter issues that are reported by Linter. In order to enable editors to not get overwhelmed by the counts, it might be useful to provide additional summary statistics:

  • How many linter issues in a category come from templates, and from top-level page itself? This statistic gives a quick scale of the effort involved.
  • For those pages with template-generated linter issues, some sort of ordered listing of templates by most # of issues so that editors can address those first.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ssastry triaged this task as Medium priority.Jul 17 2017, 7:28 PM

Looks like this is a bit involved since the linter db schema stuffs all information in the linter_params json blob. We didn't think of this use case when we were developing the schema.

<legoktm> we'd need to do a schema change and add a new column for it, and then index that column

Anyway, so this won't happen right away, but we will get this done.

On ptwiki, I prioritized the list of "tidy-font-bug" by starting with the most transcluded templates containing errors:
https://quarry.wmflabs.org/query/23521
Something like that could be useful for others too..

On ptwiki, I prioritized the list of "tidy-font-bug" by starting with the most transcluded templates containing errors:
https://quarry.wmflabs.org/query/23521
Something like that could be useful for others too..

Thanks, He7d3r. How about adding a line about it at https://www.mediawiki.org/wiki/Help:Extension:Linter/tidy-font-bug ?

This would be a useful addition, if it is possible. I have fixed a few thousand pages with Linter errors, including many transcluded pages, on en.WP, and here are some things to consider:

  • Sometimes a template is listed as having a Linter error because there is a Linter error in the text of its documentation, and the template transcludes the documentation. Since the documentation is noincluded, when the template is transcluded in another page, that page may not have an error.
  • Conversely, a template may produce Linter errors ONLY when it is transcluded, due to things like if statements and processing of parameters.
  • Some pages on the Linter error lists show a template that is allegedly the source of the Linter error, but now that almost all template Linter errors on en.WP have been fixed, the error is more likely to be present in the wikicode in the template's transclusion on the listed page itself, not something that can be fixed inside the template's code in Template space.
  • Many transcluded pages do not live in Template space. On en.WP, userboxes often live in the User space.
Sbailey subscribed.

Now that there is a linter_template text field in the database, adding features like this is now feasible.
Evaluating feature list value and prioritizing.

I was asked to comment on this. I'm not sure about the final design, product-wise but if you try to make it work as a QueryPage (an example: Special:WantedPages), it's going to be quite easy to build and maintain and you get all of the scalability protections (querying vslow, caching, etc.) for free and wouldn't need to worry about it at all. Something like this: https://github.com/wikimedia/mediawiki/blob/master/includes/specials/SpecialMostLinkedCategories.php. HTH

Change #1233925 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/extensions/Linter@master] Add a QueryPage for templates by number of lints in a cat

https://gerrit.wikimedia.org/r/1233925

Change #1233925 merged by jenkins-bot:

[mediawiki/extensions/Linter@master] Add a QueryPage for templates by number of lints in a cat

https://gerrit.wikimedia.org/r/1233925

Running updateSpecialPage on beta cluster consistently fatals with this:

Wikimedia\Rdbms\DBQueryError from line 1226 of /srv/mediawiki-staging/php-master/includes/libs/Rdbms/Database/Database.php: Error 1366: Incorrect integer value: 'Template:Delete' for column `aawiki`.`querycachetwo`.`qcc_namespace` at row 1
Function: MediaWiki\Linter\SpecialLintTemplateErrors::recache
Query: INSERT INTO `querycachetwo` (qcc_type,qcc_namespace,qcc_title,qcc_value,qcc_titletwo) VALUES ('LintTemplateErrors','Template:Delete','Template:Delete',1,'night-mode-unaware-background-color')

#0 /srv/mediawiki-staging/php-master/includes/libs/Rdbms/Database/Database.php(1210): Wikimedia\Rdbms\Database->getQueryException('Incorrect integ...', 1366, 'INSERT INTO `qu...', 'MediaWiki\\Linte...')
#1 /srv/mediawiki-staging/php-master/includes/libs/Rdbms/Database/Database.php(1184): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('Incorrect integ...', 1366, 'INSERT INTO `qu...', 'MediaWiki\\Linte...')
#2 /srv/mediawiki-staging/php-master/includes/libs/Rdbms/Database/Database.php(641): Wikimedia\Rdbms\Database->reportQueryError('Incorrect integ...', 1366, 'INSERT INTO `qu...', 'MediaWiki\\Linte...', false)
#3 /srv/mediawiki-staging/php-master/includes/libs/Rdbms/Database/Database.php(1506): Wikimedia\Rdbms\Database->query(Object(Wikimedia\Rdbms\Query), 'MediaWiki\\Linte...')
#4 /srv/mediawiki-staging/php-master/includes/libs/Rdbms/Database/DBConnRef.php(129): Wikimedia\Rdbms\Database->insert('querycachetwo', Array, 'MediaWiki\\Linte...', Array)
#5 /srv/mediawiki-staging/php-master/includes/libs/Rdbms/Database/DBConnRef.php(457): Wikimedia\Rdbms\DBConnRef->__call('insert', Array)
#6 /srv/mediawiki-staging/php-master/includes/libs/Rdbms/QueryBuilder/InsertQueryBuilder.php(317): Wikimedia\Rdbms\DBConnRef->insert('querycachetwo', Array, 'MediaWiki\\Linte...', Array)
#7 /srv/mediawiki-staging/php-master/extensions/Linter/includes/SpecialLintTemplateErrors.php(214): Wikimedia\Rdbms\InsertQueryBuilder->execute()
#8 /srv/mediawiki-staging/php-master/extensions/Linter/includes/SpecialLintTemplateErrors.php(135): MediaWiki\Linter\SpecialLintTemplateErrors->recacheInternal(5000, 'night-mode-unaw...')
#9 [internal function]: MediaWiki\Linter\SpecialLintTemplateErrors->MediaWiki\Linter\{closure}(0, 'night-mode-unaw...')
#10 /srv/mediawiki-staging/php-master/extensions/Linter/includes/SpecialLintTemplateErrors.php(132): array_reduce(Array, Object(Closure), 0)
#11 /srv/mediawiki-staging/php-master/maintenance/updateSpecialPages.php(78): MediaWiki\Linter\SpecialLintTemplateErrors->recache(5000)
#12 /srv/mediawiki-staging/php-master/maintenance/includes/MaintenanceRunner.php(694): UpdateSpecialPages->execute()
#13 /srv/mediawiki-staging/php-master/maintenance/run.php(53): MediaWiki\Maintenance\MaintenanceRunner->run()
#14 /srv/mediawiki-staging/multiversion/MWScript.php(219): require_once('/srv/mediawiki-...')
#15 {main}

Change #1237532 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[mediawiki/extensions/Linter@master] SpecialLintTemplateError: Fix bug in the query

https://gerrit.wikimedia.org/r/1237532

This should fix it. See SpecialMostLinkedCategories for comparison of query info. I‌ want to test it locally but composer is not cooperating right now.

This should fix it. See SpecialMostLinkedCategories for comparison of query info. I‌ want to test it locally but composer is not cooperating right now.

Well, my localhost would have actually work if it had any database (fun part of a new laptop). Anyway, I‌ changed it in deploy04 VM in beta cluster (I‌'d deny it if RelEng asks) and it fixed the bug. Now it works like a charm!

https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintTemplateErrors/night-mode-unaware-background-color

Running updateSpecialPage on beta cluster consistently fatals with this

Evidently I have some type coercion locally

Now it works like a charm!

🥳 Thanks

Change #1237532 merged by jenkins-bot:

[mediawiki/extensions/Linter@master] SpecialLintTemplateError: Fix bug in the query

https://gerrit.wikimedia.org/r/1237532

For tech news:

There is a new special page listing templates that trigger the largest number of lint errors. For example: [[Special:LintTemplateErrors/night-mode-unaware-background-color]]

Edit mercilessly.

It is worth noting on help pages (or on the special page listing itself) that the reason some template might repeatedly show up is not because of the template source but because of the args passed into the template. Even in those cases, it is is still an useful signal to focus on its uses on article pages.

Re: Tech News - Thanks for the draft! I'll postpone including this until next week's edition so that editors can immediately see the content when they read about, as I see it is not yet available on production wikis (e.g. Enwiki's version of that example page) (but is on Beta Cluster's version).

Re: the Special pages/feature itself:

  1. Are there more of these beyond the single example linked?
    • 1b. Could/Should there be a listing page at the root level? I.e. I imagine many editors will check Special:LintTemplateErrors directly hoping for a listing of existing subpages, but there's nothing there currently except a Category not found or malformed error message: https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintTemplateErrors -- Or is there a listing elsewhere that the root-page could automatically link to?
  2. Could/should that new page(s), be linked to from the related Special:LintErrors page, for easier discoverability? https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintErrors
  3. Plus just emphasizing Subbu's comment above: Could/should that be information about that added directly into the https://www.mediawiki.org/wiki/Help:Extension:Linter page? And/Or into the Special sub-pages themselves?
    • 3.b. If/When the page(s) are covered by documentation, could a standard "Help" indicator be added to the page(s)?

It is worth noting on help pages (or on the special page listing itself) that the reason some template might repeatedly show up is not because of the template source but because of the args passed into the template. Even in those cases, it is is still an useful signal to focus on its uses on article pages.

I tried to highlight this way back in February 2019, but the wording on the pages seems to miss this important point.

The page on the beta cluster at https://en.wikipedia.beta.wmcloud.org/w/index.php?title=Special:LintTemplateErrors/night-mode-unaware-background-color&limit=500&offset=0 is called "Templates generating lint errors", but that page is actually showing "Transclusions containing Linter error night-mode-unaware-background-color". It is appropriate that the page lists pages outside of Template space, because things like userboxes and user-space header pages can contain Linter errors in their transclusions, but the page name is wrong in multiple ways. Please fix.

The page description is similarly incorrect. It says "These templates are generating night-mode-unaware-background-color lint errors." It should say something like "Transclusions of these pages contain ...." Please fix the wording.

Again, it is not necessarily the transcluded pages themselves that are generating Linter errors; adjusting code in the transcluded page is often not the correct fix, especially more than eight years into this de-linting project.

For tech news:

There is a new special page listing templates that trigger the largest number of lint errors. For example: [[Special:LintTemplateErrors/night-mode-unaware-background-color]]

Edit mercilessly.

There is a new special page listing transcluded pages that are flagged as containing lint errors, sorted by the number of transclusions with errors. For example: [[Special:LintTemplateErrors/night-mode-unaware-background-color]]

Change #1238421 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/extensions/Linter@master] Update title / desc of Special:LintTemplateErrors

https://gerrit.wikimedia.org/r/1238421

Change #1238421 merged by jenkins-bot:

[mediawiki/extensions/Linter@master] Update title / desc of Special:LintTemplateErrors

https://gerrit.wikimedia.org/r/1238421

Change #1238427 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/extensions/Linter@wmf/1.46.0-wmf.15] Update title / desc of Special:LintTemplateErrors

https://gerrit.wikimedia.org/r/1238427

Change #1238427 merged by jenkins-bot:

[mediawiki/extensions/Linter@wmf/1.46.0-wmf.15] Update title / desc of Special:LintTemplateErrors

https://gerrit.wikimedia.org/r/1238427

Mentioned in SAL (#wikimedia-operations) [2026-02-10T21:19:13Z] <kemayo@deploy2002> Started scap sync-world: Backport for [[gerrit:1238430|Edit check suggestions beta-feature in allowlist (T399611)]], [[gerrit:1238049|Turn on Parsoid read views by default on labs (take 2) (T357054)]], [[gerrit:1238427|Update title / desc of Special:LintTemplateErrors (T170874)]]

Mentioned in SAL (#wikimedia-operations) [2026-02-10T21:44:08Z] <kemayo@deploy2002> cscott, arlolra, kemayo: Backport for [[gerrit:1238430|Edit check suggestions beta-feature in allowlist (T399611)]], [[gerrit:1238049|Turn on Parsoid read views by default on labs (take 2) (T357054)]], [[gerrit:1238427|Update title / desc of Special:LintTemplateErrors (T170874)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-02-10T21:58:59Z] <kemayo@deploy2002> Finished scap sync-world: Backport for [[gerrit:1238430|Edit check suggestions beta-feature in allowlist (T399611)]], [[gerrit:1238049|Turn on Parsoid read views by default on labs (take 2) (T357054)]], [[gerrit:1238427|Update title / desc of Special:LintTemplateErrors (T170874)]] (duration: 39m 46s)

Please fix the wording.

Should be improved by T170874#11604001 which was deployed yesterday

I ran the maintenance script for mediawikiwiki and now data is populated on https://www.mediawiki.org/wiki/Special:LintTemplateErrors/duplicate-ids and the other subpages

The task description here is also asking for

How many linter issues in a category come from templates, and from top-level page itself? This statistic gives a quick scale of the effort involved.

Is that something that would really be all that useful?

Please could someone reply to my questions/notes above (in T170874#11592944) so that we (Sandister this week) can write a good Tech News entry? (Or help us decide whether to postpone the announcement until those questions are addressed?). Thanks.

The task description here is also asking for

How many linter issues in a category come from templates, and from top-level page itself? This statistic gives a quick scale of the effort involved.

Is that something that would really be all that useful?

I imagine it would be useful for checking which category is having a lot of template issues (e.g. a template transcluded in other templates causes 100 templates to appear in the list), similar to how the base page of Special:LintErrors shows an estimate of the number of lint errors in each category. That and it would make it easier to check the subpages with errors as opposed to having to go through multiple subpages which are empty.

The new wording is much better, thank you.

I'm looking at https://www.mediawiki.org/wiki/Special:LintTemplateErrors/missing-end-tag which is populated. How do I get from "36. Template:Cquote (1)" to the list of affected pages? The template itself has no Linter errors, according to "Page information", so one of the 83 transclusions presumably has an error in it. But which one?

Please could someone reply to my questions/notes above (in T170874#11592944) so that we (Sandister this week) can write a good Tech News entry? (Or help us decide whether to postpone the announcement until those questions are addressed?). Thanks.

Maybe hold off for a bit till we look at the functionality and see what else needs tweaking (ex: Special:LintTemplateErrors erroring is probably going to be something folks will run into -- since I have run into that multiple times myself).

Re: the Special pages/feature itself:

  1. Are there more of these beyond the single example linked?
    • 1b. Could/Should there be a listing page at the root level? I.e. I imagine many editors will check Special:LintTemplateErrors directly hoping for a listing of existing subpages, but there's nothing there currently except a Category not found or malformed error message: https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintTemplateErrors -- Or is there a listing elsewhere that the root-page could automatically link to?

That's now being done in T417363

Re: the Special pages/feature itself:

  1. Are there more of these beyond the single example linked?
    • 1b. Could/Should there be a listing page at the root level? I.e. I imagine many editors will check Special:LintTemplateErrors directly hoping for a listing of existing subpages, but there's nothing there currently except a Category not found or malformed error message: https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintTemplateErrors -- Or is there a listing elsewhere that the root-page could automatically link to?

https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintTemplateErrors now contains an index and will roll out next week or I'll consider backporting it sooner.

  1. Could/should that new page(s), be linked to from the related Special:LintErrors page, for easier discoverability? https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintErrors

The individual https://en.wikipedia.beta.wmcloud.org/wiki/Special:LintErrors/duplicate-ids pages have links at the top like this,

These pages have elements with duplicate id attributes. View transclusions that contain these lint errors.

Is that enough?

  1. Plus just emphasizing Subbu's comment above: Could/should that be information about that added directly into the https://www.mediawiki.org/wiki/Help:Extension:Linter page? And/Or into the Special sub-pages themselves?
    • 3.b. If/When the page(s) are covered by documentation, could a standard "Help" indicator be added to the page(s)?

As of T170874#11604001, the top of the pages now read,

Transclusions of these pages contain duplicate-ids lint errors. Note that some transclusions might show up on this list because of the arguments passed into the transclusion and not the transclusion source.

Change #1250744 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/extensions/Linter@master] Provide a list of pages by category and template

https://gerrit.wikimedia.org/r/1250744

I'm looking at https://www.mediawiki.org/wiki/Special:LintTemplateErrors/missing-end-tag which is populated. How do I get from "36. Template:Cquote (1)" to the list of affected pages? The template itself has no Linter errors, according to "Page information", so one of the 83 transclusions presumably has an error in it. But which one?

Right, I put up a patch for that in T170874#11700296

@ABreault-WMF That all sounds and looks good to me, thanks. I now understand that the pages listed at Special:LintTemplateErrors are supplementary for the associated links at Special:LintErrors.

If we take Jonesey95's/Ladsgroup's draft message for Tech News, and add that new index link, we get this, which seems suitable for the Tech News section "Updates for technical contributors". (tweaks welcome, if anyone can see a way to improve this draft):

There is a new special page, Special:LintTemplateErrors, which lists transcluded pages that are flagged as containing lint errors, sorted by the number of transclusions with errors. For example: Special:LintTemplateErrors/night-mode-unaware-background-color.

Q:

  • If there are no more additions/changes in the code to wait for, then this could be added to the upcoming Tech News. If so, please confirm (or ideally, change the current workboard column from "Not ready to announce" to "Announce in next Tech/News").
  • If there are any more changes this should wait for, please clarify/specify. Thanks!

The draft lgtm and I moved the task on workboard. There are improvements coming but I don't think they should block the announcement at this point