Page MenuHomePhabricator

LibraryUpgrader CI normalisation tasks, June/July 2019
Open, Needs TriagePublic0 Estimate Story Points

Description

  • npm
    • General
      • npm audit fix, e.g. grunt -> 1.0.4
      • package.json should have a private property
      • Order: Ensure grunt test order has eslint before banana (as that checks the JSON)
    • eslint / jsonlint
      • .eslintrc should be renamed to .eslintrc.json - T228114
      • T206485: Set root: true in eslintrc.json
      • Ensure we're using reportUnusedDisableDirectives: true
      • Ensure we're using cache: true and that .eslintcache is in .gitignore
      • eslint-config-wikimedia -> 0.13.0 & grunt-eslint -> 21.0.0 & remove grunt-jsonlint (e.g.; needs manual fixes; can we generalise?)
    • stylelint
      • stylelint-config-wikimedia -> 0.6.0 & grunt-stylelint -> 0.11.0 & drop explicit stylelint
    • banana
      • grunt-banana-checker -> 0.7.0 (if it passes; some will fail)
  • composer
    • Order: Ensure composer fix has minus-x fix before phpcbf T214652
    • mwcs
      • Update mediawiki-codesniffer -> 26.0.0
      • Update .gitattributes for .phpcs.xml file move T183851
      • Drop "php5,inc" from .phpcs.xml files T200956
    • mediawiki-phan-config -> 0.6.1 (hits)
    • mediawiki/minus-x -> 0.3.2 (hits)
    • jakub-onderka/php-console-highlighter -> 0.4.0 (hits) - T242276
    • phan-taint-check-plugin -> 2.0.0 (hits) T227172

Details

Related Objects

Mentioned In
T242276: libup does not upgrade jakub-onderka/php-console-highlighter due to version format mismatch
T237166: Remove reportUnusedDisableDirectives when updating to eslint-config-wikimedia to 0.15.0 or higher
R2145:06a7f3814f8e: Adjust eslint config to enable caching and reporting extra disables
R2145:ff49a2477cd4: Set `root: true` in .eslintrc.json
R2145:cf48525b9688: Set `private: true` in package.json for MediaWiki extensions & skins
rLLUP06a7f3814f8e: Adjust eslint config to enable caching and reporting extra disables
rLLUPff49a2477cd4: Set `root: true` in .eslintrc.json
rLLUPcf48525b9688: Set `private: true` in package.json for MediaWiki extensions & skins
T226029: npm: upgrade stylelint to 9.8.0 or higher
Mentioned Here
T228114: Have libup rename .eslintrc to .eslintrc.json
T242276: libup does not upgrade jakub-onderka/php-console-highlighter due to version format mismatch
T233759: Update mediawiki/mediawiki-codesniffer for php7.3 support in release branches
T227172: Upgrade taint-check to 2.0.1 in all repos
T226420: Run phan secheck on PHP 7.2, not PHP 7.0
T179195: Replace use of .eslintignore with ignore in Gruntfile.js in all repos, for consistency
T204176: Decide how to configure ESLint rules and ignores
T183851: libraryupgrader needs to update .gitattributes for .phpcs.xml file move
T200956: Drop "php5,inc" from MediaWiki-CodeSniffer ruleset of extensions
T214652: Resort composer.json scripts.fix so minus-x fix comes before phpcbf
T206485: Set root: true in eslintrc.json

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 7 2019, 7:26 PM

@Umherirrender, anything else you can think of?

Jdforrester-WMF added a subscriber: Legoktm.

phan-config to 0.6.1 would be nice

The last repos for T206485 I have uploaded, also the rename from .eslintrc to .eslintrc.json - for .stylelintrc this is already done

phan-config to 0.6.1 would be nice
The last repos for T206485 I have uploaded, also the rename from .eslintrc to .eslintrc.json - for .stylelintrc this is already done

Sure, but the great thing about a bot is it will run each time and notice a far-flung new repo isn't following best practice. ;-)

Legoktm updated the task description. (Show Details)Jun 7 2019, 7:39 PM

Oops, edit conflict.

When working with .eslintrc.json be careful that TwoColConflicts has comments in it and it seems that this is support by eslint

Other points:

  • T214652?
  • T200956?
  • T183851 seems to old to need fixing
  • taint-check-plugin could be 1.5.1
  • grunt-eslint -> 21.0.0
  • banana-checker -> 0.7.0 - seems not possible, because that can break on some extensions due to require of lower case message keys
  • jakub-onderka/php-console-highlighter -> 0.4.0
  • minus -> 0.3.2
  • Some extensions have explicit stylelint in package.json, but that is now part of the config and can be removed
  • package.json should have a private property
  • eslintIgnore in package.json (T179195 / T204176)

Excellent points. (But let's not touch the eslintignore one for now as it's not settled?)

Yes, the eslintignore part is not ready

  • space vs. tabs in package.json was another point I remember, but that could already be done when libraryupgrader is writting the package.json back to the repo

Yes, the eslintignore part is not ready

  • space vs. tabs in package.json was another point I remember, but that could already be done when libraryupgrader is writting the package.json back to the repo

Yeah, I think we already get this automatically?

Woo, this is a pretty good work list :) Random questions:

package.json should have a private property

Is there a way we can automatically tell which repos shouldn't have this? e.g. I assume oojs/ui shouldn't be marked as private.

Ensure we're using reportUnusedDisableDirectives: true
Ensure we're using cache: true

Is it not possible to set these in the shared eslint config?

  • space vs. tabs in package.json was another point I remember, but that could already be done when libraryupgrader is writting the package.json back to the repo

Yeah, I think we already get this automatically?

Correct. Whenever LU touches package.json, it'll reserialize it with tabs.

Woo, this is a pretty good work list :) Random questions:

package.json should have a private property

Is there a way we can automatically tell which repos shouldn't have this? e.g. I assume oojs/ui shouldn't be marked as private.

No. If the repo starts with mediawiki/ it should be private (core/vendor/extensions/skins/tools), but for the rest I don't think we can automatically decide.

Ensure we're using reportUnusedDisableDirectives: true
Ensure we're using cache: true

Is it not possible to set these in the shared eslint config?

Sadly, no. There's an upstream task somewhere asking for it to be possible.

Legoktm updated the task description. (Show Details)Jun 10 2019, 3:27 AM

I guess seccheck should be bumped to 2.0 once it gets released and T226420 will be resolved.

Legoktm renamed this task from LibraryUpgrader CI normalisation tasks, June 2019 to LibraryUpgrader CI normalisation tasks, June/July 2019.Jul 1 2019, 10:06 PM
Daimona changed the status of subtask T227172: Upgrade taint-check to 2.0.1 in all repos from Open to Stalled.Jul 3 2019, 9:08 AM
Daimona changed the status of subtask T227172: Upgrade taint-check to 2.0.1 in all repos from Stalled to Open.Jul 6 2019, 6:08 PM

Change 523334 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[labs/libraryupgrader@master] Set private: true in package.json for MediaWiki extensions & skins

https://gerrit.wikimedia.org/r/523334

.eslintrc should be renamed to .eslintrc.json

codesearch finds no cases of this in Gerrit (https://codesearch.wmflabs.org/search/?q=%5BA-z%5D&i=nope&files=.eslintrc%24&repos=) - OK to skip?

.eslintrc should be renamed to .eslintrc.json

codesearch finds no cases of this in Gerrit (https://codesearch.wmflabs.org/search/?q=%5BA-z%5D&i=nope&files=.eslintrc%24&repos=) - OK to skip?

If we've done it all, that's great, but it'd be nice to make it automatically insisted upon for later? Low priority, of course. :-)

Change 523340 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[labs/libraryupgrader@master] Set root: true in .eslintrc.json

https://gerrit.wikimedia.org/r/523340

Change 523558 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[labs/libraryupgrader@master] Adjust eslint config to enable caching and reporting extra disables

https://gerrit.wikimedia.org/r/523558

Change 523334 merged by jenkins-bot:
[labs/libraryupgrader@master] Set private: true in package.json for MediaWiki extensions & skins

https://gerrit.wikimedia.org/r/523334

Change 523340 merged by jenkins-bot:
[labs/libraryupgrader@master] Set root: true in .eslintrc.json

https://gerrit.wikimedia.org/r/523340

Change 523558 merged by jenkins-bot:
[labs/libraryupgrader@master] Adjust eslint config to enable caching and reporting extra disables

https://gerrit.wikimedia.org/r/523558

Legoktm updated the task description. (Show Details)Jul 16 2019, 6:28 AM
Daimona updated the task description. (Show Details)Jul 16 2019, 11:51 AM

pdate mediawiki-codesniffer -> 26.0.0 is marked as ready, but I found many extensions still using 24.0.0 or 25.0.0

Example using old version for mediawiki-codesniffer: 24.0.0
MinervaNeue using old version for mediawiki-codesniffer: 24.0.0
Timeless using old version for mediawiki-codesniffer: 24.0.0

BlueSpiceFoundation using old version for mediawiki-codesniffer: 25.0.0
BoilerPlate using old version for mediawiki-codesniffer: 24.0.0
CirrusSearch using old version for mediawiki-codesniffer: 24.0.0
CollaborationKit using old version for mediawiki-codesniffer: 24.0.0
ContentTranslation using old version for mediawiki-codesniffer: 24.0.0
Copyvio using old version for mediawiki-codesniffer: 24.0.0
CrawlableAllPages using old version for mediawiki-codesniffer: 24.0.0
DonationInterface using old version for mediawiki-codesniffer: 25.0.0
EditUser using old version for mediawiki-codesniffer: 24.0.0
Flow using old version for mediawiki-codesniffer: 24.0.0
GettingStarted using old version for mediawiki-codesniffer: 24.0.0
GoogleLogin using old version for mediawiki-codesniffer: 24.0.0
GraphQL using old version for mediawiki-codesniffer: 25.0.0
ImageMetrics using old version for mediawiki-codesniffer: 24.0.0
JsonConfig using old version for mediawiki-codesniffer: 24.0.0
Memento using old version for mediawiki-codesniffer: 24.0.0
MissedPages using old version for mediawiki-codesniffer: 24.0.0
MobileFrontend using old version for mediawiki-codesniffer: 24.0.0
MultimediaViewer using old version for mediawiki-codesniffer: 24.0.0
NewUserActions using old version for mediawiki-codesniffer: 24.0.0
PluggableAuth using old version for mediawiki-codesniffer: 24.0.0
Popups using old version for mediawiki-codesniffer: 24.0.0
PreferencesList using old version for mediawiki-codesniffer: 24.0.0
PropertySuggester using old version for mediawiki-codesniffer: 24.0.0
RelatedArticles using old version for mediawiki-codesniffer: 24.0.0
RelatedSites using old version for mediawiki-codesniffer: 24.0.0
Renameuser using old version for mediawiki-codesniffer: 24.0.0
ReplaceText using old version for mediawiki-codesniffer: 24.0.0
Scribunto using old version for mediawiki-codesniffer: 24.0.0
VEForAll using old version for mediawiki-codesniffer: 24.0.0
WhoIsWatching using old version for mediawiki-codesniffer: 24.0.0

Is that another bug?

25.0.0 is currently marked as a "good version" so I think LibUp won't try to upgrade from 25 to 26. Don't know about the repos on 24.

25.0.0 is currently marked as a "good version" so I think LibUp won't try to upgrade from 25 to 26. Don't know about the repos on 24.

It should always want to go to the latest. I don't remember why I put 25 as good tbh, it doesn't make much sense. Having multiple "good versions" doesn't make sense at all really. Dunno what I was thinking at the time.

pdate mediawiki-codesniffer -> 26.0.0 is marked as ready, but I found many extensions still using 24.0.0 or 25.0.0

For all the extensions that start with R, that's my fault. I killed it in the middle, saw that it was at R, and then restarted it for all the ones after R. The next run should take care of it.

For the rest...I don't know yet. I finished the basic debug logging for libup, and now on each repo page there's a button to view the log file: https://libraryupgrader2.wmflabs.org/r/mediawiki/extensions/3D if there was an error, it should say "View error log" instead. I'll create a separate page to just list repositories with errors later tonight. Note that this is still rolling out, it'll take a few hours for all repos to have the log posted.

This is a great task, thank you everyone involved!

Similar to the normalisation tasks being done here, we had a backport to CentralAuth REL1_31 that fails CI due to mediawiki-codesniffer 17 being incompatible with php 7.3. So I have bumped it to 19.1.0 and ignored any reported failure to avoid code changes ( https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/CentralAuth/+/538973/ ).

I am thus wondering whether we should try to update codesniffer on release branches? I filled T233759 about it.

Legoktm updated the task description. (Show Details)Fri, Jan 10, 2:59 AM
Legoktm updated the task description. (Show Details)