Page MenuHomePhabricator

Add support for MW branches to LibUp
Closed, ResolvedPublic

Assigned To
Authored By
Reedy
Oct 21 2019, 2:20 PM
Referenced Files
None
Tokens
"Yellow Medal" token, awarded by Aklapper."Yellow Medal" token, awarded by Jdforrester-WMF.

Description

Per the parent task, T236070: Bump PHPCS version in REL1_31 extensions, it'd be useful if we could run LibUp against non master

Event Timeline

Change 586401 had a related patch set uploaded (by Reedy; owner: Reedy):
[labs/libraryupgrader@master] [WIP] Add branch support to libup

https://gerrit.wikimedia.org/r/586401

What's the best way going to be dealing with https://gerrit.wikimedia.org/r/plugins/gitiles/labs/libraryupgrader/config/+/master/releases.json ?

Add a "branch" under each library, and if defined, use that, else, fallback to "master"?

{
    "composer": {
        "mediawiki/mediawiki-codesniffer": [
            "30.0.0"
        ]
    }
}

becomes

{
    "composer": {
        "mediawiki/mediawiki-codesniffer": {
            "master": "30.0.0",
            "REL1_31": "19.1.0"
        }
    }
}

Change 586408 had a related patch set uploaded (by Reedy; owner: Reedy):
[labs/libraryupgrader/config@master] [WIP] Add support for branches to libraryupgrader config

https://gerrit.wikimedia.org/r/586408

What's the best way going to be dealing with https://gerrit.wikimedia.org/r/plugins/gitiles/labs/libraryupgrader/config/+/master/releases.json ?

Add a "branch" under each library, and if defined, use that, else, fallback to "master"?

{
    "composer": {
        "mediawiki/mediawiki-codesniffer": [
            "30.0.0"
        ]
    }
}

becomes

{
    "composer": {
        "mediawiki/mediawiki-codesniffer": {
            "master": "30.0.0",
            "REL1_31": "19.1.0"
        }
    }
}

That sounds fine. Though we should really drop arrays as targets; scalar (one value) or map of branch to one value only makes sense.

I took the long route on this and tried to refactor everything along the way...but I think it's paid off. Just about everything on the web interface https://libraryupgrader2.wmcloud.org/ now has a branch selector and the database backend treats branches as first class, separate repositories.

LibUp is currently processing REL1_31 and REL1_35 repositories and populating the database. Once that's done, I'll remove the hack that I have for now to prevent it from submitting patches and then we'll be on our way!

Change 586408 abandoned by Reedy:
[labs/libraryupgrader/config@master] Add support for branches to libraryupgrader config

Reason:

https://gerrit.wikimedia.org/r/586408

Change 586401 abandoned by Reedy:
[labs/libraryupgrader@master] Add branch support to libup

Reason:

https://gerrit.wikimedia.org/r/586401

Some extensions failing with on REL1_31:

Running "stylelint:all" (stylelint) task
Warning: Cannot find module 'stylelint' Use --force to continue.

When doing all steps manually on my local install it works, but there are a kind difference, for example these lines not showing up by me:

npm WARN grunt-stylelint@0.9.0 requires a peer of stylelint@^8.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN stylelint-config-wikimedia@0.4.2 requires a peer of stylelint@8.2.0 but none is installed. You must install peer dependencies yourself.

There is code in libup to remove explict stylelint/eslint from package.json. Maybe that needs a version check to keep it on older grunt-stylelint? (T242145: Remove 'eslint'/'stylelint' from package.json if 'grunt-eslint'/'grunt-stylelint' are installed)

The commit going much big with package-lock.json, rewrite of composer.json and all the other things.

Some extensions failing with on REL1_31:

Running "stylelint:all" (stylelint) task
Warning: Cannot find module 'stylelint' Use --force to continue.

When doing all steps manually on my local install it works, but there are a kind difference, for example these lines not showing up by me:

npm WARN grunt-stylelint@0.9.0 requires a peer of stylelint@^8.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN stylelint-config-wikimedia@0.4.2 requires a peer of stylelint@8.2.0 but none is installed. You must install peer dependencies yourself.

There is code in libup to remove explict stylelint/eslint from package.json. Maybe that needs a version check to keep it on older grunt-stylelint? (T242145: Remove 'eslint'/'stylelint' from package.json if 'grunt-eslint'/'grunt-stylelint' are installed)

Took a bit of digging, it looks like it needs at least 0.5.0 of stylelint-config-wikimedia (for https://github.com/wikimedia/stylelint-config-wikimedia/commit/2d6efa12af676e78f25f1d807245c7eee0c95e7b) before safely removing, I'll add that check in.

Everything appears to be on 0.4.x: https://libraryupgrader2.wmcloud.org/library/npm/stylelint-config-wikimedia?branch=REL1_31

The commit going much big with package-lock.json, rewrite of composer.json and all the other things.

We can selectively disable some fixers if we think they're frivolous or trivial but I think committing package-lock.json is the right thing to do. Also this seems to be really bad on REL1_31 because we haven't kept it up to date until now, whereas REL1_35 looks pretty decent https://libraryupgrader2.wmcloud.org/errors?branch=REL1_35 (just the usual suspects so far).

Oops, yeah, a number of changes were done blindly because "all repos have version xyz now", and now we support branches those assumptions aren't safe…

Change 652669 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[labs/libraryupgrader@master] Start pushing patches to non-master branches

https://gerrit.wikimedia.org/r/652669

Change 652669 merged by jenkins-bot:
[labs/libraryupgrader@master] Start pushing patches to non-master branches

https://gerrit.wikimedia.org/r/652669

It's happening \o/