Its really strange that this is causing wikidata on branch REL1_25 to fail. Since REL1_25 was last updated sometime last year and that patch was uploaded nearly 30 days ago.

@Mattflaschen ok. Is there any way we could prevent this happening to the branches in the future.

@Mattflaschen would you like me to do the assert.expect changes or are you doing them.

In T126073#2004134, @Mattflaschen wrote:

You can. I'll try to fix T103346: phpunit hhvm failure: LuaSandbox: TextLibraryTests[87]: json decode, invalid values (trailing comma) upstream. If you get stuck, ping me.

We should do it properly, though, not just manually change the files. So if don't want to do that, I'll take a shot at it.

@Mattflaschen ok will do it in a sec seems that tests on REL1_26 also fail but not as many fail. So for example scribunto fails with the same bug so I backported the change. But qunit fails with some kind of js file in core I think. also phplint fails with https://integration.wikimedia.org/ci/job/mediawiki-core-php53lint/1077/console

@Paladox Okay, I tried to figure out what you need to do. I think you need to update Wikibase composer.json on REL1_25 to a version with the expect fix .

However, the complication is that the version with the fix (7925b64b7256902a2059273de76963d3bc3286a6) is quite different from the version currently there on REL1_25 (0.14.0).

You could create a branch based on 0.14.0, then cherry-pick 7925b64b7256902a2059273de76963d3bc3286a6 on top, but that might get complicated.

I will ask the Wikidata people whether the build bot runs on release branches like REL1_25 (it doesn't seem so), and about this bug.. If not, we may want to go with your approach of fixing it manually.

The other approach is to run whatever version of jshint used to work. But I don't know if it's possible to run multiple versions of it. @hashar?

@Mattflaschen I found a better solution we could cut off unit tests with a config so I have been going through all of wikidata doing that. Ive name the config $wmgUseWikidataTest and disabled it by default. that way it isent a big change and can be done at a later point.

@Mattflaschen yes it is possible by using grunt or npm.

But we would need to do that In mediawiki/core.

In T126073#2004334, @Paladox wrote:

@Mattflaschen I found a better solution we could cut off unit tests with a config so I have been going through all of wikidata doing that. Ive name the config $wmgUseWikidataTest and disabled it by default. that way it isent a big change and can be done at a later point.

I don't really want to disable all QUnit on the release branches just for a missing expect count. Even manually fixing it is better than that, IMHO, if the build bot isn't running anyway.

ValueView is not the only problem its repo, lib, view possibly others.

In T126073#2004067, @Paladox wrote:

@Mattflaschen ok. Is there any way we could prevent this happening to the branches in the future.

Making sure that you always run the branch with the CI tools (QUnit, etc.) it was tested with. I think that is basically the composer test/npm test approach. @hashar and @Krinkle probably know about this issue.

I think that the issues come into play when they are creating the branches for release they set it back in certain time so it may be fixed In master but broke in the release branch.

The fix to Wikidata to make it pass that job on master was done during T117886.

This problem was created because this job didn't run for Wikidata before, so when we enabled the job it was also enabled for old release branches, even though the fix to make the job pass was only merged in master.

Builds for anything except master are done manually. You can update only one component by passing that as an argument like: composer update data-values/value-view.

So there are a few options sorting by estimated work needed in increasing order:

1. Disable Wikidata and its tests on the release branches.
2. Make the Job skip Wikidata on release branches. Like https://gerrit.wikimedia.org/r/#/c/268790/ but it is more complicated, as the job mediawiki-extensions-qunit also contains a list of repos it pulls in when it is triggered from other repos. This can probably be done by duplicating this and the other gate jobs with one copy per release branch, each having a different set of extensions, but that is probably more clutter than is worth for old extensions without users (otherwise it would be worth it to fix the extension instead).
3. Backport fixes, which probably involves a new release of a few components branched from the version used in the release branch and at least one backport into the Wikibase release branch, then updating the Wikidata build from all that.
4. Setting the release branch to the current master version via a merge. (We probably broke compatibility with older core versions often enough, that this would involve quite some changes to make things backwards compatible.)

To put this into perspective there are probably 0 users of Wikibase on release versions. I think I prefer solution 1, but find 3 acceptable (even though 3 is probably not worth the effort).

It seems https://gerrit.wikimedia.org/r/#/c/268828/ is a combination of 1 and 3, I will comment there.

In T126073#2004067, @Paladox wrote:

@Mattflaschen ok. Is there any way we could prevent this happening to the branches in the future.

We probably would need to change how new jobs are enabled so that it doesn't affect old branches, which is not a small task.

5. Fix the Wikidata extension directly on the release branch, without the build process. @aude okayed this on IRC (unless I misunderstood) on Friday.

#2 has the downside that we will not get the CI benefits of this job for the release branches, which we will if we do a one-time fix.

Yes I'm fine with 5, too.

https://gerrit.wikimedia.org/r/#/c/269310/ is an implementation for 1