Page MenuHomePhabricator

Silent batch-edit request for PHP 8.4/8.5 CI failure tasks currently tagged with the 'WMF-deployed Build Failure' project
Closed, DeclinedPublic

Description

Reason: Per this clarification by @Krinkle (the third edit of the three highlighted by that link), the scope of the ci-test-error (WMF-deployed Build Failure) tag doesn't include CI failures that only occur when running check experimental (in this case, CI errors on PHP 8.4 & 8.5).

See also this discussion in #mediawiki-core on IRC:

2026-01-02 22:56:11 <A_smart_kitten> Reedy: this is probably a relatively minor point (although in any event probably good to clarify understandings), but should tasks for PHP 8.4/8.5 CI failures be tagged with the WMF-deployed build failure Phab project?
2026-01-02 22:56:28 <A_smart_kitten> just asking the understanding that i'd personally picked up over time was that the 'WMF build failure' project was for CI errors that were blocking the merging of patches in WMF-deployed repos / that were in the main/gate-and-submit tests.
2026-01-02 22:56:30 <A_smart_kitten> it's very possible that i have a mistaken understanding, though :)
2026-01-02 22:56:31 <Reedy> TBH, it's very unclear
2026-01-02 22:56:42 <Reedy> For me, if it is stuff we deploy to WMF... yes
2026-01-02 23:00:12 <Reedy> https://phabricator.wikimedia.org/project/view/4742/
2026-01-02 23:00:18 <Reedy> >This tag is used to track issues with Jenkins jobs (in the test or gate pipeline of Zuul, or post-merge in Travis CI) that are failing due to the master branch of a WMF-deployed repository having reached a state that is not consistently passing its own tests.
2026-01-02 23:06:39 <Reedy> A_smart_kitten: and more specifically...
2026-01-02 23:06:40 <Reedy> >MediaWiki core tests failing on Travis CI for PHP versions, or database backends, not yet covered by our Jenkins jobs.
2026-01-02 23:06:56 <Reedy> I guess that travis comment is irrelevant now (let me fix it)
2026-01-02 23:09:02 <A_smart_kitten> I was gonna say, I assume it at least predates when I got more actively involved in wikimedia dev stuff :)
2026-01-02 23:11:05 <Reedy> It's been a few years since travis swapped for github actions on many repos
2026-01-02 23:12:13 <A_smart_kitten> anyhow, fair fair, that example in the Phab project description clears up the scope for me then :) mostly i just wanted to make sure i was working off the same understanding as everyone else about it
2026-01-02 23:18:21 <A_smart_kitten> the only other thing that occured to me was in case anyone wanted to track how often CI errors blocked merges in a Wikimedia-deployed repo (and wanted to use the wmf-build-failure project to do that)
2026-01-02 23:19:53 <Reedy> AFAIK we haven't tried to model that...
2026-01-02 23:20:23 <Reedy> measure
2026-01-02 23:34:49 <A_smart_kitten> fair enough
2026-01-04 06:53:15 <Krinkle> Reedy: I removed "MediaWiki core tests failing on PHP versions [..] not yet covered [..] (for example our `experimental` jobs)." because those would not be high priority to unbreak (and aren't regressions). The Travis version of that bullet point was about PHP versions that we supported and were passing but couldn't be run in Jenkins for technical reasons (eg missing packages or base images).
2026-01-04 06:54:49 <Krinkle> we track them under the PHP X.Y tags and umbrella tasks as well which seems a lot/enough. Happy to change if you think that'd be useful but afaik we haven't used it for that in recent history, and before then we didn't either for experimental / never-before-passing issues.

cc @Reedy also for info

Event Timeline

I think of PHP 8.3 support and ci-test-error as different, not overlapping. The former for fixing issues before CI is enabled for a PHP version (or issues not covered by CI), and the latter for CI errors (i.e. Jenkins-Bot sets V-1 and prevents merging a patch, because one or more jobs started failing).

I know that experimental jobs are technically "jobs" that can "fail", but that does not make them CI errors and does not prevent merging a patch. We also use job status for other things, such as for config diffs in operattions/mediawiki-config and integration/config, but those are not considered CI errors, either.

Anyway, I'm neutral on this so long as we:

  1. tag it with PHP 8.3 support which is what we use as the main way to discover and prioritise PHP support work,
  2. don't tag it with ci-test-error (WMF-deployed Build Failure), which we use to track CI-blockers/regressions.

Afaik we didn't tag them also with ci-test-error in the past and I'd be fine with that, but if others find it useful, then adding it seems fine by me as I don't use the parent tag myself.

@A_smart_kitten Can you share a bit about your workflow and why you would like to add these? Or would you be find with untagging these?

@A_smart_kitten Can you share a bit about your workflow and why you would like to add these?

@Krinkle Sure :) My reason for adding them was basically 'just' because they are CI/test failures, so I thought that they would be within the scope of the (parent) ci-test-error project (and therefore, that it might be helpful for me to include that tag).
(It occurs to me that @Reedy might also have thoughts on this)

I think of PHP 8.3 support and ci-test-error as different, not overlapping.

FWIW -- to my mind, I guess the difference between a task tagged with both tags vs. a task tagged with just PHP 8.5 support would (have) be(en):

  • PHP 8.5 support & ci-test-error: PHP compatibility issues that show up in CI / that cause CI tests to fail
  • Just PHP 8.5 support: PHP compatibility issues that didn't show up in CI / tasks that aren't reporting a CI test failure

[...] would you be find with untagging these?

Personally speaking, I have no objection to the ci-test-error tag being removed from PHP 8.4 support & PHP 8.5 support tasks (Phab query results), so long as folks are okay with those sorts of tasks being defined as out-of-scope for the ci-test-error tag (which I'm personally ambivalent on right now & wouldn't object to if other folks are okay with it -- mostly I'd be happier just to be working off a shared understanding of what's in/out of scope :)).

... although, I guess another question I'd therefore have would be: what about (other) non-voting CI jobs? Would they be in-/out-of-scope for ci-test-error? (Or e.g. would we just be defining [PHP] experimental jobs as being out-of-scope, at least for now?)

I guess I'm just sitting here with popcorn waiting for potential agreement / a decision. Or someone saying "let's decline for now". Hmm.

8.4 will be blocking CI soon enough anyway, and 8.5 soon-ish. Let's just decline?

FWIW, In the interests of being on the same page / coming to a conclusion here, I'm happy for ci-test-error as well as ci-test-error (WMF-deployed Build Failure) to be untagged, as suggested by @Krinkle in T413865#11498868 (I think this query should return the relavent tasks to be untagged).

(Also gonna cross-reference to T227992: Create #ci-test-error tag for tracking Gerrit repos failing tests, which appears to contain a bit of discussion from when the ci-test-error tag was created.)

Krinkle closed this task as Declined.EditedMon, Feb 16, 4:17 PM
Krinkle added a project: MediaWiki-Engineering.

I've clarified the intro of ci-test-error (WMF-deployed Build Failure), and added a note with an example.

Before

This tag is used to track issues with Jenkins jobs […] failing due to […] a WMF-deployed repository […] not consistently passing its own tests.

After

This tag is used to track issues with Jenkins jobs […] failing and blocking merges due to […] a WMF-deployed repository […] not consistently passing its own tests.

[…]

Note that non-voting jobs or experimental jobs are out of scope. Tasks for pending work to support a new feature or a new PHP version are not Unbreak-Now, and can be tracked under the relevant parent task or workboard about that new thing instead, such as PHP 8.5 support.

There were only two open tasks. I've untagged ci-test-error from those two. We can leave the closed ones as-is, I think.

Thanks @Krinkle :) Do you think we should also clarify the description of the parent tag ci-test-error in a similar way? /genq

Ack. done.

Before

For tasks about any Wikimedia-hosted repo where continuous integration tests fail.

After

For tasks about any Wikimedia-hosted repo where Jenkins/CI jobs fail such that they block merges. (Experimental or non-voting jobs are out of scope.)