Page MenuHomePhabricator

Run CI tests daily on master for ungated extensions
Closed, ResolvedPublic8 Estimated Story Points

Description

To earlier detect potential incompatibilities and bugs, we would like to run the CI tests daily for ungated extensions. Ideally that would just run the normal patch jobs on the most recent commits on master/main.

A good first candidate would be Cognate (see T283654 for what triggered this). But potentially also other extensions that are not gated and rarely see active development. Maybe PropertySuggester or ArticlePlaceholder?

List of extensions https://wmde.github.io/wikidata-wikibase-team/repositories/
List of gated things https://github.com/wikimedia/integration-config/blob/526dfb80a2a713cce7694db818b72a4409ef4480/zuul/parameter_functions.py#L793

Acceptance Criteria:

  • The normal CI tests on Cognate are run every day for the most recent commit on the master branch
  • If there is a failure, then an email is sent to some relevant mailing lists
  • Create tasks for each extension for which to enable daily tests

Notes:

  • Maybe we could also have a notification in some Wikidata IRC channel? Mattermost wikidata-feed bot? (potentially out of scope of this task)
NOTE: This should also include rarely edited repositories that are not extensions, like wikidata/query/gui, to let us detect problems like T286058: CI for Wikidata Query Service UI broken (line 10: syntax error near unexpected token `(') early

Event Timeline

Addshore set the point value for this task to 8.

Change 703785 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[integration/config@master] [WIP] jjb: Add wikidata daily job

https://gerrit.wikimedia.org/r/703785

The jjb succeeds with the patch and there are job configs that look sensible but I'm fairly sure the project part wouldn't work, I'm not sure how can I inject the env variable here.

I haven't forgotten about this. It's just that no one has looked at my patch to see if it'll work or not :(

Note: make sure that WikidataPageBanner is one of the covered extensions (see T288884).

Change 703785 merged by jenkins-bot:

[integration/config@master] jjb: Add wikidata daily jobs

https://gerrit.wikimedia.org/r/703785

You can see them in https://integration.wikimedia.org/ci/view/All/ (search for quibble-daily)

Interestingly, some pass and some fail.

It looks like (from far), the dependency extensions are not loaded properly.

Change 715470 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[integration/config@master] Add EXT_DEPENDENCIES to quibble-daily jobs

https://gerrit.wikimedia.org/r/715470

Change 715947 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[integration/config@master] jjb: Comment out broken daily jobs

https://gerrit.wikimedia.org/r/715947

Change 715470 abandoned by Ladsgroup:

[integration/config@master] Add EXT_DEPENDENCIES to quibble-daily jobs

Reason:

This wouldn't fix the issue. We need to find a way to inject parameters from zuul.

https://gerrit.wikimedia.org/r/715470

Change 715947 merged by jenkins-bot:

[integration/config@master] jjb: Comment out broken daily jobs

https://gerrit.wikimedia.org/r/715947

Ladsgroup subscribed.

I don't think I can do much on this without support from RelEng.

Prio Notes:

  • Does not affect Users / Production systems
  • Affects development efforts (if we notice late that something is broken, then it is often a disruptive emergency)
  • Does not affect onboarding efforts
  • Does not affect additional stakeholders (but releng might be needed to get it to work)

Task Review Notes:

  1. It appears that tests for Cognate are already running
  2. We are receiving emails for failures in the Cognate job
  3. We would like to create tickets and enable this one by one in:
    • ArticlePlaceholder [TBC @Michael]
    • PropertySuggester
    • WikibaseQualityConstraints
    • Wikidata.org
    • WikimediaBadges
  4. We can add additional extension as they break

Resolution: We shall resolve this task, and create tickets for each extension individually, to be prioritized separately.

ItamarWMDE claimed this task.