Page MenuHomePhabricator

Setup Git repo and CI for shellbox
Closed, ResolvedPublic

Description

mediawiki/libs/Shellbox needs CI

The service will be run in k8s, so it needs the pipeline jobs:

  • pipeline-test
  • pipeline-test-and-publish

It's questionable if we want to run tests without pipeline as well, then it would need

  • composer-test-package-php72-or-later

Additionally, would be nice to run Phan, however I did not find whether we have a job template for plan not for MediaWiki. Also, I'm not sure whether we should run phan as a part of the service pipeline, or as a separate CI job.

Additional problem is the test-config.json file - as I see it's required for phpunit test execution, it also requires to share a secret with mediawiki. I don't think it's supported in blubber to rename a file when building a test image.

@tstarling would you mind specifying what's needed for Shellbox tests to execute properly?

Event Timeline

Change 628454 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/libs/Shellbox@master] Initial pipeline and CI support

https://gerrit.wikimedia.org/r/628454

Change 628455 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[integration/config@master] Add initial CI config for shellbox

https://gerrit.wikimedia.org/r/628455

Krinkle renamed this task from CI for shellbox to Setup Git repo and CI for shellbox.Sep 18 2020, 11:33 PM
Krinkle added a subscriber: Krinkle.

I've done the repo grooming for for the mirror repo at https://github.com/wikimedia/shellbox (set standard description, add repo tags, turn off various GitHub features).

Change 628467 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/docroot@master] doc: Add Shellbox

https://gerrit.wikimedia.org/r/628467

Change 628455 abandoned by Ppchelko:
[integration/config@master] Add phan to mediawiki/libs/Shellbox

Reason:
Per discussion with Hashar, it's better to run phan as a part of composer test, especially if this will be in service pipeline.

https://gerrit.wikimedia.org/r/628455

Some updates on this after conversations with various people and some more poking:

  • We need the CI to run pipeline-test and pipeline-test-and-publish. This is currently blocked by not having proper images to base the blubber variants on, or perhaps my inability to find the right one.
  • Since the library will be used by both MW and Shellbox, which might not use the same PHP versions, we might want to add composer-test-package-php72-or-later jobs as well.
  • Per discussion with @hashar it's better to run phan from composer - easier to manage for us. For example, when upgrading to different PHP versions, we wouldn't need to touch integration-config.
  • CI has to set up temporary test-config.json before running integration tests with a random HMAC key.

Change 628455 restored by Jforrester:
[integration/config@master] Add phan to mediawiki/libs/Shellbox

https://gerrit.wikimedia.org/r/628455

Change 628455 merged by jenkins-bot:
[integration/config@master] layout: Install CI for mediawiki/libs/Shellbox

https://gerrit.wikimedia.org/r/628455

Mentioned in SAL (#wikimedia-releng) [2020-09-21T21:25:16Z] <James_F> Zuul: Install CI for mediawiki/libs/Shellbox T263295

Change 628454 merged by jenkins-bot:
[mediawiki/libs/Shellbox@master] Initial CI support

https://gerrit.wikimedia.org/r/628454

Partially done, but now needs some fiddles to run a server during the integration test run.

I note both of the post merge builds fail

doxygen-publish fails with error: Doxyfile not found and no input file specified! https://integration.wikimedia.org/ci/job/doxygen-publish/1858/console

And that phpunit-coverage-php72-docker-publish seems to fail with various phpunit test failures https://integration.wikimedia.org/ci/job/phpunit-coverage-php72-docker-publish/172/console but normal CI doesn't fail... Presumably because composer-package-php7[234]-docker run composer test and for shellbox, but in composer.json composer test doesn't run phpunit, while phpunit-coverage-php72-docker-publish does obviously run phpunit to generate coverage reports.

Also, might be worth creating a phab project for shellbox, or is MW-on-K8s sufficient?

Hi, just want to check in and see if anything else is needed from RelEng to support CI/building images for this project?

Some updates on this after conversations with various people and some more poking:

  • We need the CI to run pipeline-test and pipeline-test-and-publish. This is currently blocked by not having proper images to base the blubber variants on, or perhaps my inability to find the right one.

Does shellbox work with php 7.2? If so we have a production-ready base image, but we will still need to fix a couple things in blubber. I'm going to open a task.

  • Since the library will be used by both MW and Shellbox, which might not use the same PHP versions, we might want to add composer-test-package-php72-or-later jobs as well.

We will definitely keep the versions of php running in production in sync between mediawiki and shellbox, unless strictly needed.

Does shellbox work with php 7.2? If so we have a production-ready base image, but we will still need to fix a couple things in blubber. I'm going to open a task.

Yes, my test install is PHP 7.2, and composer.json says >=7.2.

Change 647807 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/libs/Shellbox@master] Add remaining parts of the library bootstrap and stub README

https://gerrit.wikimedia.org/r/647807

Change 647807 merged by jenkins-bot:
[mediawiki/libs/Shellbox@master] Add remaining parts of the library bootstrap and stub README

https://gerrit.wikimedia.org/r/647807

Legoktm claimed this task.
Legoktm added a subscriber: Legoktm.

Everything should be set up now.

Change 656650 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/libs/Shellbox@master] doc: Restore Doxygen welcome page

https://gerrit.wikimedia.org/r/656650

Change 656650 merged by jenkins-bot:
[mediawiki/libs/Shellbox@master] doc: Restore Doxygen welcome page

https://gerrit.wikimedia.org/r/656650

Change 628467 merged by jenkins-bot:
[integration/docroot@master] doc: Add Shellbox

https://gerrit.wikimedia.org/r/628467