How to trigger CI from differential:

1. Add the meta-ci-jessie tag to your repository
2. ????
3. Profit

But is there a way to decide what tests per repo will do. For example one repo may do npm but the other composer. But maybe some may include npm package.json but dosent want anything tested.

@Paladox: simple! Just include a the requisite files in your repository.

For each of the following files, if found, the corresponding test harness will be called:

@hashar: Why is CI documentation on mediawiki.org instead of wikitech?

In T130952#2393709, @mmodell wrote:

@Paladox: simple! Just include a the requisite files in your repository.

For each of the following files, if found, the corresponding test harness will be called:

language	config file	Command that CI will run
any	.arcunit	arc unit
any	.arclint	arc lint --only-new 1 --never-apply-patches --rev HEAD^
php	composer.json	composer install && composer test test
js	package.json	npm install && npm run test
ruby	Rakefile	bundle exec rake -T
python	tox.ini	tox -v

Oh we specify a file that tell the test what tests we want run.

it automatically runs the appropriate tests, based on which config file is in the repo. You don't have to specify which tests to run, just add a tox.ini to the repo to run tox tests, or add package.json to run npm tests.

Oh, but what happends if we doint want for example the npm test to run since we have package.json then doint include the test script.

In T130952#2393797, @Paladox wrote:

Oh, but what happends if we doint want for example the npm test to run since we have package.json then doint include the test script.

1. Make a unit test script that always succeeds.
2. Don't add the #meta-ci-* tag to the repo and no tests will run.

In T130952#2393716, @mmodell wrote:

@hashar: Why is CI documentation on mediawiki.org instead of wikitech?

We have all the developer related documentation on mediawiki.org and the PHPUnit related documentation got added there. Also WMF tends to document its engineering team work and programs on mediawiki.org. Hence CI is there.

Wikitech is more for the backend administration and infrastructure documentation.

In the end it is a bit of split brain, then https://www.mediawiki.org/wiki/Continuous_integration is mostly for users of the service.

On wikitech we have doc about the various bit and pieces:

• https://wikitech.wikimedia.org/wiki/Jenkins
• https://wikitech.wikimedia.org/wiki/Nodepool

https://www.mediawiki.org/wiki/Continuous_integration/Zuul should probably be moved to wikitech.

@mmodell what about creating more tags that does one job like we do in Zuul. Since running one whole test with multiples being tested can take along time for big repos

In T130952#2398368, @Paladox wrote:

@mmodell what about creating more tags that does one job like we do in Zuul. Since running one whole test with multiples being tested can take along time for big repos

Why would a repo have tests that shouldn't be running?

The best way to get efficient test runs is to create a .arcunit file and test engine. arcanist can optimize the test runs by only re-running tests affected by the patch, rather than the whole test suite.

@mmodell since I know some repo's that are mw extensions and so they need to run the mwext-testextension test with this new tag but they will need to run the non voting test since it will fail.

One can have a look at the old T111181: use one job for all CI entry points which was a random discussion with Jan From rCICFbf1168939ab4: Generic test runner entry point | https://gerrit.wikimedia.org/r/#/c/242177/2/testrunner/makefile,cm

# Generic test runner for Wikimedia repositories
#
# Copyright (c) 2015, Antoine "hashar" Musso
# Copyright (c) 2015, Wikimedia Foundation Inc.

ENTRY_POINTS := commitmessage $(wildcard composer.json package.json tox.ini)

COMPOSER := composer --ansi

.PHONY: test composer-validate $(ENTRY_POINTS)


test: $(ENTRY_POINTS)
	@echo Done

commitmessage:
	@echo '** Validating commit message **'
	@echo '> TODO'

composer.json: composer-validate
	@echo '** Found composer.json. Running composer **'
	$(COMPOSER) test

#composer-validate:
#	$(COMPOSER) validate --no-check-publish
#
#composer-update:
#	$(COMPOSER) update --no-progress --prefer-dist --profile -v
#

package.json:
	@echo '** Found package.json. Running npm **'
	npm test

tox.ini:
	@echo '** Found tox.ini. Running tox **'
	@echo Found tox.ini
	tox -v

Usage would be:

mkdir -p ~/projects/integration
git clone integration/config.git ~/projects/integration/config
make -f ~/projects/integration/config/testrunner/makefile

That is essentially what Mukunda has proposed as an Harbormaster build plan.

From the discussion on https://gerrit.wikimedia.org/r/#/c/242177/ we can invoke make with parallel running of tasks to speed the overall running time. Multiplexing of output makes it non human friendly but there are option to bufferize the output of each target:

I wrote:

There are some related parameters apparently:
https://www.gnu.org/software/make/manual/html_node/Parallel-Output.html
Seems --output-sync would yield output per target.

And others raised concern that a Makefile is not very user friendly. Then for an harbormaster build plan, it is unlikely to be seen by developers/consumers .

@hashar and @mmodell maybe we can create a new file maybe called .ci-entrypoint.yaml and then we can add what test's we want to be run but limitation is we should only support things such as npm and then on the ci side decide what test's we want to support and add them as we go along. That way users can either say true or false for enabling that specific test.

Oh ok.

In T130952#2398492, @mmodell wrote:

That is neat :)

releng-201617-q2 is over. Should this be releng-201617-q3 or not?

It's not a quarterly goal level thing, so no ;)