Page MenuHomePhabricator

Run a phpunit coverage job pre-merge for libraries
Open, Needs TriagePublic

Description

Recently the libraries I've been working on had broken @covers tags, but CI wouldn't complain about it until after the change was merged. It would be good to have a generic phpunit-coverage job that runs as part of test and gate-and-submit, but doesn't publish.

Event Timeline

Legoktm created this task.Sep 30 2016, 7:36 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 30 2016, 7:36 PM
hashar added a subscriber: hashar.Sep 30 2016, 10:23 PM

Can probably be attached to one of the subtasks of T100294: [EPIC] Encourage developers to increase code coverage

Assuming libraries have a fast test suite, it probably make sense to always run with coverage reporting?


Slightly related but not blocking:

A while ago we had a similar discussion with James Forrester T101544: Provide (pre-merge) code coverage reports on patchsets. But we would need some way to store the coverage reports and make them easily browseable (T101545).

Krinkle added a subscriber: Krinkle.Oct 1 2016, 1:35 AM

This task is just about changing phpunit to run with --coverage for smaller repos like libraries. Similar to how we run JSDuck and Doxygen on various repos pre-merge. That way, a problem in it will be discovered and block merge (as it should). Instead of breaking the publish build post-merge (usually invisible to the user).

Can probably use --coverage-clover and make it browseable in Jenkins as well. That way it isn't just a boolean pass/fail to catch typos and syntax errors, but also helps encourage browse the report in Jenkins and verify whether some methods aren't covered for code reviewers without having to verify and re-run locally.

Automated regression detection and publishing of html-reports is lower priority and a separate task indeed.