Recently the libraries I've been working on had broken @covers tags, but CI wouldn't complain about it until after the change was merged. It would be good to have a generic phpunit-coverage job that runs as part of test and gate-and-submit, but doesn't publish.
Description
Description
Related Objects
Related Objects
- Mentioned In
- T267781: Run proactive coverage checking on more repos
- Mentioned Here
- T100294: [EPIC] Encourage developers to increase code coverage
T101544: Provide (pre-merge) code coverage reports on patchsets
T101545: Provide infrastructure to store files by project/branch post-merge to compare with pre-merge
Event Timeline
Comment Actions
Can probably be attached to one of the subtasks of T100294: [EPIC] Encourage developers to increase code coverage
Assuming libraries have a fast test suite, it probably make sense to always run with coverage reporting?
Slightly related but not blocking:
A while ago we had a similar discussion with James Forrester T101544: Provide (pre-merge) code coverage reports on patchsets. But we would need some way to store the coverage reports and make them easily browseable (T101545).
Comment Actions
This task is just about changing phpunit to run with --coverage for smaller repos like libraries. Similar to how we run JSDuck and Doxygen on various repos pre-merge. That way, a problem in it will be discovered and block merge (as it should). Instead of breaking the publish build post-merge (usually invisible to the user).
Can probably use --coverage-clover and make it browseable in Jenkins as well. That way it isn't just a boolean pass/fail to catch typos and syntax errors, but also helps encourage browse the report in Jenkins and verify whether some methods aren't covered for code reviewers without having to verify and re-run locally.
Automated regression detection and publishing of html-reports is lower priority and a separate task indeed.
Comment Actions
It's been a while since this task was filed. :-)
Nowadays we do coverage validation pre-merge as a standard step (using ockcyp/covers-validator), which is the initial request of this task.
In one repo, EquivSet, we use rregeer/phpunit-coverage-check to force coverage limits (config) – do we want to make this a new standard practice in our libraries?
I'd rather come up with a standard for in-repo management rather than a centralised system, as that maps better to the GitLab CI future we're going to have RSN.
Comment Actions
Nowadays we do coverage validation pre-merge as a standard step (using ockcyp/covers-validator), which is the initial request of this task.
Ack. Is there another objective/outcome remaining here besides the "pre-merge indication of invalid covers tag". It seems like we've effectively achieved that indeed.