Codex currently only requires 80% test coverage for branches, functions, and lines. I was hoping to raise this and add the tests for the uncovered code, but I couldn't seem to find a code coverage report. Can the report be published to https://doc.wikimedia.org/cover/ ? I understand that it is generated automatically by jest
Description
Details
Related Objects
Event Timeline
Change 772492 had a related patch set uploaded (by Catrope; author: Catrope):
[design/codex@main] build: Add "npm run coverage" command
Change 772493 had a related patch set uploaded (by Catrope; author: Catrope):
[integration/config@master] Zuul: [design/codex] Publish code coverage reports
As proposed this only publishes the test coverage report for the main library (@wikimedia/codex), not for the icons or docs packages. In the future we could work on trying to merge the coverage reports into one for publication, but for now the main library report provides most of the value. The icons package contains minimal code, and the docs package is internal (not published or used outside of Codex development).
I said this on one of the patches, but I'll repeat it here for visibility - I think it would be helpful to have reports for all of the packages. Since test coverage is generally not published widely and is just used for Codex development I think its okay to have coverage reports for the docs package.
I agree, I would like to publish coverage for all of our packages. I think that would require the following things:
- Use a tool to merge all the coverage XML files into one (I think istanbul/nyc can do this)
- Merge the LCOV (HTML) reports into one somehow, perhaps by:
- Using some other tool to merge them; or
- Regenerating the LCOV report from the merged XML file; or
- Manually stringing them together with an index.html file that links to each package's LCOV report
- Update the root package.json file so that "npm run coverage" does all of those things and puts the result in the coverage/ directory
what about just putting them in separate directories?
/coverage/codex -> components /coverage/codex-docs -> docs /coverage/codex-icons -> icons /coverage/codex-design-tokens -> design tokens
at least as an initial option
Change 772492 merged by jenkins-bot:
[design/codex@main] build: Add "npm run coverage" command
Yes, I thought about that, and for the web view that would work (if you also add a basic index.html file that links to each of them). That was basically option 2C in my bullet list, I just didn't word it very well.
However, the coverage overview page at https://doc.wikimedia.org/cover/ also appears to expect an XML file that it appears to use to show the coverage total. I don't know what that UI would fall back to if the XML file is missing (right now there are no projects listed that don't have an XML file and an overall coverage percentage). We can try that and see what happens, or we can try to build a combined XML file.
It should gracefully degrade to listing but not having a progress bar (if not, that's a bug :))
Good to know, thanks! I just wasn't sure because there aren't any current examples of it doing this, since every project listed there has an XML file.
With that information, I propose we:
- Wait until https://gerrit.wikimedia.org/r/c/integration/config/+/772493 is merged and we have confirmed the basic setup (publishing coverage for the component library only) works
- Then do what @DannyS712 suggested in T303899#7794447, plus an index.html file so you can click through to each package
- Then try to do a "real" merge, hopefully resulting in a single XML file and a single LCOV tree that's all linked together
Another option could be to have multiple CI jobs that run npm run coverage, npm run coverage-docs, npm run coverage-icons, etc. and then sync each to its own top level directory under doc.wm.o/cover/ (codex, codex-docs, etc.).
Ooh, that's a great idea and would be much easier. It will require some tweaks to the CI config, where both "npm run coverage" and the directory path it pulls from are hard-coded, but I hope I can figure that out. Instead of separate npm run commands, I'd probably want to run "npm test" once, then pull the different coverage dirs from packages/*/coverage and publish each of those to a different URL. oojs-ui has a job that does multiple publishes like that, so I'll look to that for inspiration.
Change 772493 merged by jenkins-bot:
[integration/config@master] Zuul: [design/codex] Publish code coverage reports
Mentioned in SAL (#wikimedia-releng) [2022-03-24T15:00:38Z] <James_F> Zuul: [design/codex] Publish code coverage reports for T303899
@Jdforrester-WMF just attempted to fix this with rCICFf51dc108460f: Zuul: Don't use browser-direct-coverage where browser-coverage will do. However, it doesn't appear that that worked, because https://gerrit.wikimedia.org/r/c/design/codex/+/818175 did not run the coverage job when it merged.
Change 819702 had a related patch set uploaded (by Catrope; author: Jforrester):
[integration/config@master] Zuul: Run publish jobs on branches called 'main' too
OK, so good news / bad news; the custom post-merge coverage job for Codex now runs, but it fails with 'error TS2307: Cannot find module '@wikimedia/codex-icons' or its corresponding type' and then a bunch of other errors. Possibly the coverage job needs a build first?
Change 820180 had a related patch set uploaded (by Catrope; author: Catrope):
[design/codex@main] build: Run build-all-if-missing in "npm coverage"
Change 820180 merged by jenkins-bot:
[design/codex@main] build: Run build-all-if-missing in "npm coverage"
Change 823725 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):
[mediawiki/core@master] Update Codex from v0.1.0-alpha.9 to v0.1.0-alpha.10
Change 823725 merged by jenkins-bot:
[mediawiki/core@master] Update Codex from v0.1.0-alpha.9 to v0.1.0-alpha.10