Page MenuHomePhabricator

Enable integration testing in CI
Closed, DeclinedPublic

Description

The MediaModeration extension will be doing requests to external providers in order to assess whether the media contains child abuse.

The extension is covered well with unit tests, however, it would be interesting to explore if we could enable an integration test running in CI hitting the real cloud provider. Currently a test like this exist but it skipped.

Questions for Release-Engineering-Team :

  1. There are pros and cons of doing an integration test with hitting real external service, do we have a policy or already made decision regarding this?
  2. Is it even possible in our CI infrastructure?
  3. In case it is possible and desirable - we would need to make an api key available in CI via some secret management way - is that feasible?

Related Objects

StatusSubtypeAssignedTask
DeclinedNone
DeclinedNone
DeclinedNone

Event Timeline

hashar subscribed.

Sorry that task has not been noticed.

My rule of thumb is that tests running on CI should avoid relying on a third party, cause if the external resource is unreachable that means the test suite will fail and would block any change.

To exercise the logic of your code you should mock what is expected from the third party. The mock would define the interface defined by the external service. Then to ensure that contract is still honored by the third party, you would need a monitoring probe that runs on schedule and ensure that they respect the contract, but I don't have a good solution for that. Maybe an Icinga probe?

In case it is possible and desirable - we would need to make an api key available in CI via some secret management way - is that feasible?

The API key would end up being exposed publicly. We do that for example to authenticate requests to the GitHub API to drop the rate limit when using composer/npm installs. They key is exposed via an environment variable,, which is rather trivial to retrieve by sending a patch dumping the environment variable. So potentially that aims at using an Icinga probe which would not expose the key.

Ok, thank you @hashar This is mostly what I thought, just wanted to confirm.