Page MenuHomePhabricator
Create Task
Maniphest T234995

Testing for Core REST API
Closed, ResolvedPublic
Actions

Description

We need to decide how to write tests for the new CORE REST API routes.

Options are:

  • Utilize the new integration testing framework right away.
  • Write PHPUnit integration tests for the API.
  • Write unit tests for handlers in phpunit.

I think that we want to go with the integration testing framework right away, the question is how ready is it for real life usage?

Unit tests for handlers would be probably pretty hard, cause the API endpoints are fairly high-level and they utilize DB and a lot of services, so if we wanted to create proper unit tests, we'd need to mock a lot of dependencies. I'm not advocating for not doing unit tests, but maybe in the beginning integration tests with the new framework might be easier.

As for writing integration tests in phpunit - that would be an ok way to have some tests if the framework is not ready, but it would be very unfortunate to write tests this way, cause I imagine long term we would want to switch to the integration testing framework anyway

Related Objects
Search...

StatusSubtypeAssignedTask
 InvalidNoneT229661 Core REST API in MediaWiki
 Resolved• eprodromouT234995 Testing for Core REST API

Event Timeline

• Pchelolo created this task.Oct 8 2019, 7:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 8 2019, 7:39 PM
• Pchelolo updated the task description. (Show Details)Oct 8 2019, 7:44 PM
• Pchelolo added a comment.Oct 8 2019, 8:35 PM

The api-testing has abstractions and some tests set up for Action API, so we would probably need to invest a bit into setting it up for REST API routes as well before we can write actual tests.

One possible concern I have for using it right away is that the REST API seem to not yet be very stable, and moving the tests into a separate repo adds a little bit of overhead when changing the API.. I think we should start off the right track and invest in supporting REST routes in api tester.

daniel added a comment.Oct 8 2019, 8:53 PM
In T234995#5557541, @Pchelolo wrote:

The api-testing has abstractions and some tests set up for Action API, so we would probably need to invest a bit into setting it up for REST API routes as well before we can write actual tests.

One possible concern I have for using it right away is that the REST API seem to not yet be very stable, and moving the tests into a separate repo adds a little bit of overhead when changing the API.. I think we should start off the right track and invest in supporting REST routes in api tester.

Ideally, only self-tests would live in the api-testing repo. All "real" tests would move into the repo of the component under test. This should also happen for the tests we have for the action API.

However, as long as we are still developing the tooling for the tests, it makes sense to have a handful of REST tests in the api-testing repo. I'd start out that way, and then move the tests into the core repo when the dust has settled.

• eprodromou added a comment.Oct 8 2019, 8:58 PM

I'd prefer having unit tests for the handlers in PHP unit ASAP, and building infrastructure for integration tests later.

I agree that that requires double work, but waiting to figure out how to make the integration tests go means we have untested handlers.

• WDoranWMF subscribed.Oct 15 2019, 9:20 PM

For the next sprint we're going to move forward with working on implementing integration tests for the REST API using Daniel and Clara's framework. We'll do this initially for the set of iOS endpoints.

• WDoranWMF mentioned this in T235564: Implement Integration Tests for Existing iOS Endpoints.Oct 15 2019, 9:28 PM
• WDoranWMF edited projects, added Core Platform Team Initiatives (MW REST API in PHP); removed Core Platform Team Initiatives (API Integration Tests).Oct 31 2019, 5:05 PM
• eprodromou closed this task as Resolved.Nov 6 2019, 9:17 PM
• eprodromou claimed this task.

I feel like we ended up with "use the integration testing framework". Let me know if we need to re-open this.

CCicalese_WMF added a comment.Nov 7 2019, 12:36 AM

Were tests written for the endpoints that have been delivered? Will the development of future endpoints include developing tests for them?

• eprodromou added a comment.Nov 7 2019, 8:32 PM
In T234995#5642919, @CCicalese_WMF wrote:

Were tests written for the endpoints that have been delivered? Will the development of future endpoints include developing tests for them?

Yes, and they've been added to Jenkins, so they're part of the CI process.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits