Non-static data providers are executed with code similar to:
$data = ( new $testClass )->$provider();
From this we can derive the drawbacks:
- The test case constructor is executed unnecessarily, adding to provider execution time. Note that the time taken to run a few test cases with --filter is especially sensitive to data provider execution time, since all data providers need to be executed in order to get the list to filter.
- The object state is immediately thrown away. This can be unexpected to developers, who try to use $this for caching, or to share state between providers, or between providers and test cases.
Reviewing existing non-static data providers, I noticed the following rationales, which I consider to be weak:
- A non-static helper is needed. Perhaps the helper is shared between the provider and the test. But usually this means the data provider is doing too much. Most of the logic and execution time for the test should be in the test itself, not in the provider.
- The mock builder is needed. But MockBuilder takes a TestCase as a required parameter because mocks have assertions attached to them. Mocks built by the provider will be attached to the fake test case, and certain kinds of assertions will not be executed. Also, mock construction is heavyweight, implemented with eval("class ..."), so for performance reasons it should not be done in the provider.
Some providers return objects, not just plain data. This has the following problems:
- Object construction or factories, especially Title or User construction, may access services, which is unsafe to do so early.
- The data provided by tests needs to be formatted in a human-readable way when the test fails.
- Object construction is often slow.
So I think that, with very few exceptions, PHPUnit data providers should be simple and fast static functions that return plain data. The test, not the provider, should be responsible for converting the data to input suitable for the method under test.
PHPUnit 10 decided to deprecate non-static data providers end of 2022 for probably exactly these reasons.