As part of T115395 and T64424, we will be updating our mock API so that Parsoid parserTests testing is more reflective of how we actually use the M/W API in production.
However, we have to ensure that our mock API responses closely mimics mediawiki API responses and keeps up with those changes. Towards this end, a non-voting Jenkins test on our patches might be one way to flag any deviations and alert us to fixes needed on our end.
Change 338036 had a related patch set uploaded (by Arlolra):
[WIP] T115461: Record MW Api responses as the mock Api
Change 338036 abandoned by Arlolra:
[WIP] T115461: Record MW Api responses as the mock Api
Reason:
For now
One way might be to extend the PHP parserTests.php entry point so that it does all the setup phases (defining stock media & metadata, reading the parser test file and defining all the articles and templates, setting default config options) and then enters a simple server mode to accept action.php API requests. We could then run Parsoid's parserTests against this "actual" action.php (communicating out-of-band about any $wgXXX configuration changes requested for each test) and save the responses. Jenkins would also run in this way and compare the saved responses vs the responses we've checked in to the code base, to ensure that this update is done appropriately when needed.
The npm package mediawiki-express might be useful here (might!).