To understand the scheduling restraints that client-side development and testing places on backend development (e.g. "a potentially risky change must be available for testing for N days before it is deployed to production" or "apps teams want to start writing client code that uses the new API functionality on day X so it must be in production by then), I'd like to better understand the development and testing workflows of the app teams: what backends are you using, and how much time do you expect to take?
These are the backend options I can think of:
- Local test installation. I think it's reasonably easy to set up with vagrant; it also allows patches to be tested before they are merged. OTOH I'm not sure how easy or convenient it is to set up the apps to use a vagrant backend, plus simulating article content in vagrant is a bit of a pain.
- Beta cluster. MediaWiki changes are live there as soon as merged, RESTBase changes need a manual deploy but can be done any time there is a nontrivial change in the service. Still no real content though, and testing time is limited by the MediaWiki / RESTBase deploy cadence (with half a week on average between merging and deploying).
- Just wait until changes are in production before starting to develop for them. That works well for incremental changes, not so well for changes which alter existing behavior and might break something. Also, might slow down development?
- A VPS machine, or vagrant box, configured to use production data for everything but the reading list APIs (basically by proxying all REST API queries other than /data/lists/). This does not exist today but I think it's not too hard (if a bit hacky) to set up.
Did I leave anything out? Which option(s) do apps teams actually use, or would prefer to use?