In order for us to increase the velocity of our release cycle we need an objective and automated way of tracking how "healthy" the app is so that we can have a fixed cost to assess performance regressions. If we don't have this then our cost is variable and will drag down our total velocity unpredictably.

One of the challenges that you face when releasing often is that the amount of attention you keep during small iterative releases. It's easy to keep very focused for 2-3 releases but after that attention can drift to just the major releases. And while it's great to read reviews and find out a subjective metric of how we're doing we need to get in front of it with objective metrics.

Thus having an app health dashboard showcasing: search, readership, editing, etc can easily show you if you've had any regressions. These would not only be useful for small bug fix releases but would also help validate our major product releases. Alarming could be set up on top of it to track regressions.

I'll leave it with you guys to define what metrics are necessary to define a healthy app.