Based on user feedback from Google Play, we have identified several areas where users seem dissatisfied with the new look and feel - for example saying that they find it difficult to access content they are interested in, or find things like the Table of Contents unpleasant to interact with. We need to understand whether these reviews are simply negative reactions to novelty, or whether we can perceive a quantitative difference in user engagement.
We are looking for improvement/worsening in engagement metrics such as:
- Daily average users (being able to access more content they want to read)
- Average time spent in the app
- Interactions with articles (eg. saved articles in lists, sharing)
- Deeper user sessions (articles read per session, regardless of bouncing between feed or article rabbit-holing) with more switching between article reading and exploratory actions
- If possible, usage/interaction with Table of Contents
- Usage of other parts of the app (feed, nearby, reading lists, etc)
Ideally, we would like to compare per session data from users pre- and post-change.