In addition to tracking the detailed analytics needed for evaluating the A/B test (T405177), we also want engineering focused performance metrics to be able to assert that our feature works as expected on a technical level.

This explicitly includes tracking the runtime needed to figure out for which paragraph to show the Revise Tone Check. (See the internal slack conversation for context.)

• time to get data from Data Gateway
• time to get find correct paragraph in article

• number of errors
  • suggestion no longer available (no data in data gateway)
  • paragraph not found

• number of users declining with which reason in the survey

• in addition to the above: add to the KPI dashboard

Notes:

• visualization of these metrics on Grafana is not part of this task