Key metrics to be tracked:
| Indicator | Metric(s) | Dimensions | Status |
|---|---|---|---|
| Volume | Number of edits being reverted by Automoderator (absolute & percentage of all reverts) | Anonymous users, newcomers[1], non-newcomers[2] | |
| Accuracy (False positives) | Percentage of Automoderator's reverts reverted back | ||
| Accuracy (False negatives) | Proportion of reverts not performed by Automoderator while it is turned on | - | |
| Efficiency | Average time taken for Automoderator to revert an edit | - | |
| - | Average time taken for Automoderator's reverts to be reverted back | - | |
| Guardrail | Post deployment, proportion of edits reverted by performer | Automoderator, humans, and tool-assisted humans (if applicable) | |
The format will likely be a notebook until we eventually move to T369488: Develop a unified Automoderator Activity Dashboard (v1)