The goal is to do a basic exploration of activity of Automoderator on testwiki till date. This is prep for the first pilot wiki (the queries are intended to be reused).
metrics to be tracked:
| Indicator | Metric(s) | Dimensions | Status |
|---|---|---|---|
| Volume | Number of edits being reverted by Automoderator (absolute & percentage of all reverts) | Anonymous users, newcomers[1], non-newcomers[2] | |
| Accuracy (False positives) | Percentage of Automoderator's reverts reverted back | ||
| Accuracy (False negatives) | Proportion of reverts not performed by Automoderator while it is turned on | - | |
| Efficiency | Average time taken for Automoderator to revert an edit | - | |
| - | Average time taken for Automoderator's reverts to be reverted back | - | |
| Guardrail | Post deployment, proportion of edits reverted by performer | Automoderator, humans, and tool-assisted humans (if applicable) |