To improve observability of temporary accounts rollout and ongoing functionality, let's create a Grafana dashboard that will have panels for the following:
- [ ] Edit rate
- **What:** Plot edit rate for anonymous IP editors, regular accounts and temp accounts
- **Why:**
- Make it easy to see if there is a spike/drop in temp account editing
- Make it easy to see if there is a spike/drop in temp account editing relative to regular accounts
- [ ] Edit abandonment rate
- **What:** Plot edit abandonment rate for anonymous IP editors, temp accounts, regular accounts
- **Why:**
- Make it easy to see if there is a spike/drop in temp account editing
- Make it easy to see if there is a spike/drop in temp account editing relative to regular accounts
- [ ] Rate of blocks
- **What:** Plot number of blocks that target regular accounts and temp accounts
- **Why:**
- Improved visibility into anti-abuse measures being taken against temporary accounts
- Make it easy to see if there is a spike/drop in blocks of temp accounts
- Make it easy to see if there is a spike/drop in blocks of temp accounts editing relative to regular accounts
- [ ] Rate limit trips
- **What:** Plot number of rate limit trips related to temp account creation (`$wgRateLimits['edit']['ip']` at time of writing)
- **Why:** Better visibility into how often users are being prevented from temp account creation, per {T342880}
- [ ] Rate of reverts & rollbacks
- **What**: Plot number of edits made by IP editors, temp accounts that were reverted
- **Why**:
- Improved visibility into anti-abuse measures being taken against temporary accounts
- Helps us understand if the wiki is facing a spike in vandalism possibly because of the introduction of temp accounts
- [ ] Rate of page protections
- **What**: Plot number of pages that are protected over time
- **Why**:
- Improved visibility into anti-abuse measures being taken against temporary accounts
- Helps us understand if the wiki is facing a spike in vandalism possibly because of the introduction of temp accounts
- [ ] Rate of page deletions
- **What**: Plot the number of pages that are deleted over time
- **Why**:
- Improved visibility into anti-abuse measures being taken against temporary accounts
- Helps us understand if the wiki is facing a spike in vandalism possibly because of the introduction of temp accounts
- [ ] Number of local CU checks run per-wiki
- **What**: Plot number of local CU checks performed on wikis
- **Why**:
- Monitoring this metric over time lets us understand if there is an uptick in demands made of CUs on the project
- An increase in this number post temporary accounts deployment can indicate increased burden on CUs. This metric can help us understand the extent of the situation.
- [ ] Rate of temporary account creation
- **What**: Plot number of temporary accounts being created
- **Why**:
- Mostly a curiosity metric. A steady number indicates temporary accounts are able to successfully edit.
- [ ] Number of IP Reveal users
- **What**: Plot the number of users have enabled the permission to reveal IPs including the number of users who are auto-enrolled into viewing IPs
- **Why**: Capturing this metric can help us get an estimate of how many patrollers are needing to see IP addresses to patrol effectively. We had some projections for this early on but it would be good to get actual numbers to understand this further. Would be even better to get a % of how many users qualify to opt-in versus how many choose to do so.
- [ ] Active IP Reveal users
- **What**: Plot the number of users who actively reveal IP addresses
- **Why**: A step further from the above metric: Users who have opted-in to view IPs and actually use the feature frequently. This can tell us how many editors feel the need to use the feature in practice over time.
- [ ] Number of IPs Revealed as a percentage of total IPs that edited
- **What**: Plot number of IP addresses revealed over total number of IPs that edited over a given period of time
- **Why**:
- This can be a useful impact metric. It tells us how many users actually had their information exposed in practice. If this is a quite small number, we know that fewer people's PII is being looked at and that's a win. If this is a very large number, it may point to issues and we may need to investigate other tools we could build to assist in the patrolling process.
- [ ] Rate of account creation
- **What**: Plot the number of accounts created over time
- **Why**:
- This metric helps us understand how the introduction of temporary accounts affects (or does not affect) the creation of full accounts