Page MenuHomePhabricator

Special:Investigate usage metrics
Closed, ResolvedPublic

Description

The new version of CheckUser (Special:Investigate) will be deployed to all wikis in the next few weeks. This task is to track all the things we will be measuring.

Metrics

Adoption: Comparing usage for the new tool versus the old tool

  • How often Special:Investigate is accessed?
  • How often Special:Checkuser is accessed?

User experience

  • How many users does an investigation start with?
  • How many records are fetched per investigation?
  • How often did we display incomplete results to the user?
  • How long did a user spend waiting for the results?

Feature usage

  • Over the course of an investigation how often did a user access the individual tabs over the course of the investigation?

Reframe the question as we cannot track the usage over the course of the investigation yet. It depends on the development of a "Done" button.

  • How many times did a user pin a highlight?
  • How often did the block feature get used and how many users were blocked?

User reliance on external IP tools

  • Which tools under the IP address were clicked and how many times?

Other metrics we'd like to track after we have developed a 'Done' button

  • How often did a user quit the investigation (page close or back button) while the results were being generated?
  • How many users are under investigation by the time the investigation ends?
  • We'll probably be getting this data on a weekly basis for all the wikis that has local checkusers. We should get this data for all wikis on a slower cadence.
Schema:

eventlogging data

Related ticket:

https://phabricator.wikimedia.org/T246951#6148694

Report:

Dashboard is refreshed weekly and published at:
https://analytics.wikimedia.org/published/notebooks/AHT/Special_Investigate_Dashboard.html

Event Timeline

Niharika triaged this task as Medium priority.Jul 9 2020, 4:16 AM
Niharika created this task.

@jwang Filed this ticket for checkuser metrics collection. We will expand this after we talk in our next meeting.

closed by accident. reopen.

I am extracting data to answer Question 4 in ticket https://phabricator.wikimedia.org/T246951. After exploring that data, relevantTargetsCount seems related to each query action. Want to check with expert here : how to interpret relevantTargetsCount field and whether it's the field to answer Question 4?

Question 4: How many users are under investigation by the time the investigation ends?

answered by relevantTargetsCount

Data timeframe:
Ran on 2020/10/14. Data timeframe is : 2020-08-26T17:07:01Z ~ 2020-10-14T18:50:56Z

Adoption: Comparing usage for the new tool versus the old tool

  • How often Special:Investigate is accessed?

image.png (1×1 px, 293 KB)

  • How often Special:Checkuser is accessed?

adoption2.png (432×1 px, 13 KB)

User experience

  • How many users does an investigation start with?

users_investigated.png (432×1 px, 19 KB)

  • How many records are fetched per investigation?

results.png (432×1 px, 19 KB)

Top 15 number of records in return
image.png (612×220 px, 18 KB)

  • How often did we display incomplete results to the user?
resultsincomplete_count
0
  • How long did a user spend waiting for the results?

querytime.png (432×1 px, 17 KB)

top 10 waiting time
image.png (418×146 px, 12 KB)

Feature usage

  • Over the course of an investigation how often did a user access the individual tabs over the course of the investigation?

With the data we have, we only can get the overall frequency of accessing the individual tabs. Cannot get the frequency for one individual investigation as we don't have session based indicator in specialinvestigate table.

tab namevalue in schematab clicks
Default landing tab (IPs/User Agents)NaN435
IPs/User Agentscompare752
Account Informationpreliminary-check169
Timelinetimeline206
  • How many times did a user pin a highlight?
pins
88
  • How often did the block feature get used and how many users were blocked?
blocks
6

User reliance on external IP tools

  • Which tools under the IP address were clicked and how many times?
tooltool_usage
whois.toolforge.org52
Special:InvestigateLog22
Special:Contributions14
ipcheck.toolforge.org13
whatismyipaddress.com7
www.robtex.com2

Discussed with @Tchanders today. This question depends on if we have a 'Done' button. Can not measure now. So move it to the wishlist.

Question 4: How many users are under investigation by the time the investigation ends?
answered by relevantTargetsCount

kzimmerman added a subscriber: kzimmerman.

@Niharika once this gets your sign-off we'll close this as resolved

This is great. Thanks!