@dbarratt Added a column for event we want to log per your suggestion.

• How can we track when an investigation "ends"?

We discussed this in our engineering meeting - it's difficult for a few reasons. We can't detect a tab close, but even if we could, it wouldn't necessarily indicate that the investigation was over, since the URL for this investigation could be reused until it expired. Since the feature is new and only users with high engagement use Special:Investigate, we concluded that the best option could be to have a temporary "Done" button (with a message to indicate that clicking it helps analytics).

I think we should be able to answer most questions with the schemas below. I'm not sure whether the best practice is to use fewer schemas or multiple, simpler schemas, but here's an approach that uses fewer.

Schemas

If I've understood https://www.mediawiki.org/wiki/Extension:EventLogging/Guide correctly, these will be added to each record without us specifying them in the schema: https://meta.wikimedia.org/wiki/Schema:EventCapsule - this includes timestamp, and the user's IP and UA.

Back-end schema

Front-end schema

Implementation details

The 'submit' event could be captured in SpecialInvestigate::onSubmit; 'query' could be captured in SpecialInvestigate::addTabContent; 'end' when the user clicks an 'end' button, and 'block' when the user makes a block, after we know which blocks were successfully made.

The front-end events could be captured in the click handlers, and since neither navigates away from the page, this should be straightforward.

Questions 1 and 2

Since I don't have analytics access, I'm not sure whether question 1 and 2 can just be answered from the data we already capture. @cwylo @jwang Would you know this?

Also, do we want to know how many people visit these URLs (even if they don't have access); how many people land on these pages who can access them; or how many people actually submit a form on these pages? The proposed schemas can answer the last one for Special:Investigate (specifically targetsCount is nonzero only when the first form is submitted).

Questions 3 to 13

For questions that ask what happened per investigation, we may be able to reconstruct whether records came from the same investigation by using their timestamps and IPs...

Question 3: How many users does an investigation start with?

• answered by targetsCount

Question 4: How many users are under investigation by the time the investigation ends?

• answered by relevantTargetsCount

Question 5: How many records records are fetched per investigation?

• resultsCount tells us how many result rows were fetched for each new page of results

Question 6: How often did we display incomplete results to the user?

• answered by resultsIncomplete

Question 7: How long did a user spend waiting for the results?

• we can time how long the tab takes to load using startTime and the timestamp

Question 8: How often did a user quit the investigation (page close or back button) while the results were being generated?

• unsolved - this is a bit tricky, but perhaps we could check if an unload event fires before the DOMContentLoaded event?

Question 9: Over the course of an investigation how much time did a user spend on Preliminary check versus Compare versus Timeline? OR how often did they access the individual tabs over the course of the investigation?

• answered by tab, startTime and timestamp

Question 10: Over the course of an investigation how many times did a user use filters and which ones?

• answered by excludeTargetsCount
• we'd need something similar for time filters, once they are added

Question 11: How many times did a user pin a highlight?

• answered by event (front-end schema)

Question 12: How often did the block feature get used and how many users were blocked?

• answered by relevantTargetsCount (passing the number of targets successfully blocked)

Question 13: Which tools under the IP address were clicked and how many times?

• answered by event (front-end schema)

Since I don't have analytics access, I'm not sure whether question 1 and 2 can just be answered from the data we already capture. @cwylo @jwang Would you know this?

I don't believe this is already captured. They do capture pageview data for some special pages but not for sensitive ones. I did a quick check on pageviews tool and it seems like Special:CheckUser does not have any data. Special:Version does, for instance.

Also, do we want to know how many people visit these URLs (even if they don't have access); how many people land on these pages who can access them; or how many people actually submit a form on these pages? The proposed schemas can answer the last one for Special:Investigate (specifically targetsCount is nonzero only when the first form is submitted).

The last one is fine. How many times they submit a form on these pages. Can we also track that for Special:CheckUser?

Can we also track that for Special:CheckUser?

We can, although I'd imagine there are likely to be multiple submits per investigation, since the form has to be resubmitted for every type of check (users from this IP, IPs from this user, checking more than one target, etc). On the other hand, the Special:Investigate form is only submitted once at the start of an investigation. (A checkuser could start an investigation again for some reason, but it seems less likely.) That might make these numbers difficult to compare. I'm not sure how to solve this...

Following a discussion about this, we decided to compare usage of Special:Investigate and Special:CheckUser using the cu_log table. We talked about dropping the 'submit' event from the schema, but it seems we need to keep it for tracking the count of targets and excluded targets.