Page MenuHomePhabricator

NewImpact module: Instrumentation
Closed, ResolvedPublic

Description

From the measurement specification

The instrumentation for the new Impact module should be similar to that of the existing Impact module. This means we need to ensure that the following events are captured:

  • Impressions
    • For these events, we will also capture whether the user was shown the activated or unactivated state of the module
    • We will capture metadata about the state of the module as shown to the user:
      • The number of edits the user had made at the time, and the number of edits made in the past 60 days.
      • The number of thanks received.
      • The time since their last edit.
      • The number of days of their best streak.
  • When the user clicks on a link in the module
    • We also capture a symbolic name of the link so that we can identify whether it was a link to an article, to the Pageview Tool, or to the list of user contributions.
    • For a click on an article link we also want to capture the number of pageviews that were displayed for that link.

Event Timeline

I'm wondering if it makes sense to continue logging the main home page module impressions on the server in SpecialHomepageLogger.php. I guess the assumption the module has been seen when it loads was fine when these were 100% server side rendered. Is that assumption still fine since we now build the html in the client?

Another thing that's confusing me is the logger name, SpecialHomepage sounds like it should only log impressions on Special:Homepage but I also see log calls when loading Special:Impact. Is that expected?

Change 859108 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] User impact: add impact data to instrumentation events

https://gerrit.wikimedia.org/r/859108

Change 859113 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[schemas/event/secondary@master] Add user new impact data to the impact homepagemodule

https://gerrit.wikimedia.org/r/859113

Sgs changed the task status from Open to In Progress.Nov 23 2022, 5:47 PM
Sgs moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.

Change 859113 merged by jenkins-bot:

[schemas/event/secondary@master] Add user new impact data to the impact homepagemodule

https://gerrit.wikimedia.org/r/859113

Change 859108 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] User impact: add impact data to instrumentation events

https://gerrit.wikimedia.org/r/859108

(1) Recording impression info is as per the specs:

Impressions

$schema: "/analytics/legacy/homepagemodule/1.6.4"
client_dt: "2023-01-05T03:46:32.935Z"
event:
action: "impression"
action_data: "timeframe_in_days=60;timeframe_edits_count=3;thanks_count=0;last_edit_timestamp=1637707164;longest_streak_days_count=2;top_articles_views_count=21"
homepage_pageview_token: "ob2k4sptlb3o3rr27i6hi58h2mr0t52s"
is_mobile: false
mode: "desktop"
module: "impact"
state: "activated"
user_editcount: 8
user_id: 44486
user_variant: "control"

(2) Looks according to the specs:

When the user clicks on a link in the module
We also capture a symbolic name of the link so that we can identify whether it was a link to an article, to the Pageview Tool, or to the list of user contributions.

"event":
{
"action": "link-click",
"action_data": "no_cached_user_impact=true;linkId=impact-total-edits",
"event":
{
"action": "link-click",
"action_data": "timeframe_in_days=60;timeframe_edits_count=2;thanks_count=0;last_edit_timestamp=1668040822;longest_streak_days_count=2;top_articles_views_count=0;linkId=impact-thanks-log",

(3) @Sgs - can you review the following? the total number of pageviews (not the pageviews per article) is recorded:

For a click on an article link we also want to capture the number of pageviews that were displayed for that link.

{"event":{"action":"link-click","action_data":"timeframe_in_days=60; timeframe_edits_count=538; thanks_count=44;last_edit_timestamp=1671485037; longest_streak_days_count=9; top_articles_views_count=1252; linkId=impact-article-title","user_id":6768,"user_editcount":1274,"user_variant":"control","module":"impact"

If a pageviews link is clicked, then the article's pageviews number is recorded as : linkData=1250

{"event":{"action":"link-click","action_data":"timeframe_in_days=60;timeframe_edits_count=538;thanks_count=44;last_edit_timestamp=1671485037;longest_streak_days_count=9;top_articles_views_count=1252;linkId=impact-pageviews;linkData=1250","user_id":6768,"user_editcount":1274,"user_variant":"control","module":"impact","is_mobile":false,"mode":"desktop",

Change 879140 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] User impact: add missing data to instrumentation event

https://gerrit.wikimedia.org/r/879140

Change 879140 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] User impact: add missing data to instrumentation event

https://gerrit.wikimedia.org/r/879140

(3) @Sgs - can you review the following? the total number of pageviews (not the pageviews per article) is recorded:

I pushed a patch to add the missing pageviews per article to the event. Can be tested in beta now. Thanks for the catch!