Page MenuHomePhabricator

[SPIKE] Determine whether new instrumentation is needed for sticky header A/B test
Closed, ResolvedPublic

Description

In T294506, we will be running an A/B test to evaluate the impact introducing an edit affordance within desktop reading experience's fixed "sticky" site header has.

This task represents the work with determining what – if any – additional instrumentation will be required to report on the metrics T294506 defines.

Open questions

  • What – if any – new instrumentation needs to be implemented in order to report on the Metric(s) for evaluation defined within T294506's ===Hypotheses section?

Done

  • Answers to all ===Open questions are documented

Event Timeline

@ppelberg - based on our conversation on Dec 10, I believe there are two parts to this:
T298518: [SPIKE] Tracking clicks to edit button within the sticky header which looked at instrumenting the sticky header within the edit button
T298634: Track edit button clicks in sticky header in the EditAttemptStep schema which will ensure we can track those clicks

I'm going to go ahead and close this as a duplicate but feel free to re-open if there's anything missing

MNeisler triaged this task as Medium priority.

I've completed the instrumentation review and spec for tracking the metrics identified for evaluating the editing affordance being added to the Sticky header, as identified in T294506.

OPEN QUESTION(S):
How do we want to track edit button clicks in the sticky header? Two possible sources of tracking clicks:

EditAttemptStep:

  • Will clicks to the sticky header editing affordance be logged as event.init_type = 'page' in EditattemptStep? Would it be worthwhile to add a new init_type in EditAttemptStep to decipher clicks to the sticky header editing affordance vs full-page editing within EditAttemptStep?

DesktopWebUIActionsTracking:

  • The web team confirmed that they are able to track clicks within the Sticky header as part of the DesktopWebUIActions. Per T298518, it would be straightforward to add clicks to the edit button in the stick header but this would still need to be implemented. We would also need to confirm that we can join this data to EditAttemptStep using the session id. Task to look into this: T298634

16 March Meeting Outcomes
Below are the decisions @MNeisler and I came to during today's meeting...

  • DECIDED: for the analysis we have planned to evaluate the impact of the editing affordance within the sticky header, we will add a new event to EditAttemptStep`; that event will be named and implemented in T298634.
  • OPEN: Web Team to decide whether they think it would be worthwhile to add events to monitor interactions with the sticky header editing affordances to DesktopWebUIActions.
    • @ovasileva: I'm assigning this task over to you to decide on the above.

@ppelberg - I've set up T304072: Track edit button clicks in sticky header in DesktopUIClickTracking schema to track adding the event to the DesktopWebUIActions schema

Great. With the T298634 and T304072 scoped, this ticket can be resolved.