Page MenuHomePhabricator

Analyze performance of related pages feature
Open, LowPublic

Description

Currently 90% of users see related pages. 10% don't. We want to analyze the performance of related pages feature in these two buckets focusing on:

  • Feature on - does the user have the feature on?
  • CTR
  • Pageviews
  • Session length

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 6 2017, 12:03 PM
ovasileva updated the task description. (Show Details)Feb 6 2017, 7:48 PM
Jdlrobson updated the task description. (Show Details)Feb 6 2017, 7:48 PM
Jdlrobson moved this task from Incoming to Product Owner Backlog on the Readers-Web-Backlog board.
ovasileva updated the task description. (Show Details)Feb 7 2017, 4:52 PM
ovasileva added a subscriber: Tbayer.
ovasileva updated the task description. (Show Details)Feb 7 2017, 4:56 PM

How is this going? Need any help?

ovasileva triaged this task as High priority.Apr 13 2017, 6:49 PM

@ovasileva any progress here...?

Chatted today with @ovasileva I advise a few things:

  • don't assume a user will be bucketed correctly. If a user exhausts localstorage on their browser (very easy in mediawiki) they will be re-bucketed on subsequent page views

Look at unique session ids in each bucket and compare to total amount to get a sense of click through rate in each bucket.
e.g.

  • events in feature-enabled bucket with unique user token / all events in feature-enabled bucket
  • events in feature-disabled bucket with unique user token / all events in feature-disabled bucket

@Jdlrobson Thanks! Just to be extra sure, is this sentence true too: "On each new page load, we decide randomly whether to record events during that pageview (1% probability) or not (99%)" - or is that instead decided once per session (when the new value for userSessionToken is generated)?

The first statement is not true. All sampling is per session.

I.e. If a user is bucketed for logging events and also bucketed with related pages disabled this should stay true until they clear their browser cache.

Jdlrobson closed this task as Resolved.May 31 2017, 5:04 PM

Looks good to me!

ovasileva reopened this task as Open.Aug 25 2017, 2:13 PM

re-opening as continuation of analysis

ovasileva changed the task status from Open to Stalled.Oct 4 2017, 11:33 AM
ovasileva moved this task from 2016-17 Q4 to Product Owner Backlog on the Readers-Web-Backlog board.
Tbayer added a subscriber: phuedx.EditedOct 10 2017, 6:46 PM

Note for the record: If I understand @phuedx correctly (T175918#3663909 ), the underlying data (from Schema:RelatedArticles) is affected by {T175918]}, i.e. all session-based metrics will be invalid.

ovasileva closed this task as Declined.Jun 1 2018, 1:08 PM

Related pages data deprioritized due to more imminent requests.

ovasileva reopened this task as Open.Jun 1 2018, 5:30 PM
ovasileva lowered the priority of this task from High to Low.
Tbayer moved this task from Triage to Backlog on the Product-Analytics board.Jun 21 2018, 8:20 PM
Restricted Application edited projects, added Product-Analytics; removed Product-Analytics (Kanban). · View Herald TranscriptOct 16 2019, 5:48 PM