Page MenuHomePhabricator

Define high-level metrics to monitor during the desktop improvements project
Closed, ResolvedPublic

Description

Background

In addition to our project metrics and kpi's, we would like to set up a dashboard which monitors high-level metrics such as account creation and overall pageviews so that we can determine if any of the releases done as a part of the desktop improvements project have significant effects on these metrics
Note: This will need consultation with Product Analytics team and discussions with Web team before finalizing the metrics.

Acceptance criteria

  • Define the project metrics necessary for monitoring
  • Create a dashboard with the metrics defined above

Dashboard v1

General metrics

  • pageviews on pilot wikis
  • account creation on pilot wikis
  • content translation rates

Specific metrics

  • sidebar usage

Event Timeline

ovasileva triaged this task as Medium priority.Jan 15 2020, 4:39 PM
ovasileva created this task.

These are metrics affected INDIRECTLY due to features being added as a part of DIP.
Refer to following documents to think more about this:

Discuss with PA during Sharing meeting.

Brainstormed with the Product Analytics team during sharing meeting and sent the following ideas to Olga for further discussion :

Here are the ideas that we came up with:

Collapsible Sidebar
Monitor traffic to every link (critical and non-critical) on the sidebar and see if it drops
*We can only monitor what people are doing, not what people are not doing
Language switcher moves to article header
Evaluate impact on ‘Neighboring language’ display
Neighboring language = List displayed under ‘Languages’ above the “xx more” link. See here for more info.
If there are greater than 10 neighboring languages how will that be displayed on the article header?
Understand intention of language switching, since now we have an explicit option to do so
Which languages do the users read content in? English vs other languages
Isaac Johnson in the Research Team has done some research on language switching, and would probably be really interested in this kind of data
Sticky site header/logo
Evaluate number of visits to main page
Consolidated user menu
Monitor traffic to each user menu option
Alert for Drop in account creation, Log In etc.
Search moves to site header
Monitor search usage : Alert for Drop
Web team to consult with Search platform team to evaluate potential impacts
Dynamic language switching recommendations
From where does this feature select the language for switching? Is it from user preferences or browser preferences?
Table of contents always accessible
Instrument table of contents to see how many users are actually using the link or scrolling down to each section

will discuss with Olga once she is back.

Hi @ovasileva , I'm moving this task to the next (Upcoming) quarter column of the product-analytics board so that we can continue our work on this in Q4.

LGoto lowered the priority of this task from Medium to Low.May 18 2020, 4:14 PM

Per discussions with @ovasileva, we'll start with a dashboard to monitor pageviews and account creation. Additional metrics can be added to the dashboard during the project as we determine what will be useful to track.

LGoto raised the priority of this task from Low to Medium.Jul 20 2020, 4:41 PM
LGoto edited projects, added Product-Analytics (Kanban); removed Product-Analytics.

@ovasileva

Here is a current version 1 draft of the superset dashboard to monitor metrics for the desktop improvements project.

I've included all of the metrics identified in the task description except for opt-ins. I can add this pending the completion of T261842.

Let me know if you have any questions or suggestions. This can be iterated upon to include additional charts, breakdowns, and other specific metrics as new features are deployed over the course of the project.

@ovasileva

Here is a current version 1 draft of the superset dashboard to monitor metrics for the desktop improvements project.

I've included all of the metrics identified in the task description except for opt-ins. I can add this pending the completion of T261842.

Let me know if you have any questions or suggestions. This can be iterated upon to include additional charts, breakdowns, and other specific metrics as new features are deployed over the course of the project.

Looks great! Thank you @MNeisler!

ovasileva updated the task description. (Show Details)

All done here. Dashboard will be subsequently expanded as we continue deployments and testing