Page MenuHomePhabricator

[SPIKE] Identify steps for Web Team performance Dashboards
Closed, ResolvedPublic3 Estimated Story Points

Description

Background

Create a simple dashboards on graphana that contains the performance indicators for the web team needs.

We need to try to duplicate existing data sets that the performance team already provides, to have a performance centric dashboard for showing a number of SLOs that will help is identify problems and enhancements.

We need to try to capture as much of the following:

Dashboard #1 Mobile and Desktop Lab

  • Time to First Byte
    • Goal: <= 800ms on average hardware
    • Budget: < Goal on average hardware or the highest time in the last two weeks
  • First Contentful Paint:
    • Goal:: <= 1.8 seconds on average hardware
    • Budget: Goal on average hardware or the highest time in the last two weeks
  • Largest Contentful Paint:
    • Goal: <= 2.5 seconds on average hardware
    • Budget: < Goal on average hardware or the highest time in the last two weeks
  • Total Blocking Time:
    • Goal: < 200ms on average mobile hardware
    • Budget: < Goal on average hardware or the highest time in the last two weeks
  • Cumulative Layout Shift:
    • Goal: <= 0.1
    • Budget: < Goal on average hardware or the highest time in the last two weeks

Dashboard #2 Mobile and Desktop Field (Segmented by country)

  • Time to First Byte
    • Goal: For the 75th percentile: <= 800ms, 800ms - 1800ms, > 1800ms
    • Budget: < Goal on average hardware or the highest time in the last two weeks
  • First Contentful Paint:
    • Goal:: For the 75th percentile: <= 1.8 seconds, 1.8s - 3s, > 3s
    • Budget: Goal on average hardware or the highest time in the last two weeks
  • Largest Contentful Paint:
    • Goal: For the 75th percentile: <= 2.5 seconds, 2.5s - 4s, > 4s
    • Budget: < Goal on average hardware or the highest time in the last two weeks
  • Interaction to Next Paint (don’t currently track, but do collect number of long tasks, total long task time):
    • Goal: For the 75th percentile: 0 - 200ms, 200ms - 500ms, > 500ms
    • Budget: < Goal on average hardware or the highest time in the last two weeks
  • Cumulative Layout Shift:
    • Goal: For the 75th percentile: <= 0.1, 0.1 - 0.25, > 0.25
    • Budget: < Goal on average hardware or the highest time in the last two weeks

For more context and references please check this document https://docs.google.com/document/d/1sqRMjG8NqF7sLZoiNtcHI09oYAyeVSAx7LHSSCbqUL4/edit#heading=h.q4s6y6l2cibx

Spike outcome

  1. Create specification for each dashboard (data source, scope, etc)
  2. Create a ticket for each metric, including specification and instructions on how to set up the dashboard

Event Timeline

Edtadros set the point value for this task to 5.Jun 22 2023, 5:56 PM

@Mabualruz - discussed in standup and we thought this ticket might be easier to tackle if we split it up into individual pieces. Could you take a look and potentially propose some options for how we split? A couple of things we discussed was maybe tackling each new metric in an individual ticket.

ovasileva renamed this task from Web Team performance Dashboards to [SPIKE] Identify steps for Web Team performance Dashboards.Aug 28 2023, 5:37 PM
ovasileva changed the point value for this task from 5 to 3.Aug 28 2023, 5:46 PM
This comment was removed by Mabualruz.
Tasks to be created:
  • Document all current boards we own or access regularly (use chores page and the recording video and transcript of the performance dashboards)
  • Create Web Team Performance Dashboard (one is initially enough, we can have the segmentation per chart for countries or as filters on charts) with First Contentful Paint charts.
  • Add Time to First Byte charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Add Largest Contentful Paint charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Add Total Blocking Time charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Add Cumulative Layout Shift charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Revise the Dashboard if it requires splitting into more specialised and general ones. Update the documentation of the charts and boards of the web team use

We need to fill the specs of each chart inside it's corresponding Task:

  • We can derive the specs from the current performance dashboards as an initial state, then iterate on improving them if needed.

@ovasileva Can you revise if this initial thoughts about breaking down and where to add the speccing is fine for moving forward.

Mabualruz reassigned this task from Mabualruz to ovasileva.
Mabualruz updated Other Assignee, added: Mabualruz.

@ovasileva Can you revise if this initial thoughts about breaking down and where to add the speccing is fine for moving forward.

Looks good to me. Let's proceed.

Specs for data sources will be detailed by the 1st sub task https://phabricator.wikimedia.org/T345990 Document all web monitored grafana boards, as the new board will use same data sources

Mabualruz updated Other Assignee, removed: Mabualruz.

Hey @Mabualruz would this dashboard replace https://grafana.wikimedia.org/d/000000205/reading-web-performance?orgId=1 ? For example that page already contains a Time to First Byte chart.

Tasks to be created:
  • Document all current boards we own or access regularly (use chores page and the recording video and transcript of the performance dashboards)
  • Create Web Team Performance Dashboard (one is initially enough, we can have the segmentation per chart for countries or as filters on charts) with First Contentful Paint charts.
  • Add Time to First Byte charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Add Largest Contentful Paint charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Add Total Blocking Time charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Add Cumulative Layout Shift charts to Web Team Performance Dashboard, update the documentation of the charts and boards of the web team use. Supporting Countries segmentation.
  • Revise the Dashboard if it requires splitting into more specialised and general ones. Update the documentation of the charts and boards of the web team use

We need to fill the specs of each chart inside it's corresponding Task:

  • We can derive the specs from the current performance dashboards as an initial state, then iterate on improving them if needed.

In terms of execution I would recommend doing all of this in a single timeboxed task - it seems like this could be done in 1-2 days. It doesn't make sense to me to have a single task for each of these. Any overflow can be captured in a follow up task. Does that make sense?

Hey @Mabualruz would this dashboard replace https://grafana.wikimedia.org/d/000000205/reading-web-performance?orgId=1 ? For example that page already contains a Time to First Byte chart.

That would be part of the last task to check if will replace the old one, or deprecate some charts there if it is for our use only

In terms of execution I would recommend doing all of this in a single timeboxed task - it seems like this could be done in 1-2 days. It doesn't make sense to me to have a single task for each of these. Any overflow can be captured in a follow up task. Does that make sense?

I think the documentation alone would take 1-2 days, and then each task will need from 1 hour - 1 day depending on the data, knowledge of existing charts, and the assignee experience with grafana,

I've reviewed the follow up tickets and am happy to sign this off. I am not convinced I have a shared understanding of the goals of this work.

I have left some questions and suggestion on the subtasks.

I suggest T345997 in a SHDT discussion would be a good starting point to move this work forward further.