Page MenuHomePhabricator
Create Task
Maniphest T383874

Create grafana dashboards using Prometheus
Closed, ResolvedPublic
Actions

Description

In order to finalize the statslib migration new dashboards using Prometheus as the data source need to be created. See steps from T350592: EPIC: migrate in use metrics and dashboards to statslib, most of the code has been migrated and merged, remaining steps are (6), (7) and (8):

  1. Identify the metric (or group of metrics) that will be to be converted.
  2. Create/assign a Phabricator subtask linked to this task (with granularity of individual metric or group of metrics) and update task description to reflect which task(s) have been created for which metric(s).
  3. Follow the migration process as outlined below.
  4. Secure/Conduct code review(s).
  5. Deploy the changes to production via the train (https://wikitech.wikimedia.org/wiki/Deployments/Train).
  6. Verify that the changes have been successfully implemented.
  7. Place the metrics subtask in a 2-3 week waiting period to allow prometheus time to establish 2-3 weeks of metric history
  8. After 2-3 week waiting period is complete, update the dashboard:
    • Save a copy of the dashboard using legacy metrics as-is into the Legacy grafana dashboard folder
    • Replace the old Graphite metric(s) with the new Prometheus metric(s) and save/update the live dashboard

Dashboards and Sections to migrate:

  • Special:Homepage and Suggested Edits
    • Page rendering (server-side)
    • Page rendering (client-side) (except for that one transfer-size panel => T383563)
    • Impact module
    • Loading suggested task (client-side)
    • Image recommendation service
    • Errors
    • Task pool
    • Task suggester
    • UserImpact API
    • [WIP] Time to editor read for suggested edits tasks
  • Growth team product KPI
    • Clicks/Save/Reverts
    • Questions posted to mentors or help desk
    • Mentorship
    • Account registration

Related Objects
Search...

Event Timeline

Sgs created this task.Jan 16 2025, 12:47 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 16 2025, 12:47 PM
Sgs triaged this task as Medium priority.Jan 16 2025, 12:47 PM
Sgs updated the task description. (Show Details)Jan 16 2025, 12:49 PM
Sgs edited projects, added Growth-Team; removed Growth-Team (Maintenance).Jan 16 2025, 5:06 PM
Sgs edited projects, added Growth-Team (Maintenance), GrowthExperiments; removed Growth-Team.
Sgs added subscribers: Cyndymediawiksim, Michael.EditedMar 4 2025, 3:48 PM

Should we create a new dashboard that we all can edit or re-use the existing and tag WIP sections until finalized? I'm leaning towards the second since we would be able to look at both data sources at the same time. Thoughts? cc @Michael @Cyndymediawiksim

Edit: If we re-use the existing one we should do step 8.1 Save a copy of the dashboard using legacy metrics as-is into the Legacy grafana dashboard folder before.

Michael added a comment.Mar 4 2025, 3:50 PM
In T383874#10601322, @Sgs wrote:

Should we create a new dashboard that we all can edit or re-use the existing and tag WIP sections until finalized? I'm leaning towards the second since we would be able to look at both data sources at the same time. Thoughts? cc @Michael @Cyndymediawiksim

Yes, I agree with using the existing dashboard. That makes the progress more visible.

Michael updated the task description. (Show Details)Mar 7 2025, 1:36 PM
Michael updated the task description. (Show Details)Mar 7 2025, 1:48 PM
Michael mentioned this in T268700: Investigate setting up alerts for Growth dashboards.Mar 10 2025, 10:42 AM
Michael mentioned this in T345202: Implement alerting for Growth-consumed or Growth-managed services/pipelines.Mar 10 2025, 11:11 AM
Michael updated the task description. (Show Details)Mar 20 2025, 5:55 PM
Michael updated the task description. (Show Details)Mar 21 2025, 10:46 AM
Michael updated the task description. (Show Details)Mar 21 2025, 12:44 PM
Michael updated the task description. (Show Details)Mar 21 2025, 2:09 PM
Michael closed subtask T389522: Migrate panels in the "Task pool" and "Erros" sections on the Special:Homepage dashboard to use Prometheus/Thanos as datasource as Resolved.Apr 29 2025, 8:20 PM
Michael added a comment.Apr 29 2025, 9:21 PM

On the KPI board, "Questions asked to mentors via mentorship module on Special:Homepage or help panel" still needs migrating.

The panels in the "Mentorship" section still need to be double-checked.

Michael added a subscriber: KStoller-WMF.Apr 29 2025, 9:27 PM

Also, do we still need the two panels for the "Users (not) opted into Growth features" at the very bottom? As far as I can tell, we are providing our features to 100% of our users. Do we intend to change that again in the future? If not, then I'd propose we remove the GEHomepageNewAccountEnablePercentage config option and also this tracking.

@KStoller-WMF for consideration.

This should then probably considered as part of T367724: Reconsider GrowthExperiments feature flags in spirit.

Michael updated the task description. (Show Details)Apr 29 2025, 9:27 PM
Michael updated the task description. (Show Details)
Michael mentioned this in T393020: User Impact cache is almost always stale.Apr 30 2025, 3:32 PM
Sgs added a comment.Apr 30 2025, 3:33 PM
In T383874#10778451, @Michael wrote:

On the KPI board, "Questions asked to mentors via mentorship module on Special:Homepage or help panel" still needs migrating.

I did that one. I'm observing less events in Prometheus than Graphite though.

The panels in the "Mentorship" section still need to be double-checked.

These are looking good, very similar values and trends

KStoller-WMF added a comment.May 1 2025, 4:03 PM

Also, do we still need the two panels for the "Users (not) opted into Growth features" at the very bottom? As far as I can tell, we are providing our features to 100% of our users. Do we intend to change that again in the future? If not, then I'd propose we remove the GEHomepageNewAccountEnablePercentage config option and also this tracking.

Good point—this was a leftover from when we had a control group without Growth features on pilot wikis. We removed that group because it confused experienced editors helping newcomers, and because newcomers clearly did better with Growth features enabled.

We should continue A/B testing new features, but in my view, the control group should never be "no Growth features." I think it's safe to remove those two panels unless we believe they might help us detect bugs—like users not being opted into Growth features—but that seems unlikely.

Sgs updated the task description. (Show Details)May 5 2025, 12:27 PM
Sgs updated the task description. (Show Details)May 5 2025, 3:00 PM
Michael mentioned this in T394435: Remove GEHomepageNewAccountEnablePercentage config option.May 15 2025, 4:28 PM
Michael added a comment.EditedMay 15 2025, 4:35 PM
In T383874#10783706, @KStoller-WMF wrote:

Also, do we still need the two panels for the "Users (not) opted into Growth features" at the very bottom? As far as I can tell, we are providing our features to 100% of our users. Do we intend to change that again in the future? If not, then I'd propose we remove the GEHomepageNewAccountEnablePercentage config option and also this tracking.

Good point—this was a leftover from when we had a control group without Growth features on pilot wikis. We removed that group because it confused experienced editors helping newcomers, and because newcomers clearly did better with Growth features enabled.

We should continue A/B testing new features, but in my view, the control group should never be "no Growth features." I think it's safe to remove those two panels unless we believe they might help us detect bugs—like users not being opted into Growth features—but that seems unlikely.

Thanks for clarifying! I created T394435 Remove GEHomepageNewAccountEnablePercentage config option to remove that.

Michael closed this task as Resolved.Jun 6 2025, 11:00 AM
Michael claimed this task.

T394435: Remove GEHomepageNewAccountEnablePercentage config option has been done.

I'll close this task as resolved. What needs to still be done for the overall statslib migration work:

  • prune the now Graphite tracking in our code: it doesn't do anything anymore
  • (later) remove the (hidden) instructions that read from (now read-only) Graphite in our Grafana boards. That should/could maybe happen after the read-only Graphite has been removed as well.
Restricted Application added a project: User-Michael. · View Herald TranscriptJun 6 2025, 11:00 AM
Michael closed subtask T387876: Grafana dashboard for section "Page rendering" as Resolved.Jul 2 2025, 9:36 AM
Michael closed subtask T387875: Grafana dashboard for section "Impact module" as Resolved.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits