Page MenuHomePhabricator

[MI-2] Core Metrics Development
Open, In Progress, HighPublic

Description

Mission
Foundation decision-makers understand how our suite of ‘core’ and ‘essential’ metrics are useful to inform their work. The Movement Insights team maintains awareness of needed improvements and develops plans for advancing our suite of metrics.

This epic relates directly to the strategy and maintenance needs of our core annual plan metrics used for understanding our online contributors, relevance, progress against content gaps, and overall effectiveness as an organization. This work also provides essential work prioritization inputs for the needed continuation of efforts output from SDS 1.1 strategic development and documentation of core and essential metrics.

KPI(s)

  • [MI 2.1] Draft a measurement strategy and phased development roadmap for a series of metrics to represent the contributors’ pipeline.
  • [MI 2.2] Editor Activation Metric Pipeline Phased Development Plan
  • [MI 2.3] Deliver phase 1 of a pipeline of contributors-focused metrics for reporting by the end of Q2.
  • [MI 2.4] Curate a prioritized backlog of metric definition improvements for upcoming quarters.

Charter

Event Timeline

JAnstee_WMF changed the task status from Open to In Progress.Jul 2 2024, 6:21 PM
JAnstee_WMF triaged this task as High priority.
JAnstee_WMF added a project: Epic.

Progress update for the week

  1. Solicited review and input from key stakeholders on the logic model for contributors (MI-2.1)
  2. Developed initial options mapping for editor activation metrics pipeline development planning for consultation with key stakeholders and dependencies (MI-2.1)

Priorities for next week

  1. Integrate feedback to logic model and share metric to outcome mapping (MI-2.1)
  2. Solicit input from consulted list regarding assessment of options for development phasing. (MI-2.1)

Progress update for the week

  1. Integrated first-round review and input to logic model and outlined initial strategy for phased development this FY in miro for stakeholder review and input. (MI-2.1)
  2. Onboarded analyst and research stakeholders to the needed next steps and dependencies in the proposed timeline for their critical review. (MI-2.2)
  3. Updated key stakeholders on the potential metric to outcome path connections, proposed phases and timeline, as well as broader stakeholder mapping along with a request for their review and input (MI-2.1)

Priorities for next week

  1. Solicit input from consulted list regarding the assessment of options for development phasing and integrate concerns as alternative options for decision-making in the decision brief. (MI-2.1)
  2. Collaborate within Movement Insights team to clarify expected output specifications for metrics of Phase I development to initiate in August. (MI-2.2)

Progress update for the week

  1. Completed second-round review and input to initial strategy for phased development and integrated feedback. (MI-2.1)
  2. Collaborated with analyst in mapping the needed changes to existing output data tables for editing metrics as well as the related data visualization needs. (MI-2.2)
  3. Prepared the decision-brief for next week's sign-offs (MI-2.1)

Priorities for next week

  1. Route the decision brief for sign-off by agree and decide stakeholders. (MI-2.1)
  2. Collaborate within Movement Insights team to clarify expected output specifications for data visualizations and review rounds. (MI-2.2)

Progress update for the week

  1. Collaborated within Movement Insights team to clarify expected output specifications for data visualizations and tables. (MI-2.2)
  2. Staged miro board for data and initial visualization review rounds. (MI-2.2)
  3. Routed the decision-brief for agree-role sign-offs (MI-2.1)

Up next:

  1. August through September the Movement Insights team will work to implement the first four metrics. (see T369104)

Progress update for the week (MI-2.2)

  • Regrouped on discovery for the standardization of definitions for the first four contributor metrics.
  • Collaborated across RDS on needs and challenges for initial data visualizations.

Up next:

  • Investigate alternative accounts registered metric definition data outputs
  • Explore potential funnel visualization options outside of matplotlib
  • Coordinate next steps for key stakeholder review of initial data visualizations of the first four metrics independently

Progress update for the week (MI-2.2)

  • Connected on remaining decision-making input needs for standardizing the initial metrics including treatment of proxy registrations and needs for understanding registrations to new projects within the Wikmedia universe as well as first-time registrations to any Wikimedia project.
  • Reviewed the current visualization options within matplotlib and finalized plans for initial shareouts to key stakeholders.

Up next:

  • Review alternative accounts registered metric definition data outputs and decide on the method.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.
  • Communicate about the first four metrics and new data columns and visualizations with key stakeholders for basic review and input.

Progress update for the week (MI-2.2)

  • Connected on data explorations.
  • Reviewed the revised visualizations and finalized plans for charts, colors, and callout options for initial shareouts to key stakeholders next week.
  • Initiated documentation for the first four metrics essential metric criteria reviews.

Up next:

  • Review alternative accounts registered metric trend lines and decide on initial approach.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.
  • Communicate about the first four metrics and new data columns and visualizations with key stakeholders for basic review and input.

Progress update for the week (MI-2.2)

    • Finalized initial visualizations for charts, colors, and callout options for initial shareout to key stakeholders.
  • First pass on documentation for the first four metrics essential metric criteria reviews.

Up next:

  • Review alternative accounts registered metric options and inputs for decision making for initial metric viz.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.

Progress update for the week (MI-2.2)

  • Reviewed alternative accounts registered metric options and inputs for decision making for initial metric viz.
  • Reviewed and integrated feedback from key stakeholders to notes
  • Agreed to next steps for initial metric deployment and improvements documentation

Up next:

  • Communications back to key stakeholders and prep for broader communications with the December deployment of the new metrics for November's data.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.

Progress update for the week (MI-2.2)

  • Consolidation of metric improvement recommendations for consideration in prioritization.
  • Final code changes and submission for code review
  • Communications back to key stakeholders

Up Next:

  • Deploy the code for the new metrics within the repo for use in December's analysis and reporting
  • Communicate more broadly about the metrics with December's reporting
  • Continued exploration of alternative funnel visualization options outside of matplotlib.