Page MenuHomePhabricator

[MI-2] Core Metrics Development
Open, In Progress, HighPublic

Description

Mission
Foundation decision-makers understand how our suite of ‘core’ and ‘essential’ metrics are useful to inform their work. The Movement Insights team maintains awareness of needed improvements and develops plans for advancing our suite of metrics.

This epic relates directly to the strategy and maintenance needs of our core annual plan metrics used for understanding our online contributors, relevance, progress against content gaps, and overall effectiveness as an organization. This work also provides essential work prioritization inputs for the needed continuation of efforts output from SDS 1.1 strategic development and documentation of core and essential metrics.

FY 2025/26 KPI(s)

  • [MI 2.1a] Draft theory of change and measurement model for a series of metrics to represent the responsible use of infrastructure strategy and metric trees.
  • [MI 2.1b] Draft theory of change and measurement model for a series of metrics to represent the consumers' strategy and metric trees.
  • [MI 2.2a] Align on a multi-year measurement strategy for Responsible Use of Infrastructure (WE 5) in partnership with implementation teams
  • [MI 2.2b] Align on a multi-year measurement strategy for Consumers' strategy (WE 3) in partnership with implementation teams
  • [MI 2.3] Deliver next phase of development roadmap for contributors-focused metrics for reporting.
  • [MI 2.4] Curate a prioritized backlog of metric definition improvements for upcoming quarters.

Charter

FY 2024/25 KPI(s)

  • [MI 2.1a] Draft a measurement strategy and phased development roadmap for a series of metrics to represent the contributors’ pipeline.
  • [MI 2.2a] Editor Activation Metric Pipeline Phased Development Plan
  • [MI 2.3a] Deliver phase 1 of a pipeline of contributors-focused metrics for reporting by the end of Q2.
  • [MI 2.4a] Curate a prioritized backlog of metric definition improvements for upcoming quarters.
  • [MI 2.1b] Align on a multi-year contributor's focused measurement strategy WE 1.5 in partnership with contributors' product teams (Sonja P.)
  • [MI 2.2b] Outline the next phases of the Editor Activation Pipeline Phased Metric Development Plan
  • [MI 2.3b] Deliver phase 2 of a pipeline of contributors-focused metrics for reporting by the end of Q4.
  • [MI 2.4b] Curate a prioritized backlog of metric definition improvements for upcoming quarters.

Charter

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
JAnstee_WMF changed the task status from Open to In Progress.Jul 2 2024, 6:21 PM
JAnstee_WMF triaged this task as High priority.
JAnstee_WMF added a project: Epic.

Progress update for the week

  1. Solicited review and input from key stakeholders on the logic model for contributors (MI-2.1)
  2. Developed initial options mapping for editor activation metrics pipeline development planning for consultation with key stakeholders and dependencies (MI-2.1)

Priorities for next week

  1. Integrate feedback to logic model and share metric to outcome mapping (MI-2.1)
  2. Solicit input from consulted list regarding assessment of options for development phasing. (MI-2.1)

Progress update for the week

  1. Integrated first-round review and input to logic model and outlined initial strategy for phased development this FY in miro for stakeholder review and input. (MI-2.1)
  2. Onboarded analyst and research stakeholders to the needed next steps and dependencies in the proposed timeline for their critical review. (MI-2.2)
  3. Updated key stakeholders on the potential metric to outcome path connections, proposed phases and timeline, as well as broader stakeholder mapping along with a request for their review and input (MI-2.1)

Priorities for next week

  1. Solicit input from consulted list regarding the assessment of options for development phasing and integrate concerns as alternative options for decision-making in the decision brief. (MI-2.1)
  2. Collaborate within Movement Insights team to clarify expected output specifications for metrics of Phase I development to initiate in August. (MI-2.2)

Progress update for the week

  1. Completed second-round review and input to initial strategy for phased development and integrated feedback. (MI-2.1)
  2. Collaborated with analyst in mapping the needed changes to existing output data tables for editing metrics as well as the related data visualization needs. (MI-2.2)
  3. Prepared the decision-brief for next week's sign-offs (MI-2.1)

Priorities for next week

  1. Route the decision brief for sign-off by agree and decide stakeholders. (MI-2.1)
  2. Collaborate within Movement Insights team to clarify expected output specifications for data visualizations and review rounds. (MI-2.2)

Progress update for the week

  1. Collaborated within Movement Insights team to clarify expected output specifications for data visualizations and tables. (MI-2.2)
  2. Staged miro board for data and initial visualization review rounds. (MI-2.2)
  3. Routed the decision-brief for agree-role sign-offs (MI-2.1)

Up next:

  1. August through September the Movement Insights team will work to implement the first four metrics. (see T369104)

Progress update for the week (MI-2.2)

  • Regrouped on discovery for the standardization of definitions for the first four contributor metrics.
  • Collaborated across RDS on needs and challenges for initial data visualizations.

Up next:

  • Investigate alternative accounts registered metric definition data outputs
  • Explore potential funnel visualization options outside of matplotlib
  • Coordinate next steps for key stakeholder review of initial data visualizations of the first four metrics independently

Progress update for the week (MI-2.2)

  • Connected on remaining decision-making input needs for standardizing the initial metrics including treatment of proxy registrations and needs for understanding registrations to new projects within the Wikmedia universe as well as first-time registrations to any Wikimedia project.
  • Reviewed the current visualization options within matplotlib and finalized plans for initial shareouts to key stakeholders.

Up next:

  • Review alternative accounts registered metric definition data outputs and decide on the method.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.
  • Communicate about the first four metrics and new data columns and visualizations with key stakeholders for basic review and input.

Progress update for the week (MI-2.2)

  • Connected on data explorations.
  • Reviewed the revised visualizations and finalized plans for charts, colors, and callout options for initial shareouts to key stakeholders next week.
  • Initiated documentation for the first four metrics essential metric criteria reviews.

Up next:

  • Review alternative accounts registered metric trend lines and decide on initial approach.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.
  • Communicate about the first four metrics and new data columns and visualizations with key stakeholders for basic review and input.

Progress update for the week (MI-2.2)

    • Finalized initial visualizations for charts, colors, and callout options for initial shareout to key stakeholders.
  • First pass on documentation for the first four metrics essential metric criteria reviews.

Up next:

  • Review alternative accounts registered metric options and inputs for decision making for initial metric viz.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.

Progress update for the week (MI-2.2)

  • Reviewed alternative accounts registered metric options and inputs for decision making for initial metric viz.
  • Reviewed and integrated feedback from key stakeholders to notes
  • Agreed to next steps for initial metric deployment and improvements documentation

Up next:

  • Communications back to key stakeholders and prep for broader communications with the December deployment of the new metrics for November's data.
  • Continued exploration of alternative funnel visualization options outside of matplotlib.

Progress update for the week (MI-2.2)

  • Consolidation of metric improvement recommendations for consideration in prioritization.
  • Final code changes and submission for code review
  • Communications back to key stakeholders

Up Next:

  • Deploy the code for the new metrics within the repo for use in December's analysis and reporting
  • Communicate more broadly about the metrics with December's reporting
  • Continued exploration of alternative funnel visualization options outside of matplotlib.

Progress update for the week (MI-2.2)

  • Completed the essential metric criteria reviews for the first four metrics to be merged into monthly and quarterly reporting in January.
  • Due to some delay in code review we pushed the deployment of the new metrics and reporting to January for December's monthly data and our next quarterly reporting.

Up Next:

  • Deploy the code for the new metrics within the repo for use in January's analysis and reporting
  • Communicate more broadly about the metrics expanding in early January.
  • Create summary charts for the essential metric reviews and include along with appropriate GitLab links following January deployment.
  • Complete priority lens classifications of developed backlog of metric improvement needs and requests
  • Continued exploration of alternative funnel visualization options outside of matplotlib.

Progress update for the week (MI-2.2)

  • Created summary charts for the essential metric reviews.
  • Completed code review and revisions for deployment of new metrics.
  • Drafted communications about the update to the metrics
  • Drafted communications for initiating a small working group to align core and essential contributor metrics to product strategy through a shared measurement model.

Up Next:

  • Communicate more broadly about the metrics expanding in early January.
  • Deploy the code for the new metrics within the repo for use in January's analysis and reporting
  • Communicate out to working group
  • Connect with research team on phase 2 priority dependencies

Progress update for the week (MI-2)

  • Completed revision code review deployed code for new metrics to gitlab repo.
  • Shared communications about the updates to the metrics
  • Coordinated on drafting a new hypothesis under WE1.5 to coordinate a shared measurement model to align core and essential contributor metrics to product contributor strategy.

Up Next:

  • Communicate out to working group
  • Connect with research team on phase 2 priority dependencies

Progress update for the week (MI - 2)

  • Developed initial artifacts for the process including a working group project overview, a miro board for capturing our process, and an outline of the measurement strategy brief we will be working to complete over the course of the 8-week working group. (WE 1.5.2) (see also asana reporting
  • Messaged the working group.
  • Reached out to Research team about needed support in the implementation design of returning newcomers and active experiences editor metrics as part of Phase 2 developments.
  • Began logging known metric documentation pages in a gsheets inventory

Up Next

  • Scheduling the synchronous sessions.
  • Developing protocols in the miro board for the kickoff.
  • Advancing on review of existing thresholds for "experienced" editors examined and currently in use.
  • Reaching out on key channels to learn of other documentation spaces for essential metrics

Progress update for the week (MI - 2)

  • [MI 2.1b] Scheduled the working group's first two sync sessions for developing a strategy-specific action logic model and measurement strategy options for vetting and strategized protocols for inclusion in the miro board for the kick-off next week.
  • [MI 2.3b] Shared a crowd-sourcing request to data slack channels for identifying active approaches to identifying experienced active editors and coordinated for Research support to the phase 2 metrics development priorities.
  • [MI 2.5] Reached out on key channels to learn of other documentation spaces for essential metrics.

Up Next

  • [MI 2.1b] Working group kickoff
  • [MI 2.3b] Deeper dive into phase 2 development priorities
  • [MI 2.5] Deeper dive into the identified documentation spaces.

Progress update for the week (MI - 2)

  • [MI 2.1b] Facilitated the working group's first session to develop a strategy-specific action logic model and measurement strategy options for vetting. Strategized protocols for inclusion in the miro board for the second sync session next week to develop measurement strategies for consideration.
  • [MI 2.3b] Reviewed inputs to the inventory of experienced active editor definitions and initiated a similar inventory for retained newcomers metric definitions for the phase 2 metrics development priorities.
  • [MI 2.5] Reviewed inputs to the inventory of documentation spaces for essential metrics.

Up Next

  • [MI 2.1b] Working group session 2
  • [MI 2.3b] Deeper dive into phase 2 development priorities
  • [MI 2.5] Deeper dive into the identified documentation spaces.

Progress update for the week (MI - 2)

  • [MI 2.1b] Facilitated the working group's second session to map potential indicators for a measurement strategy aligned with the action logic model developed last week.
    • Outlined outcome indicators along change routes to measure ten potential key impact aims in terms of our online:
      • contributors:
        • Increased online contributor retention
        • Diversified humans contributing
        • Increased active editors
      • contribution patterns:
        • Increased constructive edits
        • Increased online contributing
        • Decreased workload
        • Increased edit and contribution quality
        • Increased number of overall contributions via collaboration
        • Reduced mobile editing gap
      • data landscape for guiding contributors
        • Increased data for expanding data-informed support
  • [MI 2.3b] Reviewed inputs and began outreach to those owners identified in the inventories.
  • [MI 2.5] Began deeper dive into the identified documentation spaces.

Next steps

  • [MI 2.1b] Working group asynchronous input to:
    • identify each of the relevant audiences and methods for the indicators identified, work to identify the overlaps and unique components across measurement models,
    • select and interrogate alternative measurement models
  • [MI 2.1b] Scheduling the remaining synchronous sessions.
  • [MI 2.3b] Continue investigation into examples of metric approaches for phase 2 development priorities
  • [MI 2.5] Continued discovery of documentation spaces.

Progress update for the week (MI - 2)

  • [MI 2.1b] Led the working group's asynchronous next steps to map potential indicators for identifying measurement strategy options for more careful vetting.
    • We have been charting, transferring, and striving to simplify the synthesis of the shared and distinct elements identified for potential measurement across the impact aims. Currently, a new spreadsheet page titled “Impact Indicator Mapping”) is available via our miro board. In the same workbook, there are also some pivot tables (“Indicators Pivots”) that help you to see the roughly 62 potential indicators we have identified and their connections to different impact aims and strategic threads.
  • [MI 2.1b] Began drafting the assumptions section of the strategy brief.
  • [MI 2.3b] Integrated feedback and added to the reading list for discovery.
  • [MI 2.5] Stalled this week

Next Steps

  • [MI 2.1b] We are working to gather the details about the products and tools associated with each of the potential indicators to map alternative measurement models. From there we will work to get very specific and technical about the methods and potential research or design gaps to input to the working group's recommendation.
  • [MI 2.3b] Integrated feedback and added to the reading list for discovery.
  • [MI 2.5] Reach out to technical documentation staff for input to discovery and request input to strategy

Progress update for the week (MI - 2)

  • [MI 2.1b] Led the working group's asynchronous next steps to map potential measurement strategy options for more careful vetting. Following activities to tally and synthesize the thematic connections across logic model pathways and potential outcome indicators, we are interrogating and circulating with the broader group pf product managers, a straw dog; a synthesis of the strategy and measurement concepts into a high-level metric tree focused on changes we might expect to see within the first 1-3 years of the strategy's successful implementation.
  • [MI 2.1b] Synced 1:1 with working group members to walk each through the updates to the board and process to arrive at the initial straw dog and review and input next steps.
  • [MI 2.1b] Presented the working group's process, progress, and straw dog metric tree with Product Managers at their bi-weekly Contributor's Strategy meeting.
  • [MI 2.3b] Minor progress in document review this week.
  • [MI 2.5] Stalled this week

Next Steps

  • [MI 2.1b] This week we are expanding the circle of review and input for interrogating the initial straw dog metric trees and relationships to the multi-year strategy aims and workplans. We are working to get very specific and technical about the methods and potential research or design gaps related to the indicators identified in optional measurement models to input to the working group's recommendation which we will shift focus to at this week's working group meeting.
  • [MI 2.3b] Map thresholds and known challenges to identified examples.
  • [MI 2.5] Reach out to technical documentation staff for input to discovery and request input to strategy

Progress update for the week (MI - 2)

  • [MI 2.1b] Led the WE1.5.2 working group's synchronous next steps to integrate vetting information about the model indicators elevated for consideration so we can begin mapping the metric tree components and their specifications in an evaluation plan. We will continue asynchronously in this effort as we work to ensure the logic of each metric tree identified in the model and identify the specific needs for data collection and reporting to identify challenges and potential solutions in terms of alternative measurement models.
  • [MI 2.3b] Completed integrating threshold information into the inventory this week along with notes on challenges and shared back with collaborators.
  • [MI 2.4b] Added local vs global sysops partition request to the backlog for core metrics
  • [MI 2.5] Still on pause / Stalled this week

Next Steps

  • [MI 2.1b] We are working asynchronously the next two weeks to get very specific and technical about the methods and potential research or design gaps related to the indicators identified. As we work through challenges and potential solutions we will outline optional measurement models to input to the working group's final recommendation.
  • [MI 2.3b] Collaborator review of alternative threshold approaches and challenges for consideration of needed next steps for phase II metrics development

Progress update for the past 2 weeks (MI-2)

  • [MI 2.1b] Led the working group's asynchronous next steps to integrate vetting information about the model indicators and mapping the metric tree components and their specifications in an evaluation plan ensuring the logic of each metric tree identified. Here we are working to identify specific needs for data collection and reporting as well as challenges and potential solutions in terms of alternative measurement models.
  • [MI 2.1b] Led the working group's asynchronous next steps to continue the vetting and decision-making between alternative indicators and methodologies to outline a revised measurement model.
  • [MI 2.1b] Completed the mapping of measurement method options for each of the 32 potential key indicators identified for measuring change along three key thematic change theories: onboarding, habit-forming, and enablement. (see Evaluation plan sheet)
  • [MI 2.1b] Worked to document the known elements of the strategy recommendation process and outputs into the strategy brief for working group review and input as we work toward our penultimate recommendation to share with relevant Product Managers.

Next Steps

  • [MI 2.1b] We continue asynchronously the next week on the pros and cons of the current proposed measurement model and articulate the research or design gaps related to the indicators identified.
  • [MI 2.1b] We work to complete the strategy brief document for broader internal consumption

Progress update for the past 2 weeks (MI-2)

  • [MI 2.1b] Continued to document and review the known elements of the strategy recommendation process and outputs into the strategy brief for working group review and input as we work to share our draft recommendation with relevant Product Managers.
  • [MI 2.3b] Regrouped with the working group on challenges and inventory examples to review for upcoming synchronous discussion.

Next Steps

  • [MI 2.1b] We regroup with key product managers and directors to share progress and the latest model under the working groups consideration.
  • [MI 2.1b] We continue asynchronously the next week on the pros and cons of the current proposed measurement model and articulate the research or design gaps related to the indicators identified in the background documentation.
  • [MI 2.1b] We work to complete the strategy brief document for broader internal consumption
  • [MI 2.3b] Synchronous discussion on next steps where we will select which approaches to test run.

Progress update for the past 2 weeks (MI-2)

  • [MI 2.1b] Regrouped with agree and decide stakeholders as well as Product Managers of the contributors strategy.
  • [MI 2.1b] Identified a need to revise the model further, separating out retained editors who are not moderating, mentoring, or collaborating.
  • [MI 2.1b] Identified a potential single metric and model revisions to align this metric across the onboard, habit, and enablement measurement pathways identified.
  • [MI 2.1b] Together these integrations led to the development of Option D, now outlined in the miro and strategy brief.
  • [MI 2.1b] Placed a calendar hold for the final working group meeting on Apr 15
  • [MI 2.3b] Regrouped with the working group and set next steps for extracting data for experienced active editors and exploring session capture options for exploration of the retained newcomer approaches.

Next Steps

  • [MI 2.1b] We share updates with the working group to continue vetting the latest model and evaluation plan asynchronously.
  • [MI 2.1b] We work to reach consensus to arrive at a recommendation and work to complete the strategy brief document for sign-off and broader internal consumption.
  • [MI 2.3b] Working group will meet again next week to review data and set additional next steps.

Progress update for the week (MI-2)

  • [MI 2.1b] Continued asynchronous to update with working group members on latest model adjustments and new metric identified.
  • [MI 2.1b] Prepared for upcoming synchronous meeting for final decision-making snd sign-off on the recommendation.
  • [MI 2.3b] Regrouped with working group on initial data pulled and editors distributions around potential experienced active editors thresholds

Next Steps

  • [MI 2.1b] Reach consensus on recommendation in final synchronous meeting and work to complete the strategy brief document for sign-off and broader internal consumption.
  • [MI 2.3b] Regroup with working group on second data pull and editors distributions around potential experienced active editors thresholds as well as potential for sessions approaches to retained newcomers.

Progress update for the week (MI-2)

  • [MI 2.1b] Held last synchronous working group session to confirm the latest model and finalize recommendations around specific measurement approaches and/or research and development gaps.
  • [MI 2.1b] Followed-up async to resolve remaining measurement choices and wrked to draft the executive summary of the working groups final recommendations for the overall measurement model and specified measurement approaches for each stage of the theory of change.
  • [MI 2.3b] Regrouped with working group on second data pull to review distributions of editor time and frequency of crossing the 100 edit threshold. Identified next steps for advancing on the active experienced editor metric.

Next Steps

  • [MI 2.1b] The working group is in review of the executive summary and specific recommendations captured for final sign off. We are also working to simplify the language of the overview section of the Measurement Strategy brief and to close out the hypothesis.
  • [MI 2.3b] We will review the distribution based on two alternative time window for having passing the suggested 100 edit threshold as well as a second, higher cut point to explored their utility for the active experienced editor metric.
  • [MI 2.5] Advance: Consolidate and and deprecate metric definition pages out from "stalled" status

Progress update for the week (MI-2)

  • [MI 2.1b] Confirmed working group sign-off of executive summary recommendations.
  • [MI 2.1b] Drafted WE 1.5.2 Hypothesis close-out report.
  • [MI 2.1b] Drafted new hypothesis for Q4 implementation efforts.
  • [MI 2.1b] Routed overview for support in reducing lexical complexity before broader sharing.
  • [MI 2.3b] Regrouped with working group on third data pull to review distributions of editor time and frequency of crossing the various edit thresholds in the past 1 year.
  • [MI 2.3b] Identified next steps for advancing on the active experienced editor metric and exploring sessions approaches to the retained newcomers metric.
  • [MI 2.5] Re-engaged with the initial discovery and documentation spaces shared.

Next Steps

  • [MI 2.2b] Finalize and set-up new hypothesis
  • [MI 2.3b] We will review the distribution based on alternative data pulls looking for alignment with the observed sample to confirm the potential utility of a two- vs. three-tier threshold approach for the active experienced editors metric.
  • [MI 2.5] Addition of new core metrics added in Q3 to the out-dated, but most recent, documentation space. Continued review of documentation spaces shared for next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Closed out Hypothesis WE 1.5.2 as supported. Initiated new hypothesis, WE 1.5.4: If we develop an editor retention core metric configuration and an implementation plan for other contributor-focused indicator metrics, we will enable the evaluation of the multi-year strategy for contributors beginning in FY 25/26.
  • [MI 2.2b] Revised the penultimate draft of the Measurement Strategy Brief to bring down the language complexity for broader sharing.
  • [MI 2.2b] Kicked off Editor Retention Measure - Feasibility Investigation
  • [MI 2.2b] Initiated mapping of internal stakeholders needed for input to the implementation plan.
  • [MI 2.3b] Regrouped with working group on fourth data pulls to review distributions of editor time and frequency of crossing key thresholds as cohorts to confirm alignment with monthly distribution thresholds.
  • [MI 2.3b] Identified next steps for advancing on the active experienced editor metric and further exploring sessions approaches to the retained newcomers metric.
  • [MI 2.5] No significant progress this week.

Next steps

  • [MI 2.2b] Carry out the investigation into Editor Retention configurations.
  • [MI 2.2b] Clarify data engineering and analysis burden for metric development needs so that we can collaboratively design a feasible metrics development timeline with clear responsibilities across teams.
  • [MI 2.2b] Identify steps for communications development for the broader internal and community sharing about the measurement strategy and metric development plans.
  • [MI 2.3b] We will finalize our recommendation and approach for the active experienced editors metric and review comparison data for retained newcomers examining the sensitivity and utility of session-based vs edit-based thresholds.
  • [MI 2.5] Addition of new core metrics added in Q3 to documentation space. Continued review of documentation spaces shared for next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Outlined May-June wokplan and identified initial collaborators
  • [MI 2.2b] Connected on Editor Retention Measure - Feasibility Investigation - For discussion to input to the selection of the retention metric type (i.e., repeated action vs group membership)
  • [MI 2.3b] Regrouped with working group on fifth data pulls to confirm alignment with monthly distribution thresholds for experienced active editors and explore sessions approaches to the retained newcomers metric.

Next steps

  • [MI 2.2b] Continued investigation into Editor Retention configurations.
  • [MI 2.2b] Continued identification of targets and collaborators
  • [MI 2.2b] Prepare documentation of theory of change and measurement model improvement recommendations
  • [MI 2.3b] Finalize our recommendation and approach for the active experienced editors metric and review comparison data for retained newcomers examining the sensitivity and utility of session-based vs edit-based thresholds.
  • [MI 2.5] Addition of new core metrics added in Q3 to documentation space. Continued review of documentation spaces shared for next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Shared workplan
  • [MI 2.2b] Met with designer for support to documentation of theory of change and measurement model for broader sharing
  • [MI 2.2b] Connected on Editor Retention Measure - Feasibility Investigation - Decided to use a count version of the metric and not the retention rate which can be calculated separately and used in early experimentation use cases. Identified three straw dog metrics for potential experimentation
  • [MI 2.3b] Regrouped with working group on latest data pulls to explore sessions approaches to the Retained Newcomers metric. Decided to recommend advancing an initial Experienced Active Editors metric using thresholds at 100 and 300 edits within the last year to indicate progression from "Junior" 1-99 edits, to "Experienced" 100-299, and "Very Experienced". The working group recommends further monitoring and investigation into a potential upper threshold around the 85-90th percentile for "Very Very Experienced" editors.
  • [MI 2.5] Addition of new core metrics to documentation space.

Next steps

  • [MI 2.2b] Continued investigation into Editor Retention configurations.
  • [MI 2.2b] Connect with key collaborators to clarify estimated burden/lift and responsibilities
  • [MI 2.2b] Prepare documentation of theory of change and measurement model improvement recommendations
  • [MI 2.3b] Document our final recommendation and approach for the active experienced editors metric and review comparison data for retained newcomers examining the sensitivity and utility of different session-based indicators.
  • [MI 2.5] Continued review of documentation spaces shared for next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Continued investigation into Editor Retention configurations.
  • [MI 2.2b] Began connecting with key collaborators to clarify estimated burden/lift and responsibilities
  • [MI 2.3b] Reviewed comparison data for retained newcomers examining the sensitivity and utility of different session-based indicators.
  • [MI 2.5] Updated most current metrics page with the Q3 metrics added. Continued review of documentation spaces shared for next steps.

Next steps

  • [MI 2.2b] Continued investigation into Editor Retention configurations.
  • [MI 2.2b] Connect with key collaborators to clarify estimated burden/lift and responsibilities
  • [MI 2.2b] Prepare documentation of theory of change and measurement model improvement recommendations
  • [MI 2.3b] Document our final recommendation and approach for the active experienced editors metric and review final comparison data for retained newcomers to conclude regarding sensitivity and utility of different session-based indicators and make recommendations for next steps in development.
  • [MI 2.5] Continued review of documentation spaces shared for next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Continued investigation into Editor Retention configurations.
  • [MI 2.2b] Scheduled messaging to key dependencies identified to clarify estimated burden and opportunities for collaboration.
  • [MI 2.3b] Reviewed comparison data for retained newcomers examining the sensitivity and utility of session-based vs edit indicators.
  • [MI 2.5] Reached out to technical documentation specialist

Next Steps

  • [MI 2.2b] Continued investigation into Editor Retention configurations.
  • [MI 2.2b] Connect with key dependencies to clarify estimated burden/lift and responsibilities
  • [MI 2.2b] Follow-up on theory of change and measurement model design support
  • [MI 2.3b] Document our final recommendation and approach for the active experienced editors metric and review final comparison data for retained newcomers to conclude regarding sensitivity and utility of different session-based indicators and make recommendations for next steps in development.
  • [MI 2.5] Continued review for next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Continued investigation into Editor Retention configurations.
  • [MI 2.2b] Engaged with messaging among key dependencies to clarify estimated burden and opportunities for developing improved survey sampling and indicators needing further experimentation and final decisions on selection.
  • [MI 2.3b] Finalized recommendation and approach for the active experienced editors metric.
  • [MI 2.3b] Reviewed comparison data for retained newcomers examining the sensitivity and utility of session-based vs edit indicators due to the unknown data gaps in the edit attempts tables we are leaning most to the single edit configuration with either a first or second fortnight follow-up window.
  • [MI 2.5] Received additional documentation for review from technical documentation team for review and potential alignment.

Next Steps

  • [MI 2.2b] Final inquiries and adjustments to the Editor Retention configurations.
  • [MI 2.2b] Conclude about dependencies estimated burden/lift and responsibilities for documentation and timeline prioritization
  • [MI 2.2b] Follow-up on theory of change and measurement model design support
  • [MI 2.3b] Review final comparison data for retained newcomers to conclude regarding sensitivity and utility of different follow-up windows and make recommendations for next steps in development.
  • [MI 2.5] Review of shared examples and scheduling connect for developing next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Editor Retention recommendation drafted
  • [MI 2.2b] Set next steps for onboarding additional key dependencies and meeting targets for developing timeline for improved survey sampling.
  • [MI 2.3b] We are down to just the follow-up window recommendation for Newcomer Retention which will either be 1st or 2nd fortnight - it will be based on repeated action of a single edit, and count edits to any namespace, unlike active editors. This will be mapped specifically to constructively activated newcomers in their first weeks of activation rather than a splice of the second-month returning new editors.
  • [MI 2.5] Stalled this week

Next Steps

  • [MI 2.2b] Final decision making on the Editor Retention metric configuration.
  • [MI 2.2b] Conclude about dependencies estimated burden/lift and responsibilities for documentation and timeline prioritization
  • [MI 2.2b] Follow-up on theory of change and measurement model design support
  • [MI 2.3b] Review final comparison data for retained newcomers to conclude regarding sensitivity and utility of different follow-up windows and make recommendations for next steps in development.
  • [MI 2.5] Review of shared examples and scheduling connect for developing next steps.

Progress update for the week (MI-2)

  • [MI 2.2b] Posted hypothesis closeout report for WE 1.5.4 as supported pending final review and sign-offs next week.
  • [MI 2.2b] Scheduled synchronous meeting for final review and sign-off of the Editor Retention metric configuration.
  • [MI 2.2b] Concluded about dependencies, estimated burden/lift and responsibilities for documentation and timeline prioritization for continued development phases of the contributors' metric area.
  • [MI 2.2b] Followed-up on theory of change and measurement model design iteration.
  • [MI 2.3b] Reviewed final comparison data for retained newcomers to conclude regarding sensitivity and utility of different follow-up windows and make recommendations for next steps in development. Began updating and testing code with current data to set code for review and deployment next month.

Next Steps

  • [MI 2.1] Draft initial communications (WE 5.4.4)
  • [MI 2.2] Next steps pending establishment of working group for WE 5.4.4 / MI 2.1
  • [MI 2.3] Scheduling of synchronous meeting for key operators of, and adjacent to, the Contributors' strategy, for onboarding to the measurement strategy from it’s underlying theory of change to the metric gaps and development priorities for enabling timely research and evaluation throughout the course of the strategy.
  • [MI 2.3] Scheduling of synchronous meeting of key engineering and product managers for advancing our needed survey infrastructure alongside instrumentation efforts.
  • [MI 2.4] Complete Esssential metric reviews of Phase II metrics and integrate gaps to the development backlog.
  • [MI 2.5] Review of shared examples and scheduling connect for developing next steps.

Progress update for the week (MI-2)

  • [MI 2.1] Initiated documentation review and project mapping (WE 5.4.4)
  • [MI 2.3] Scheduled synchronous meeting for key operators of, and adjacent to, the Contributors' strategy, for onboarding to the measurement strategy from it’s underlying theory of change to the metric gaps and development priorities for enabling timely research and evaluation throughout the course of the strategy.
  • [MI 2.4] Initiated esssential metric review queue for Phase II metrics
  • [MI 2.5] Stalled.

Next Steps

  • [MI 2.1] Draft initial communications and schedule with objective owner (WE 5.4.4)
  • [MI 2.2] Next steps pending establishment of working group for WE 5.4.4 / MI 2.1
  • [MI 2.3] Scheduling of synchronous meeting of key engineering and product managers for advancing our needed survey infrastructure alongside instrumentation efforts.
  • [MI 2.4] Complete Esssential metric reviews of Phase II metrics and integrate gaps to the development backlog.
  • [MI 2.5] Review of shared examples and scheduling connect for developing next steps.

Progress update for the week (MI-2)

  • [MI 2.1] Completed documentation review and initiated project mapping
  • [MI 2.3] Scheduled synchronous meeting of key engineering and product managers for advancing our needed survey infrastructure alongside instrumentation efforts.
  • [MI 2.4] Initiated review for "Experienced active editors

Next steps

  • [MI 2.1] Continued project mapping and communications planning - Connect with objective owner
  • [MI 2.3] Slide deck narration for onboarding session
  • [MI 2.4] Continued review for essential metric criteria documentation and additions to the development backlog.

Progress update for the week (MI-2)
[MI 2.1] Completed project mapping, connected with objective owner, scheduled working group syncs for August
[MI 2.3] Connected with decision-makers on remaining steps for sign-off.

Next steps
[MI 2.1] Stage logic model inputs for first sync session and start stakeholder mapping
[MI 2.3] Slide deck narration for onboarding session
[MI 2.4] Continued review of new core metrics for essential metric criteria documentation and additions to the development backlog.

Progress update for the week (MI-2)
[MI 2.1] Completed first working group sync
[MI 2.3] Completed slide deck narration for onboarding session & connected on topline indicators as remaining steps for sign-off. Identified additional core metric for development in Phase III, Retained Moderators, after the completion of Monthly Active Moderators.

Next steps
[MI 2.1] Second sync session and potential indicator mapping
[MI 2.3] Onboarding session for contributors' measurement strategy
[MI 2.4] Continued review of new core metrics for essential metric criteria documentation and additions to the development backlog.

Progress update for the week (MI-2)

  • [MI 2.1] Regrouped on process and reset async expectations for completing the logic model and metric mapping for the 4 KR areas.
  • [MI 2.3] Presented slide deck for onboarding more broadly to our aligned approach for the measurement of the Contributors' strategy.
  • [MI 2.3] Prepped comms and documentation for upcoming release of three new movement health metrics including the new core annual plan metric for the Contributors' strategy.
  • [MI 2.3] Connected with Engineering and Research leads to develop next steps for addressing survey infrastructure gaps of quicksurveys.
  • [MI 2.4] Completed essential metric criteria documentation for remaining new retention metrics being released.

Next steps

  • [MI 2.1] Follow-up on async efforts for outcomes and indicator mapping.
  • [MI 2.3] Share more broadly the high-level from the onboarding session and calls to action.
  • [MI 2.3] Communicate out about the latest metrics release.
  • [MI 2.4] Additions to the development backlog for gaps in essential metric criteria for new metrics released.

Progress update for the week (MI-2)

  • [MI 2.1] Working group is focused on async expectations for completing the logic model and metric mapping for the 4 KR areas:
  • [MI 2.1] Followed up about async assignment - we are now at 75% completion
  • [MI 2.3] Communicated to key channels about latest metrics being deployed in our monthly reporting.
  • [MI 2.3] Amplified onboarding session and materials to key slack channels for broader sharing.
  • [MI 2.3] Scheduled follow-up for key RDS folks to meet and discussion potential approaches and timeline to the prioritized FY25/26 queue for contributors' strategy relevant research.
  • [MI 2.3] Continued connecting with Engineering in navigation of next steps for QuickSurveys development needs.
  • [MI 2.4] Additions to the development backlog for gaps in essential metric criteria for new metrics released.

Next Steps

  • [MI 2.1] Follow-up on async efforts for outcomes and indicator mapping.
  • [MI 2.3] Connecting on research queue next steps, preparing for potential RDS share

Progress update for the week (MI-2)

  • [MI 2.1] Working to integrate and model the metric trees identified and reaching out async to follow-up on questions on shared theories of change and indicators for mapping.
  • [MI 2.3] Continued connecting with Engineering in navigation of next steps for QuickSurveys development needs.

Next Steps

  • [MI 2.1] Follow-up on questions about async efforts for outcomes and indicator mapping.
  • [MI 2.1] Complete final pillar mapping and transfer back to the board for synthesis and modeling.
  • [MI 2.1] Sync with objective owner on progress and inputs to next steps for the working group's sync Sep 17th.
  • [MI 2.3] Connecting on research queue next steps, preparing for potential RDS share

Progress update for the week (MI-2)

  • [MI 2.1] Working to integrate and model the metric trees identified and reaching out async to follow-up on questions on shared theories of change and indicators for mapping.
  • [MI 2.3] Continued connecting with Engineering in navigation of next steps for QuickSurveys development needs.

Next Steps

  • [MI 2.1] Follow-up on questions about async efforts for outcomes and indicator mapping.
  • [MI 2.1] Complete final pillar mapping and transfer back to the board for synthesis and modeling.
  • [MI 2.1] Sync with objective owner on progress and inputs to next steps for the working group's sync Sep 17th.
  • [MI 2.3] Connecting on research queue next steps, preparing for potential RDS share

Progress update for the week (MI-2)

  • [MI 2.1] Synced back with objective owner to share progress and align on next steps.
  • [MI 2.1] Reaching out async to follow-up on questions on shared theories of change and indicator readiness for use.
  • [MI 2.2] Shared initial modelling of the metric trees reported.
  • [MI 2.2] Working to integrate 5.4 measures and model the indicators identified and identify potential intersections and divergences.
  • [MI 2.3] Input to kickoff of reporting and operationalization planning for product reporting
  • [MI 2.3] Continued connecting with Engineering in navigation of next steps for QuickSurveys development needs.

Next Steps

  • [MI 2.1] Synthesis of the identified theories of change
  • [MI 2.2] Working group meets synchronous to discuss and confirm the logic and measurement model(s) and to identify key core and indicators metrics and potential research gaps related to the needed measures.

Progress update for the week (MI-2)

  • [MI 2.1a] Began synthesis of the identified theories of change to reduce excess words
  • [MI 2.1b] Reached out to connect with Objective owner for the Consumers' strategy and scheduled initial sync to kick-off theory of change and measurement mapping in October.
  • [MI 2.2a] Working group met synchronously to discuss and confirm the logic and measurement model(s) and to identify key core and indicators metrics and potential research gaps related to the needed measures. We have identified two potential core metrics for the objective at this time, (Re)Directed Users and Traffic Quality (i.e., Confidence in knowing who are our users and user agents are).
  • [MI 2.2a] Reached to to get async input from collaborators unable to participate in sync session.
  • [MI 2.3] Added links to data query to metric documentation page on meta, followed-up on research queue prioritization in terms of Contributors' strategy insights needs.

Next Steps

  • [MI 2.1b] Connect with WE 3 Objective owner
  • [MI 2.2a] Work to develop alternative measurement models based on the conversations and inputs from this last week.
  • [MI 2.2a] Follow-up with WE 5 Objective owner.

Progress update for the week (MI-2)

  • [MI 2.1b] Connected with WE 3 Objective owner to coordinate kicking off consumer measurement strategy work in October.
  • [MI 2.2a] Working to develop alternative measurement models and to clarify topline metrics based on the conversations and inputs.
  • [MI 2.2a] Followed-up with WE 5 Objective owner on initial remapping of the straw dog measurement model

Next Steps

  • [MI 2.1b] Set-up hypothesis in asana for WE 3 measurement strategy for Q2.
  • [MI 2.1b] Communicate to WE 3 measurement strategy collaborators for Q2 kickoff.
  • [MI 2.2a] Continue to develop measurement models based on the conversations and follow-up inputs.
  • [MI 2.2a] Set time to connect on Enterprise interconnections.

Progress update for the week (MI-2)

  • [MI 2.1b] Set-up hypothesis in asana for WE 3 measurement strategy for Q2 and communicated out to WE 3 measurement strategy collaborators for Q2 kickoff.
  • [MI 2.2a] Continued to develop measurement models based on the conversations and follow-up inputs.
  • [MI 2.2a] Set time to connect on Enterprise and WE5.

Next Steps

  • [MI 2.1b] Deep dive into WE 3 strategy & measurement.
  • [MI 2.1b] Follow along and synthesize WE 3 measurement strategy collaborator inputs.
  • [MI 2.2a] Continue to develop measurement models based on the conversations and follow-up inputs.
  • [MI 2.2a] Connect on Enterprise strategic alignments.

Progress update for the week (MI-2)

  • [MI 2.1b] Continued documentation review, Sent reminders to KR owners about pre-work, Scheduled first sync meeting
  • [MI 2.2a] Set another an alternative final sync session with better attendance potential
  • [MI 2.2a] Finished pushing metrics and notes to the evaluation plan spreadsheet
  • [MI 2.2a] Connected on Enterprise strategic alignments.

Next Steps

  • [MI 2.1b] Deep dive into WE 3 strategy & measurement.
  • [MI 2.1b] Follow along and synthesize WE 3 measurement strategy collaborator inputs.
  • [MI 2.2a] Continue to develop measurement models based on the conversations and follow-up inputs.
  • [MI 2.2a] Follow-up to fill gaps in the evaluation plan on definitions and readiness estimates

Progress update for the week (MI-2)

  • [MI 2.1b] Continued documentation review, reset first sync meetings
  • [MI 2.2a] Identified and began filling gaps in the evaluation plan spreadsheet
  • [MI 2.2a] Connected on Enterprise strategic alignments.
  • [MI 2.3] Outlined story points and contributors for the journey of the Contributors' measurement strategy development.

Next Steps

  • [MI 2.1b] Continued deep dive into WE 3 strategy & measurement.
  • [MI 2.2a] Connect with Objective owner on latest measurement models
  • [MI 2.2a] Follow-up to fill gaps in the evaluation plan on definitions and readiness estimates
  • [MI 2.3] Diagram and flesh out story points for presenting the journey of the Contributors' measurement strategy development. Share to collaborators.

Progress update for the week (MI-2)

  • [MI 2.1b] Continued documentation review.
  • [MI 2.2a] Connected with Objective owner on the latest measurement models and on remaining gaps in the evaluation plan spreadsheet
  • [MI 2.3] Shared story points and contributors for the journey of the Contributors' measurement strategy development and presentation plan. Began to map diagram.

Next Steps

  • [MI 2.1b] Prep for Oct 30 sync session.
  • [MI 2.2a] Sync with working group to confirm measurement model and identify any remaining research gaps.
  • [MI 2.2a] Draft strategy brief.
  • [MI 2.2a] Follow-up to fill gaps in the evaluation plan on definitions and readiness estimates
  • [MI 2.3] Finish diagram and flesh out story points for presenting the journey of the Contributors' measurement strategy development. Share to collaborators.

Progress update for the week (MI-2)

  • [MI 2.1b] Synced in kickoff session for the consumers' measurement strategy prework.
  • [MI 2.2a] Followed-up to fill gaps in the evaluation plan on definitions and readiness estimates.
  • [MI 2.2a] Synced with working group to confirm measurement model and identify any remaining research gaps.
  • [MI 2.2a] Followed-up with next steps for input to the pennultimate model and suggested measurement concepts.
  • [MI 2.3] Reconnected with KR pillar lead and WE 1 objective owner to clarify connections to key north star / topline indicator. Adjusted the model to add one mid-term indicator for impact awareness and a direct arrow from collaborations to editors related to the flow of not just collaboration leaders but also contributors to retained editors. Corresponding to this clarification we adjusted documentation to include count of collaborations at the start of the funnel to collaboration leaders and participants, as it is the area's key indicator despite only flowing to the retained editors metric through the count of contributors who participate.
  • [MI 2.3] Finished diagram and fleshed out basic story points and collaborators to a draft deck for presenting the journey of the Contributors' measurement strategy development. Shared to manager.

Next Steps

  • [MI 2.2a] Connect with WE 5 Objective owner on next steps. Draft strategy brief.
  • [MI 2.1b] Check-in on / Follow-along and synthesize WE 3 measurement strategy collaborator inputs.
  • [MI 2.2b] Prep miro board for sync with full WE 3 working group Nov 12

Progress update for the week (MI-2)

  • [MI 2.1b] Followed-up on progress at consumers' measurement strategy prework.
  • [MI 2.2a] Followed-up again with next steps for input to the penultimate model and suggested measurement concepts.
  • [MI 2.3] Added links and story points to presentation deck scheduled working session for collaboration.
  • [MI 2.3] Synced with WE1 objective owner on research prioritization and status and next steps for advancing new metrics.
  • [MI 2.3] Synced with cross-sectional team and objective owner on quicksurveys improvements and nexts steps for potential implementation.

Next Steps

  • [MI 2.2a] Connect with WE 5 Objective owner on next steps.
  • [MI 2.1b] Check-in on / Follow-along and synthesize WE 3 measurement strategy collaborator inputs.
  • [MI 2.2b] Prep miro board for sync with full WE 3 working group Nov 12
  • [MI 2.3] Circle back to manager with input on WE1 metrics advancement and additional research prioritization input
  • [MI 2.3] Regroup with WE1 objective owner and research collaborators for survey implementation strategy and next steps mapping.

Progress update for the week (MI-2)

  • [MI 2.2a] Connected with WE 5 Objective owner on next steps for unblocking review and input. We now have 2/5 inputs recorded.
  • [MI 2.1b] Followed-along WE 3 measurement strategy collaborator inputs for synthesis.
  • [MI 2.2b] Began to prep miro board for sync with full WE 3 working group Nov 18th.
  • [MI 2.3] Circled back to manager with input on WE1 metrics advancement and additional research prioritization input

Next Steps

  • [MI 2.2a] Reconnected with WE 5 Objective owner on progress in unblocking review and input and any additional process adjustment recommendations.
  • [MI 2.1b] Facilitate group session for measurement model mapping and then begin to vet the identified metrics.
  • [MI 2.3] Identify shared tracking and status practice for metrics needing development
  • [MI 2.3] Regroup with WE1 objective owner and research collaborators for survey implementation strategy and next steps mapping.

Progress update for the week (MI-2)

  • [MI 2.2a] No additional progress. We have 2/6 inputs recorded.
  • [MI 2.2b] Prepped miro board and straw dog measurement model for sync with full WE 3 working group
  • [MI 2.1b] Facilitated group session for measurement model mapping and now carrying out async follow-up steps to revise the model and begin to vet the identified metrics.

Next Steps

  • [MI 2.3] Identify shared tracking and status practice for metrics needing development
  • [MI 2.3] Regroup with WE1 objective owner and research collaborators for survey implementation strategy and next steps mapping.
  • [MI 2.2b] Advance iterations of the model on the miro board and begin to vet the metrics

Progress update for the week (MI-2)

  • [MI 2.2a] Mapped engagement strategy for offsite and potential follow-up interaction session. Still partially blocked, we have 3/6 inputs recorded, and are moving forward with the additional inputs shared there and via monthly check-in.
  • [MI 2.1b] Connected with objective owner on open comments from sync and new connections and metric additions.
  • [MI 2.2b] Prepped miro board with take 2 on the straw dog measurement model integrating session feedback and follow-up clarifications for additions for sharing with WE 3 working group
  • [MI 2.3] Identified shared tracking and status practice for metrics needing development

Next Steps

  • [MI 2.1b] Follow-up to open questions and input gaps in the theories of change - begin to synthesize.
  • [MI 2.3] Add additional information columns and details to draft tracker shared for practice in status sharing for metrics in varied stages of reaearch and development
  • [MI 2.3] Regroup with WE1 objective owner and research collaborators for survey implementation strategy and next steps mapping.
  • [MI 2.2b] Advance iterations of the revised detailed model on the miro board and begin working at a simplified model, begin to vet the metrics.
  • [MI 2.2a] Prep for offsite engagement strategy.

Progress update for the week (MI-2)

  • [MI 2.1b] Following up on open questions and input gaps in the theories of change - begining to synthesize.
  • [MI 2.3] Added additional information columns and details to draft tracker shared for practice in status sharing for metrics in varied stages of research and development
  • [MI 2.3] Connected on lower level indicators for integrating to the dashboard insights.
  • [MI 2.2b] Advanced a second iteration of the revised detailed model on the miro board and begin working at a simplified model.
  • [MI 2.2a] Prepped for offsite engagement strategy.

Next Steps

  • [MI 2.1b] Continue to iterate on model and synthesis.
  • [MI 2.3] Regroup with WE1 objective owner and research collaborators for survey implementation strategy and next steps mapping.
  • [MI 2.2b] Advance iterations of the revised detailed model on the miro board and begin working at a simplified model, begin to vet the metrics.
  • [MI 2.2a] Offsite