Page MenuHomePhabricator

[Hypotesis] 6.3.5 Obtain a shortlist of categories for the Toolforge sustainability scoring framework
Closed, ResolvedPublic

Description

Starting from the document of the refined user stories gathered in previous stages, complete the following:

Phase 3 - Shortlisting and Voting
Timeline: Complete by end Q2
Participants: Toolforge admins
Goal: Evaluate and vote on items from a sustainability perspective
Aim to shortlist 10-20 most promising ones
Considerations:
a) Relevance to platform sustainability
b) Alignment with WMCS team's scope and resources

Details

Other Assignee
dcaro

Event Timeline

Slst2020 changed the task status from Open to In Progress.Oct 10 2024, 12:40 PM
Slst2020 claimed this task.
Slst2020 triaged this task as High priority.
Slst2020 moved this task from Backlog to Toolforge iteration 15 on the Toolforge board.
Slst2020 edited projects, added Toolforge (Toolforge iteration 15); removed Toolforge.
Slst2020 updated Other Assignee, added: dcaro.
Slst2020 moved this task from Next Up to In Progress on the Toolforge (Toolforge iteration 15) board.
dcaro changed the task status from In Progress to Stalled.Oct 29 2024, 1:07 PM
Slst2020 changed the task status from Stalled to In Progress.Nov 8 2024, 10:35 AM
Slst2020 renamed this task from [Hypotesis] 6.3.5 Develop the sustainability score to [Hypotesis] 6.3.5 Obtain a shortlist of categories for the Toolforge sustainability scoring framework.Jan 15 2025, 8:14 AM
Slst2020 updated the task description. (Show Details)
Slst2020 claimed this task.

This hypothesis is now complete. Through a structured survey of Toolforge admins, we evaluated and prioritized sustainability categories based on platform sustainability impact and WMCS scope alignment.

What was accomplished during the hypothesis work?

    • Conducted and analyzed a structured admin survey measuring:
    • Platform sustainability impact (0-3 scale)
    • WMCS scope alignment (0-3 scale)
  • Identified top priority areas based on combined scores:
    • Log Management and Centralization (2.50/2.50)
    • Comprehensive Tool Health and Status Monitoring (2.33/2.20)
    • Simplified and Secure Toolforge Access (2.17/2.33)
    • Unavailable Source Code (2.17/2.20)
    • Complex Deployment Workflows (2.00/2.60)
    • Limited Visibility on Problematic Tool Behavior (2.00/2.50)
  • Developed initial scoring framework proposal with three core categories:
    • Observability (40 points)
    • Platform Access (40 points)
    • Source Code Availability (20 points)

Survey Results

Next steps

  • Define concrete, measurable metrics for each framework category
  • Establish clear "definition of done" criteria for binary metrics
  • Create measurement methodologies for percentage-based metrics
  • Define framework evolution process for existing and new capabilities