Page MenuHomePhabricator

Request from Desiree: Dev skills survey to determine who has which skills
Closed, ResolvedPublic

Description

For the individuals & teams that we support, we want to know:

  1. What skills do they have?
  2. What skills do they think they need?
  3. What is the team's capacity to deliver on projects requiring specific skills, tools, or languages?

User Stories:
As a WMF Product or Technology team member,
I want to understand the breadth and depth of skills/tools/coding languages of individuals and teams,
So that when we assign work to teams that requires special expertise, we ensure that team has the required expertise,
And we can better understand training needs on our teams,
And we can better align owners/maintainers to our tools, services, and workstreams.

As a WMF Product or Technology team member,
I want to understand the skills/tools/coding languages in which I need experience & expertise,
So that I can provide value to my team and be successful in my roles/responsibilities.

Acceptance Criteria:

  • A document (your choice of an internal Google doc/sheet, or a public-facing wiki page) that includes:
  • A list of skills/tools/coding languages that the team thinks are relevant/valuable to the team.
  • Node.js is a required line item
  • A team-wide assessment of its skill/capacity for each item on the list created in the previous bullet. This should be the average of individually-assessed skill levels for all engineers for each item. If a team has already completed this on the individual level, help teams convert that to a team-level assessment by averaging the scores.
    • Recommendation to work in partnership with managers to gather individual team members' skill assessments in order to help mitigate cognitive biases inherent to self-assessments (e.g., a manager may be able to help identify situations where their report's subjective assessment of their skill level in an area is not aligned with their objective performance).

Skill levels should be standardized as follows:

Numeric ValueSkill LevelDefinition
1MinimalNew to you, or rudimentary comprehension
2BasicDeveloping comprehension, gaining applied experience (not necessarily on Wikimedia projects)
3IntermediateDeepening comprehension, applied experience on 1+ Wikimedia projects
4AdvancedDeep comprehension & applied experience within Wikimedia context; have not provided training to others
5ExpertDeep comprehension & applied experience within Wikimedia context; have provided training to others

Optionally, consider including:

  • Skills/tools lists and assessments for UX Designers, Design Researchers, Community Relations Specialists, Program Managers, Product Managers

Simple Example:

Skill/toolTeam Average
CSS4
Docker5
JavaScript4
MediaWiki Action API5
MediaWiki production deployment3
MySQL4
Node.js4
PHP4
Phabricator5
Puppet4
Python3
React4
Testing strategy3
Unit testing4
Vue.js4

Examples & Resources:

Please check when complete and provide link(s) to page(s):

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
MBinder_WMF shifted this object from the S1 Public space to the Restricted Space space.Apr 7 2021, 8:53 PM
MBinder_WMF added a project: Unknown Object (Project).
MBinder_WMF renamed this task from Dev skills survey to Request from Desiree: Dev skills survey to determine who has which skills.Apr 7 2021, 8:56 PM
LGoto triaged this task as Low priority.Apr 21 2021, 8:40 PM
ggellerman raised the priority of this task from Low to Medium.May 26 2021, 8:37 PM
ggellerman moved this task from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.
LGoto set Due Date to Jun 30 2021, 7:00 AM.
ggellerman changed Due Date from Jun 30 2021, 7:00 AM to Jun 30 2021, 12:00 AM.May 26 2021, 8:40 PM
MBinder_WMF changed Due Date from Jun 30 2021, 12:00 AM to Jun 30 2021, 7:00 AM.

Please put this ticket on hold until we hear back from Marcella. She will align with Desiree about the requirements for the survey and asked us to pause for the moment on collecting info.

MBinder_WMF moved this task from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.Jun 2 2021, 8:29 PM

@LZaman to get in touch with Marcella or Desiree

LGoto changed the task status from Open to Stalled.Aug 11 2021, 8:40 PM
ggellerman changed the task status from Stalled to Open.Aug 11 2021, 8:40 PM
ggellerman updated the task description. (Show Details)
ggellerman removed a subscriber: Rileych.

Hi team, I spoke with Marcella about this and we can pick up this work again, it is now unblocked.

@ggellerman to chat with @ldelench_wmf about a good starting point for the artefact.

Would welcome any suggestions to clarifying our user stories & acceptance criteria - thanks all!

@ldelench_wmf , here are my 2 cents:
Based on the examples shared in the description of this ticket, some teams have already created a skill matrix for themselves. Most of these examples used a 0-5 scale where 0 means "no idea" and 5 means "expert". So we don't reinvent the wheel, should we stay consistent with this original scale or give people an option to use a 0-5 scale or 1-9 one as you suggested?

MShilova_WMF moved this task from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.

Thanks for this @MShilova_WMF , I love using "wheels" that already exist. :)

MBinder_WMF set Due Date to Mar 31 2022, 7:00 AM.

Regarding Due Date stamp not showing on this task:

The form used for this task is 3, but the other tasks are 54. I think probably 3 shouldn't allow Due Dates but somehow has. The stamp is a UI hack, and is probably tied to the other form. I could ask a Phabmin to change it, but given the restricted nature of our tasks that might be more trouble than it's worth. We could also recreated the task, but that might lose some history. I suggest we just leave it (and resolve it soon so we never have to think about it again)!

MBinder_WMF shifted this object from the Restricted Space space to the S1 Public space.Feb 2 2022, 9:46 PM

Shifted task visibility deliberately :)

Okay, I know i'm late to the party, -- got some questions...

We decided on the 1-5 scale?
Did we decide on every team adding this to their wiki pages? or a Google doc? Doesn't matter?

Discussed 16 Feb:

  • We'll use the 1-5 scale
  • Individual scores should be private (office wiki or google doc, but office wiki is not accessible to contractors)

Discussed 23 Feb:
re "A self-assessment of each Engineer, Tech Lead, Engineering Manager, Quality Test Engineer (QTE), and Data Scientist/Data Analyst on the team of their skill level against each skill/tool/coding language on the list."

For final deliverable it's OK to report by team, not individually.

If only one person on a team has "5" on a skill and everyone else is "1" or "2", might not want to list that skill

@ldelench_wmf

Update the acceptance criteria for this ticket: focus on the team's skills rather than individual (individual assessment, but output reflects the team). If a team has already completed this on the individual level, help teams convert that to a team-level assessment.

@ggellerman to reach out to non-TPgM team folks about status

@CBogen @MaryMunyoki

The due date for this task is 2022-03-31, and there is an issue with a form that is preventing the due date from showing where it usually shows. Does this give you enough time to fulfill the request?

@CBogen @MaryMunyoki

The due date for this task is 2022-03-31, and there is an issue with a form that is preventing the due date from showing where it usually shows. Does this give you enough time to fulfill the request?

The SD team met and had a discussion about this, and @MarkTraceur is working on documenting. Last we talked about it, the 2022-03-31 deadline worked for him - @MarkTraceur can you confirm? Thanks!

ggellerman changed Due Date from Mar 31 2022, 7:00 AM to Apr 7 2022, 7:00 AM.Mar 23 2022, 6:38 PM
MShilova_WMF changed Due Date from Apr 7 2022, 7:00 AM to Apr 29 2022, 7:00 AM.Mar 30 2022, 6:44 PM

Status update: Campaigns is in progress (one more engineer needed); kicking off with Design Systems' new EM next week.

Status update: Campaigns & Design Systems are in progress.

@ggellerman Updated with T&S's skill Matrix. Jay will be working on AHaT's next week.

Currently blocked by Desiree's conversation with the Engineering Managers sponsoring this request.

Added Campaigns! DST is still blocked.

LGoto changed the task status from Open to Stalled.May 25 2022, 6:46 PM

@MShilova_WMF: Hi, the Due Date set for this open task passed a while ago. Could you please either update or reset the Due Date (by clicking Edit Task), or set the status of this task to resolved in case this task is done? Thanks! (Also, what exactly is this stalled on?)

@Aklapper Thanks for the reminder about the Due Date. For some reason, this task does not show the Due Date stamp on the board view, so it's often overlooked. If that's causing problems, we could probably just clear that field. We think it might be a bug related to the form (perhaps being changed midway?) but we're not sure. See https://phabricator.wikimedia.org/T279593#7672410

(Also, what exactly is this stalled on?)

I believe this task is stalled on https://phabricator.wikimedia.org/T279593#7904487, and it simply took the team a few weeks to update the status to match that comment. :)

@MBinder_WMF: Uhm, I see... Could you raise an explicit ticket about the Due Date stamp not working as expected? TIA!

From Desiree, after I reached out to sync:

Yes. I'd like to touch base. We're actually trying to use them right now, so I'll have more feedback in the next week or 2. I'll add some time towards end of June if that works

I will wait to hear from her, sounds like I will meet with her in a couple of weeks.

MBinder_WMF moved this task from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.Aug 1 2022, 5:38 PM

Update after meeting with Desiree!

  • No more matrix gathering. What we have is what we have.
  • Desiree would like to treat this as a complete experiment:
    • What did we learn from teams that did or didn’t want to participate?
    • For teams that did participate, what approach did they take and why?
    • What are the pros and cons of what we currently have?
    • Etc

We’ll gather our perspective on this, and Max will bring it to Desiree to codify with her own data and ideas about core issues (philosophy, culture, safety, etc). That will then be something for a future VP of Engineering and/or Selenar perspective on this, and Max will bring it to Desiree to codify with her own data and ideas about core issues (philosophy, culture, safety, etc). That will then be something for a future VP of Engineering and/or Selena.

I think this means we can close this ticket, so I'll move Done. We can open a followup ticket for the next step.

MBinder_WMF mentioned this in Unknown Object (Task).Aug 1 2022, 5:42 PM