Page MenuHomePhabricator

Developer Satisfaction Survey: analyze and visualize December 2023 survey results
Closed, ResolvedPublic

Description

Goal: Provide support for RelEng, Cloud Services, Technical Doc, and Research teams for current FY iteration of the Developer Satisfaction Survey: cleaning, analysis, and visualization of survey data.

Q3 Tasks:

  • Close survey after Jan 5
  • Cleaning and initial descriptive stats
  • Finish updaing R scripts (cleaning, wrangling, analysis, and viz) per survey changes/additions
  • Provide initial visualizations to @thcipriani, @nskaggs, @Pablo, @KinneretG, etc. and determine any visualizations needed
  • Finish hand coding with @thcipriani
  • Provide cleaned data to @thcipriani, @nskaggs, @Pablo, @KinneretG, etc.
  • Apply weights with @YLiou_WMF
  • Finalize visualizations
  • Upload visualizations to Wikimedia Commons
  • Support report finalization on MediaWiki
  • R code to GitLab

Event Timeline

Weekly update:

Since last weekly update, I have completed the following

  • Closed survey after Jan 5
  • Cleaning and initial descriptive stats
  • Finished updating R scripts (cleaning, wrangling, analysis, and viz) per survey changes/additions, as well as
    • Applying color palettes that are on brand and meet accessibility recommendations
    • Resizing and saving visualizations as PNGs
    • Adding footnotes/caveats to visualizations
  • Continued hand coding
  • Shared initial visualizations to team for feedback, requests for additional breakdowns

Weekly update:

  • Finished hand coding with @thcipriani
  • Provided cleaned data to team
  • Reran 125 plots with cleaned data; uploaded new plot PNGs to share with team
  • Uploaded rerun plots to our shared visualizations doc, where team continues to provide feedback/requests

(Next week)

  • Finish adding the newly requested plots to the shared doc

Weekly update:

  • Finished adding newly requested plots to our shared visualizations doc
  • Finished adding the responses to open-ended questions as well as 'Other' response write-ins to the shared visualizations doc
  • Worked with Yu-Ming to apply weights to the data (weighted on Gerrit commits, based on actual Gerrit commits targets)
  • Produced weighted visualizations i.e., visualizations using the weighted data, for questions deemed interesting/appropriate (QC still in progress, fyi)

Weekly update:

  • Produced and reviewed final exploratory/comparative/breakdown visualizations for @thcipriani
  • Developed plan for next week
    • prep files, file names, and metadata CSV for bulk uploading PNGs to Commons
    • finish formatting our draft doc (aligned with format of last mediawiki report)
    • begin drafting report text (e.g., "We asked '...?'", basic descriptives, etc.)
    • create table of questions to include (aligned with last mediawiki report question table)
    • add Ns to all likert plots

Weekly update:

  • Formatted our working doc like the mediawiki report will be formatted (e.g., see last report format)
  • Started adding summary/explanatory text to our doc
  • Made table for questions page (like last time: 2023/Survey_questions)
  • Added Ns to all likert plot visualizations (bar plots and stacked bar plots already have Ns)
  • Finalized visualization file names (i.e., how we want them named on Commons)
  • Created a CSV for bulk uploading the 100+ visualization files to Commons using, hopefully, MassUploadLibrary

Weekly update:

Weekly updates:

  • New deadline for report publication: pushed to next week
  • Stakeholders continue to review report final draft
  • Began data sanitization and archiving of de-identified aggregated stats in prep for data purge

Weekly update:

  • Completed visualizations. Compared to the previous iteration of the survey, the time it took to complete the visualizations from the close of the survey was reduced from 18 weeks to 8 weeks via automation and other previously-put-into-place documentation practices and templates.
  • Report is in draft mode 📝

Weekly update:

  • Generated missing/additional visualizations for the report; uploaded to Commons
  • Discussed with @thcipriani visualizations and how to talk about weighted subset of data for relevant questions
    • weighting is based on Gerrit commits made last year, so it only applies to the subset of respondents who used Gerrit last year and answered the question about commits made last year
    • for the public report, it likely only makes sense to use the weighted analysis in the question about Gerrit satisfaction (overall, and broken down by e.g. role)
  • Report is in draft mode 📝

Weekly update:

  • Provided info and visualizations for the new section of the report about weighted Gerrit responses; uploaded viz's to Commons
  • Report is in draft mode 📝

Weekly update:

Weekly update:

  • Public code published to GitLab repo (on April 3)
  • No major updates this week

Weekly update:

  • Supported report finalization, helped finish qualitative coding and summary analysis
  • Report will likely be published to on MediaWiki by EOD Friday