This task is for the technical collaboration team to discuss what we are going to do to help support and continue to engage newcomers after the Wikimania Hackathon event. Unlike Vienna, we will not target 10% because we are unsure how many newcomers there will be and we also know that Wikimania Hackathon participants are less likely to be at the event for the event's sake and may just be dropping in. Also the shorter event and the fact that many of the participants are not technical also make a different in what technical collaboration can do in the long run.
Lets talk!
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Rfarrand | T170226 Organize the Wikimania Hackathon 2017 in Montreal | |||
| Resolved | srishakatux | T170495 Retaining Newcomers From Wikimania 2017 (Montreal) Hackathon | |||
| Resolved | • Rfarrand | T176137 Final lists of newcomers at the hackathons in Vienna and Montreal |
Event Timeline
Comment Actions
From measuring perspective, as per discussion in T163440, we know there is no automated way to do so.
In addition to encouraging newcomers to document projects on-wiki (which most of the times leaves us with little documentation), a few changes that we could consider:
- Format of the spreadsheet @Rfarrand shared with us for Vienna hackathon -- we add Github/ Phabricator links to separate columns, we also add columns to gather information about projects newcomers worked on, who they worked with, link to their GitHub/Gerrit commits (if there are any) from during the event, etc.
- We also help newcomers document the same information on-wiki wherever applicable.
- We ensure that we were able to gather information about all newcomers in the same format
We pre-assign 2-3 people for this in the same way we are assigning volunteers for other tasks.
Comment Actions
(I am setting priority to "Low" only to remove "To triage"; feel free to change it.)
Very good idea. I agree that setting a specific retention goal (10% or whatever) is not the point. What we don't have and it seems that we need is a systematic way to measure onboarding and retention of new developers in relation to events.
We are already working on KPIs related to new developers and retention on a quarterly basis. There is no need to reinvent that wheel. The specific points related to events are:
- From the data of new & retained developers captured in our KPIs, how many new and retained developers came from event X?
- Were there other newcomers at event X that our KPIs are not registering? For instance, genuine new developers who didn't work on projects that went through code review (i.e. a new prototype hosted in GitHub).
I was thinking of a page (could be https://meta.wikimedia.org/wiki/Technical_Collaboration/Metrics) where we have a table with this information (more or less) about events organized in the past 15 months:
| Name of event | Date of event | Location | Number of new developers | % active after 1 quarter | 2 quarters | 3 quarters | 1 year | Trend (that graph) |
| Wikimedia Hackathon 2017 | 19-21 May 2017 | Vienna (Austria) | 38 | 32% | 17% | 15% | 11% | |
Events older than 15 months would be archived somewhere else. This could give us an idea about how well each event is doing, and maybe we could start seeing trends about events or locations more successful or more tought at getting and retaining newcomers.
I can already hear @Aklapper saying that we don't have tools to measure this properly. Totally true, as of today this is basically manual work. However, note that this manual work could be shared with the organizers of the events (sometimes it is Developer-Advocacy, many times it is not). This way we would also involve these organizers in the measurement of retention, and also the activities to improve retention. We can assume that most newcomers of a regular event will come from the same region where the organizers operate (i.e. Vienna & Wikimedia Austria), and therefore they also have a vested interest in good retention awareness and rates.
Comment Actions
When registering, forcingencouraging more people to provide their email address and/or mediawiki.org user name and/or Phabricator user name would already help a lot. :)
Details on wikimedia.biterg.io only: Ever remaining problem is that even if a user had created a Phab account, as long as that account has never been active (in wikimedia.biterg.io for Phab accounts that currently means "user has created a task", not "has commented on a task" or such) it cannot be searched for in the DB hence we don't know if the person has ever registered in Phab or has registered and has had activity not indexed in the DB.
Comment Actions
To clarify: I assume that "New developers" are not "any first-time attendees", but folks who actually proposed a code change.
- From the data of new & retained developers captured in our KPIs, how many new and retained developers came from event X?
https://wikimedia.biterg.io/app/kibana#/dashboard/C_Gerrit_Demo lists the names of new authors in Gerrit (hence code activity only). We can manually compare these names to event registration data. However Gerrit username can again be anything, very different from mediawiki.org or Phab username...
(And maybe https://wikimedia.biterg.io/app/kibana#/dashboard/Git-Demographics would allow to get either slightly better data or an easier way, but currently cannot judge what these widgets will allow me to do until T171240 is fixed.)
- Were there other newcomers at event X that our KPIs are not registering? For instance, genuine new developers who didn't work on projects that went through code review (i.e. a new prototype hosted in GitHub).
If we have GitHub usernames we could manually check. Looking at my GitHub stats for 2015 I can see repository names listed at least, to differentiate (as I can be active in many projects unrelated to Wikimedia).
And yes, all that sounds like a lot of manual work. :)
Comment Actions
That is a good point, currently I am unsure if everyone on our list actually proposed a code change or not. They just self identified as newcomer and attended and were not taken off the list after review.
Comment Actions
So yes, there is a lot of manual work. But there is no way around this. We are supposed to follow up on newcomers, send them feedback surveys, ask them what they need... No matter what, onboarding new developers is manual work at least in our current stage.
A required first step is to go through the list of newcomers of Vienna and Montreal, see who uploaded patches, and continue from there.
Comment Actions
Developer Relations will be following a similar process to the process followed for Vienna.
More details coming later!
Comment Actions
For the future - we need a better strategy to understand who we should follow up with, what we should follow up with them on and how we can help them.
Ideas - better questions in registration and a very simple survey asking about what we can do to help them / where they are stalled / what to interest them in.
Comment Actions
I'm not sure what's left in this task that is actionable. @srishakatux: Any better clue?
(For the records, Srishti sent a survey to new developers who submitted code to Wikimedia Gerrit (which would include those Wikimania Montreal Hackathon newcomer attendees that managed to get code into Wikimedia Gerrit, and https://www.mediawiki.org/wiki/New_Developers/Quarterly/2018-01#Outreach_events and https://www.mediawiki.org/wiki/New_Developers/Quarterly/2018-04#Outreach_events list retention statistics.)
Comment Actions
Yes, that's true! There isn't anything more actionable on this task. Taking the liberty to close it!