Offer personalized support to all newcomers at the event.

• We had a very robust mentoring program at the hackathon. We advertised this program on the MediaWiki event page, by email directly to anyone who identified during registration as a newcomer, by email to all participants, and at the hackathon itself. We had multiple newcomer sessions and check ins. We had a large team of dedicated mentors at the hackathon.

Follow up with them after the event, connecting them to tasks and mentors.
Check retention levels in September and December.

We have followed up with newcomers by asking about their experiences in the feedback survey and are also sponsoring three of them to attend Wikimania. More direct follow up is needed in the coming months. One idea is to inform them about the new featured projects tasks and let them know that they can participate remotely in the Wikimania Hackathon.
Developer Relations will work together to determine how to check retention levels and do more follow up with the Wikimedia Hackathon newcomers.

Thank you! I think the next step is to share the list of newcomers with @srishakatux and @Aklapper (last time I checked they hadn't seen it), so they can start helping defining and executing the retention plan.

In T163440#3424419, @Qgil wrote:

Thank you! I think the next step is to share the list of newcomers

Yes please - for example, https://wikimedia.biterg.io/ allows to check for activity by username(s) or email address.

I have re-shared the doc by email. It is the same one everyone had access to in Vienna.

I would like to start the discussion about "following up" and what is appropriate.
No newcomer opted into being emailed or followed up with even if they did opt into receiving updates about upcoming events. So because of this I do not support mass emails with a bunch of people on BCC. Yesterday when I was thinking about this I had the idea about how we can do the following up step.

If we like this plan, here are the steps that I see:

• DR goes through the list of newcomers and really researches each person and write out everything we know about their work and interest areas.
• Then we request to the whole team to prioritize taking 30 min each (we could even schedule a sprint time for it) and go through the list of people and make individualized recommendations based on what we know about them for next steps.
• Then we send a personal email with both generic information as well as the personalized information to each person. (somebody from DR will have to be willing to respond to these emails if the person write back and / or connect them to a mentor.)

I don't mind sending the initial email because I think most people who attended the event would recognize my name because I sent all of the pre-event communication but in the email I would need to introduce the person to either Andre or Srishti as a contact for email questions and next steps.

Personalized emails and the rest of the plan sound great!

It would be nice if we combine this email with the one that we will send out in the next few weeks or so to gather responses for T167085. As most of these newcomers, I am assuming will meet the criteria for a target audience that we've set for the survey.

Check retention levels in September and December.

[FYI] When it comes to measuring 'activity', I'd just like to share a quick impression what https://wikimedia.biterg.io can (not) offer here:
We can track some activity (tickets created in Phab, activity on mailing lists, MediaWiki edits, patchsets, but nothing in GitHub) per person and if I know/find Phab/Git usernames or email addresses in the database behind wikimedia.biterg.io. Some examples of folks who attended the Vienna hackathon and are listed in the doc:

• https://wikimedia.biterg.io:443/goto/dd4fe3878e6786b608e98136a023bf2f
• https://wikimedia.biterg.io:443/goto/74847780c2ad9cc8f42375b5a2f2537a
• https://wikimedia.biterg.io:443/goto/26750f6ed004fb40527ef73a7ab86ea0

No good idea whether and how to actually use that data though (plus it's a bit cumbersome to look up each user handle manually in the DB).

Currently the list has 45 alleged newcomers, and after a review a few will probably will be removed for not being actual newcomers. I guess the first checks for activity will be done manually quarter by quarter, until we find a better process. Anyway, as the discussion goes, we are seeing that retention (at least initially, with these numbers) will be based more on personal and manual activities, rather than automated processes.

Victoria had questions about retention at our quarterly check-in, I'll try to summarise her point later, so this is more of a reminder for me to do that :)

(I have since recapped her words to DevRel and now it's up to them to reach out to her.)

@Elitre we did! :)

@Rfarrand @Aklapper As discussed in our DR meeting last week "Srishti to proceed accordingly (contact via survey etc.) for those [not] active in Gerrit?" It's my quick guess that majority of the folks from Wikimedia hackathon will not be included in the "New Developers" study as the survey will be targeted at participants who submitted patches between July-September timeframe. So, this would apply in the context of T170495.
We can still send them a survey, but responses will not be part of our actual study. If not, then I think we don't need to wait for the survey to be ready to follow-up with the participants from Wikimedia hackathon.

Random thoughts:

One thing that we could possibly offer is an hour of 1:1 mentoring from a dev or some long term person for newcomers in their area of interest.
They can get to know someone, ask questions in a private way, maybe gain some momentum.

Even better it could be a two-part session - one where they get started and two where they get feedback and suggestions for their work.

I guess it is also important to somehow determine who actually WANTS to be involved. Just because they came to a hackathon and worked does not mean they want to spend evenings and weekends on our projects. If they don't want this at all for sure we can a) respect that b) focus more energy on others who do.

In T163440#3581912, @Rfarrand wrote:

I guess it is also important to somehow determine who actually WANTS to be involved. Just because they came to a hackathon and worked does not mean they want to spend evenings and weekends on our projects. If they don't want this at all for sure we can a) respect that b) focus more energy on others who do.

Agree.. in the follow-up email, we could cover the following:

• Appreciate their contribution at the hackathon

• Share resources
  • New Developers page
  • Outreach programs
  • Events guide for mini events/ hackathon they want to run in their community
  • #wikimedia-devrel or link to the technical advice meeting
  • Wiki Meetups in their area
  • Participants list from the hackathon with a disclaimer

• Ask them questions listed below:
  • Is there anybody who they met at Wikimania, and want us to connect them?
  • Is there any project that they started at the hackathon or are currently involved with, and need technical advice on it or need to be connected with a mentor?

And, then we see who gets back to us?

In T163440#3428408, @Aklapper wrote:

[FYI] When it comes to measuring 'activity', I'd just like to share a quick impression what https://wikimedia.biterg.io can (not) offer here:
We can track some activity (tickets created in Phab, activity on mailing lists, MediaWiki edits, patchsets, but nothing in GitHub) per person and if I know/find Phab/Git usernames or email addresses in the database behind wikimedia.biterg.io.

This turned out to be an "interesting problem" so I created T175854 for it.

Quick update on the "Get activity data of newcomers in wikimedia.biterg.io" part of this: T175854#3605693 in theory works to get the retention data I want.

• For the Vienna Wikimedia Hackathon 2017, newcomer (based on email address, Phab username, mw.org username if provided in registration data) online activity in Wikimedia infrastructure can be seen on wikimedia.biterg.io.
• For the Montreal Wikimania Hackathon 2017, newcomer (based on email address, Phab username if provided in registration data) online activity in Wikimedia infrastructure can be seen on wikimedia.biterg.io.

In practice, please note:

• The displayed data is not fully reliable until T176135: Filtering on "author_name" in Git can show unrelated activity by other users is fixed.
• Montreal data includes users' email addresses but only 5 Phab names and 0 mw.org names. If people use a different address for code etc, we cannot find their activity.
• The two query links above include anybody who identified themselves as newcomer (also staff, also veteran volunteers, etc). To only show volunteers, click the "Independent" part of the "Organizations" diagram to filter the view. After that you could filter the view for activity of a specific person by clicking the name in the "Top Authors" widget (but note the first bullet point!).
• In the above two query links, you cannot filter (via clicking an item) for activity of one specific newcomer only in the above two query links. In the above two query links, you can by manually editing the "Advanced Filter" field to only list a single name. After applying the "Independent" filter view (see bullet point above), you could filter the view for activity of a specific person by clicking the name in the "Top Authors" widget (but note the first bullet point!).
• Some accounts are detached in the DB. Which is not a big issue in practice. For example, I'm going to merge the author_names "ginger", "GroundGinger" and "Ingwer, Ginger H." into a single entity once updating the DB actually works.)

Gerrit activity statistics of Vienna Hackathon newcomers in 2017Q3: 3 people out of 37. Calculated by following these steps.

(I initially wanted to link to the "Overview" instead of "Gerrit" but had to learn that's not the best idea.)

Disclaimer: Of course we can only 'track' people who we can find. If they did not give us their on-wiki username and use a different email address for Gerrit than for the event registration and don't have their real name set in Phab or Gerrit they might still be active in our technical community, we just cannot "locate" them.)

Need to dump this somewhere so we'll have this handy in the future: Same query for Montreal newcomers (of course need to adjust the timeframe in the upper right corner). It also shows that we obviously missed to exclude one WMF engineer (mepps) from our final list.

As I own this task now (:P), and we also have a final list, should I work on a quick plan for following up with participants from Montreal and Vienna? Keeping in mind some of the points we discussed in our DevRel's meeting recently.

"a quick plan for following up with participants from Montreal and Vienna" is part of T176488: Creation of a short term plan to improve outreach and increase retention.

Update > Follow-up emails have been sent to 84 new developers who attended Wikimedia and Wikimania Hackathon in Vienna and Montreal.

Out of 84:

• 70 new developers received "generalized" email Everyone in the “Developer Relations: Event Newcomers Tracking spreadsheet” excluding the ones who we know worked on a project or we consider as retained to date (as they received a slightly modified version of the same email, see below).

• 8 received "slightly personalized email" Who we know worked on a project based on our conversations with them at the hackathon, their contributions in Gerrit and notes in the new developer's event tracking spreadsheet maintained by Developer Relations.

• 3 received "slightly more personalized" email Who we consider retained from Wikimedia Hackathon in Vienna based on new developers metrics and trends.
• We could not send emails to 3 developers who didn't provide us their contact information.

The Developer Relations event tracking spreadsheet has now an additional column "Follow-up emails as part of the retention plan" that tells which type of email from above did we sent to a newcomer.

Link to the Follow-up planning document (currently only available for viewing with a WMF id, send a request if you would like to see it):
https://docs.google.com/document/d/1b7aDjPWoVay14MxIvw1MTW9rGtYTCuYqUPmIJ9FLpoM/edit

Thank you!! I am really excited to hear about any responses received. :)

Out of 84 participants to whom emails were sent, 12 responded. Out of the 12, two never made to the event due to visa issues, five volunteered to participate in the research study (four via Google Hangouts, and one via email). Developer Relations team now has access to the "Interviews" document. A few not so *significant* conclusions:

• Those who volunteered to be interviewed already knew Srishti through the hackathon and had chatted with her before.
• Most of the volunteers said that they stopped contributing after the hackathon because they got busy with work. But they have enough information that they gathered from the event or is already out there on MediaWiki, that they could use if they want to start contributing again.

Can’t comment much on whether these emails were useful or not, but a slight hunch that it’s no harm to send emails like these and that we should send them right after the event to get more responses. They should either be sent by Rachel (like other event-related emails), or if Srishti sends them, then probably Rachel should be cc'd on it. Just so that participants don't feel they are receiving a follow-up from a stranger :-/

About retaining them, I think this should be a continuing topic of brainstorming as part of T176488.