Google Code-in 2017: Collect Feedback and Lessons Learned
Closed, ResolvedPublic

Description

In this task, we collect feedback from Google-Code-in-2017 admins, mentors, students about organizational Wikimedia aspects.

Feel free to add comments!

Aklapper created this task.Nov 30 2017, 5:38 PM

Dedicated communication channel for GCI Wikimedia students?
<Albert221> What do you think of creating a Discord server, Facebook conversation or something for us, GCI competitors to talk about stuff? :)
<andre__> divadsn, there is no dedicated channel; this might be a good idea for smalltalk. #gsoc on Freenode is the general IRC channel for GCI talk, but that is not about Wikimedia in GCI.

Aklapper added a comment.EditedNov 30 2017, 5:40 PM

On contacting org admins: We need some mailing list so mentors can reach out to more than one admin (bottleneck!), e.g. when they suddenly become unavailable (family reasons, illness, etc). Make this scale better!

Edit: Requested in T181800

Stock answers / canned responses in tasks?:
Maybe sync what to reply in beginner tasks like "Get on IRC" to be motivational?
Like "Thanks a lot! We hope that IRC was an interesting experience and hope to see you around. Good luck with your next tasks!"

Dedicated communication channel for GCI Wikimedia students?

In the "new mentors" Hangout session on 20171201 the "dedicated/focused [IRC] channel like #wikimedia-gci" vs "general noisy channel where other devs are" was also mentioned - there are pros and cons for both. (We currently mention #wikimedia-dev in our GCI org profile, in most of our (not GCI-only) docs for new devs, and in the GCI beginner task to get onto IRC.

Input on deciding who are the winners:
At the end of GCI, admins need to decide on the Top 5 out of the 10 students who worked on the most tasks. This requires feedback from mentors. We ask mentors to please take notes if students go beyond expectations, went an extra mile, helped each other, etc., but maybe we should already do that in a centralized non-public place shared with all mentors while the contest is running? (GoogleDocs sigh, or is there some better way?)

Have a delay to reply over week ends. ;)

Elitre added a subscriber: Elitre.Dec 5 2017, 10:04 AM

Input on deciding who are the winners:
At the end of GCI, admins need to decide on the Top 5 out of the 10 students who worked on the most tasks. This requires feedback from mentors. We ask mentors to please take notes if students go beyond expectations, went an extra mile, helped each other, etc., but maybe we should already do that in a centralized non-public place shared with all mentors while the contest is running? (GoogleDocs sigh, or is there some better way?)

Would Etherpad help?

No, because it's public.

Dedicated communication channel for GCI Wikimedia students?

In the "new mentors" Hangout session on 20171201 the "dedicated/focused [IRC] channel like #wikimedia-gci" vs "general noisy channel where other devs are" was also mentioned - there are pros and cons for both. (We currently mention #wikimedia-dev in our GCI org profile, in most of our (not GCI-only) docs for new devs, and in the GCI beginner task to get onto IRC.

My personal opinion is that #wikimedia-dev was not the most appropriate channel and that #wikimedia-tech should have been used.

There is some value to the botspam to show gci students what else people are working on in wikimedia world and so they can see that their patch/bug comment really did go through. Or so they immediately see someone leave a comment on their patch. It even worked well on the weekend imho. But during sf work week, the level of bot spam was just unmanagable.

I would be opposed to a specific wikimedia-gci channel as part of the intent of getting students on irc is to have them integrate with the community. Using pre existing channels is critical to ensure they have casual encounters with community members not participating directly in gci.

Input on deciding who are the winners:
At the end of GCI, admins need to decide on the Top 5 out of the 10 students who worked on the most tasks. This requires feedback from mentors. We ask mentors to please take notes if students go beyond expectations, went an extra mile, helped each other, etc., but maybe we should already do that in a centralized non-public place shared with all mentors while the contest is running? (GoogleDocs sigh, or is there some better way?)

Private phab task?

I wouldnt neccesarily consider that a better tool than google docs, but it is open source.

Private phab task?
I wouldnt necessarily consider that a better tool than google docs, but it is open source.

Yeah, currently thinking the same. :-/

Private phab task?

I went for this option. I will announce it in an email to GCI mentors within the next days.

Find some way how to make sure mentors have "enough" tasks published, so mentors don't have to ping "please publish more tasks of mine; I'm running out".

RexxS added a subscriber: RexxS.Dec 18 2017, 12:25 AM

From an organisational point-of-view, I'd be interested in hearing other folks' opinions - and that probably should include the students - about tasks marked as Basic. I marked the first of my batch of "Learning Lua" as Basic to try to encourage beginners to try them. This was especially as some students might not find many tasks that they feel they could do, I'm hoping that those students may get some sense of accomplishment, at least.

However, I see two problems:

  1. The first is that students are only allowed to do 2 Basic tasks - so might not do any of the suite because they have already done 2 Basic tasks and each of my tasks require the previous one. I've allowed one student to start on [Lua task #2] for that very reason.
  2. The second is that tasks achievable by absolute beginners are too simple for those with programming experience, so there may be a temptation for the more able to snap up all of my tasks to inflate their count. We may have to give weightings to tasks to compensate.

Do others have any thoughts?

I assume "Basic" = "Beginner". Quite some mentors marked tasks as "Beginner" and I often did not understand why and reverted (as an org admin before publishing those tasks), as mentors deliberately limit the number of students who can claim these tasks.
Beginner Tasks are: "Simple, and less technical tasks that help students onboard and learn how things work. Students can work on a maximum of two beginner tasks. Examples: Getting on IRC, making first wiki edits, setting up the development environment and providing a screenshot of it, triaging three bug reports, etc"
I wonder how to rephrase, to have less mentors set the "Beginner" checkbox. (I'm serious.) Edits welcome.

Task weightings have been discussed in the past on the global gci-mentors at googlegroups dot com list (as it's not a Wikimedia specific problem) - in my humble opinion it's hard to slice tasks into similar sizes, and it'll be as hard to find a shared understanding for "easy", "medium", "advanced" categories among mentors.

RexxS added a comment.Jan 10 2018, 7:30 PM

I've found the limit of 1500 characters quite restrictive. Taking beginners through their first steps in a new language is rewarding, but requires a lot of detail in the instructions to keep them "on the rails".

In future, if we can't get an extension to what i consider a very arbitrary limit, we may need to use pages on-wiki to store detailed instructions, rather than in GCI itself. That may not necessarily be a bad thing.

I've found the limit of 1500 characters quite restrictive. Taking beginners through their first steps in a new language is rewarding, but requires a lot of detail in the instructions to keep them "on the rails".

In future, if we can't get an extension to what i consider a very arbitrary limit, we may need to use pages on-wiki to store detailed instructions, rather than in GCI itself. That may not necessarily be a bad thing.

Yeah. Ive been using phab tickets for the full details as I hit the 1500 limit really quick

For multiple times I was wondering and almost going to ask some admin whether there was enough tasks or should I put effort in creating more. Some more insight into that status would be nice. I think I decided there will be a CTA if we are seriously running out of tasks.

Access to the ongoing task from the dashboard is apparently not possible. You can just edit them from there, not access to the ongoing conversation with the student.

I've found the limit of 1500 characters quite restrictive. Taking beginners through their first steps in a new language is rewarding, but requires a lot of detail in the instructions to keep them "on the rails".

(It was 1000 chars once upon a time.) Google's take was that any info requiring >1500 chars should indeed be linked from the GCI task, e.g. to a MediaWiki page or to a Phab task.

Access to the ongoing task from the dashboard is apparently not possible. You can just edit them from there, not access to the ongoing conversation with the student.

@Trizek-WMF: Being on https://codein.withgoogle.com/dashboard/ , under "My Tasks", there is a link in "XX students are working on your assigned tasks" which brings you to the list of task instances that you mentor. From there you can click any task instance and access the ongoing conversation with the student.

Thank you @Aklapper

New feedback: that 36 hours limit to review tasks is not that cool over the weekend.

New feedback: that 36 hours limit to review tasks is not that cool over the weekend.

Google will not change that.

Google will not change that.

We should have a meme with that sentence.

On the subject of things google will not change - It would be nice if well one task is waiting for review, the student would be allowed to start the next task to reduce time where student has nothing to do (Provided that they only have one waiting for review and they've succesfully completed at least 2)

@Bawolff: That would mean that the student ends up with two tasks assigned if the mentor's review on the first one turns out to be "more work needed".

  • Graphic tasks like "Propose a design for stickers" must make it clear that when reusing existing work, students must provide sources (URL) and understand licenses and link to the license (URL).
  • I had a task requesting a logo proposal in SVG format. Many students embedded a bitmap image in an otherwise empty SVG file. That was not my intention. :P

@Bawolff: That would mean that the student ends up with two tasks assigned if the mentor's review on the first one turns out to be "more work needed".

Like IRL. :)

Ebe123 added a subscriber: Ebe123.Jan 17 2018, 3:55 AM

Positive:

  • This year I was a bit lazy and created tasks on Phab only and let the GCI admins import them into the website for me. I really appreciated that, because I felt like I always struggled writing proper descriptions for the GCI website.

Not so great:

  • There were a few cases where mentors were just added on the GCI website, and not mentioned in Phabricator, which got a little confusing to keep track of. Once it was the opposite - I was added to mentor a task without realizing it :-) (which I'm generally OK with)
  • Sometimes mentors (with good intentions of course) would prematurely approve tasks that weren't merged / might have had problems. At least for some tasks, I was thinking about having a "primary mentor" which would be the expected person to sign off on tasks, with other mentors still being able to "needs more work". I'm not sure how to balance that with not being a bottleneck for students.
  • Like every year, we need a better solution to getting students added to the jenkins whitelist. Maybe we can have a GCI task itself like "once you've had two merged patches in Gerrit, submit a patch to add yourself to the jenkins whitelist".

Feature requests:

  • +1 to @Nikerabbit's suggestion of being able to see easily if we're running low on tasks
  • Some kind of alerting if a Wikimedia GCI task hasn't been reviewed in 24h? (context: Once a student on IRC said their task hadn't been reviewed in a day, so I looked at it and approved it.)

For the task review part: I wonder if each task should have only one or two admins assigned to it and also tasks are equally distributed among admins. And, that way perhaps we would be able to increase slightly the number of tasks that get reviewed before the due date.

RexxS added a comment.Jan 19 2018, 1:49 PM

For the task review part: I wonder if each task should have only one or two admins assigned to it and also tasks are equally distributed among admins. And, that way perhaps we would be able to increase slightly the number of tasks that get reviewed before the due date.

The problem with that is that different mentors will have different interests and differing amounts of time available. I suspect that we'd be better off doing an analysis on where bottlenecks actually existed, and using that to inform future planning.

jayvdb added a subscriber: jayvdb.Jan 19 2018, 10:21 PM

Re notifications, I saw that a feature Google added towards the end of this years program was emails sent when the task is 24 hrs elapsed for mentors to review, i.e. 12 hrs to go.

Also the Zulip integration uses the GCI web-hooks, which we could extend to ping mentors and admins, or and we could implement our own web-hook receiver in one of the IRC bots.

Zulip org has a task uploader. https://github.com/zulip/zulip-gci/tree/master/tasks This also allows GCI students to get very actively involved in the GCI task creation process. e.g. hackerkid and synicalsyntax on https://github.com/zulip/zulip-gci/graphs/contributors , and probably others. Using a git repo to hold all the repeatable task descriptions allows a lot of checks to be created before the tasks are merged and then automatically uploaded into GCI, and could also be used as a way to store meta information from GCI. coala also created a task uploader, but it relys heavily on coala's very formalised system of GitHub issue labels.

I think a dedicated channel for GCI is preferred. It can be used as a congregating area for students and mentors alike , and students are then referred to other rooms depending on the tasks they are working on. Most other orgs do this. The students inevitably also hang out in those other 'work' rooms, and grow more comfortable chatting in them, but can have idle chatter in the GCI room from day 0 because it is 'their' primary work room.

I like @Legoktm 's suggestion of a jenkins whitelist task.

Looking ahead, it would be good to add our GCI students as GSOC project co-mentors for projects they were competent in. As co-mentors, GCI students can add a lot of value, especially at the beginning of the GSoC project where they can ensure the GSoC students are following coding and testing guidelines, ensure code is regularly getting merged. The GCI students learn a lot of project management skills this way, and also helps prepare them for proposing and completing their own GSOC projects in a year or two. All GCI orgs I've talked with try this to some degree. Terasology even had their GCI student be one of their GSOC org admin.

Also we should be helping the GCI students get connected with their local chapter/usergroup where possible, in order to engage with coding projects that their chapter is working on, or wanting, and may have a higher local impact. The high social impact is something Wikimedia GCI students often indicate made them want to participate with Wikimedia instead of some other org. Local engagement keeps student momentum between GCI, and helps them understand how their developer skills can be used to help the editing community. Also it will help GCI students who are English as a Second Language.

Aklapper raised the priority of this task from Low to Normal.Jan 22 2018, 11:40 PM
divadsn added a subscriber: rafidaslam.EditedJan 23 2018, 12:31 AM

Here are some thoughts from my side as a GCI participant:

First of all, thank you for organizing the participation, many things has been improved when looking back at GCI 2016. I was enjoying doing work by completing tasks and the idea of having a mentor to talk personally and ask about anything you stuggled through, that's where GCI stands out and makes the open-source world more attractive to young people like me.

To start off with the positive points, I found the selection of available tasks more balanced when looking back at 2016. I had no problems finding a task which would fit my needs and interests, while also keeping it more diversified so I was never bored during the 50 days. Also the split of a clonable task into multiple smaller tasks like in T183674 was a great idea to prevent students from working on the same extension simultanously and reduced situations where someone who claimed the task couldn't work on the task anymore, as other students have already completed it.

But I also noticed that most GCI students who were completing the IRC beginner task weren't not coming back later after their task got approved. In my opinion the problem lies in the way how IRC works and it's requirement to have the client connected and open 24/7 in order to keep up with the previous messages in a channel. I solved it myself by self-hosting ZNC as IRC bouncer and The Lounge as open-source web client, but not everyone has a server available or the knowledge and time to host those things themselves, especially students who had never got any experience with Linux.

And while I could help out @Albert221 and @rafidaslam by giving them access to my IRC bouncer and web client, I still think that the communication during GCI should be done on another platform like Telegram, as this would attract a lot more students and it's widespread and available on any device.

I've made a research about alternatives last year for GCI 2016 here:
https://meta.wikimedia.org/wiki/Research_on_open_source_team_communication_tools

Aside of that and some issues with the GCI Dashboard, everything else went fine with no big issues :)

  • Some [separate] channel for GCI students to talk about stuff?
  • Potentially unclear responsibilities among admins who publishes tasks?
  • Maybe sync what to reply in beginner tasks like "Get on IRC" to be more motivational? Like "Thanks a lot! We hope that IRC was an interesting experience and hope to see you around. Good luck with your next tasks!"
  • Maybe sync what to reply in last tasks before contest ends
  • As anyone who is a mentor could add themselves as a mentor to any task of that org, I think it can happen that someone adds themselves as a mentor without necessarily fully understanding the task they are supposed to mentor. We might want to make it clearer that mentoring means that you must be able to understand the code base sufficiently enough to review the patch, to not have people who want to help with best intentions but might not help.
RexxS added a comment.Jan 27 2018, 5:33 PM

Commenting on Andre's questions/suggestions:

The experience of most students not making use of IRC other than the first hour suggests to me that only a handful of the more experienced students are comfortable discussing on open or shared channels. Personally I doubt that a separate channel on whatever platform would attract more than the same handful, and it might discourage students from interacting with the wider community even more. I would suggest that we need to "sell" to the inhabitants of one of dev/tech channels the idea that their interaction with GCI students would be beneficial to Wikimedia if they saw the bigger picture and thought a little more long-term. The #Wikimedia-dev channel usually has around 200 folks idling there, but you're lucky if you see two comments an hour normally. Perhaps there's an equally useful channel that has a few more welcoming denizens where we could do some preparatory work prior to the next round. Any suggestions?

The online conference call for new mentors was invaluable for me, although it would have been more useful earlier. I appreciate all the work the organisers did (thank you! if I haven't made that clear before) and I know everyone is a volunteer, but it would pay dividends to give new mentors the earliest possible opportunity to understand more precisely what they will be doing, and what their responsibilities will be.

If we want to motivate students to interact more on IRC,we have to offer an incentive. What's in it for them?

It's worth asking all mentors to encourage students to continue their engagement with WIkimedia after the event closes. Perhaps having a page on meta specifically targeting CGI students with leads, contacts, suggestions for how they can continue, etc. might be useful as well?

I was a little disappointed that I only interacted as a mentor with a few other mentors (thank you particularly, Derick). Having a sense of working as a team is something I'd like to see developed among the mentors. Do we need a regular comms channel? make use of a separate mailing list? have regular conference calls?

I'm just thinking out loud here, and I understand that my style of teaching doesn't suit everybody, but I hope my reflections may be of some use to some mentors.

Bawolff added a comment.EditedJan 27 2018, 6:27 PM

Commenting on Andre's questions/suggestions:

The experience of most students not making use of IRC other than the first hour suggests to me that only a handful of the more experienced students are comfortable discussing on open or shared channels. Personally I doubt that a separate channel on whatever platform would attract more than the same handful, and it might discourage students from interacting with the wider community even more. I would suggest that we need to "sell" to the inhabitants of one of dev/tech channels the idea that their interaction with GCI students would be beneficial to Wikimedia if they saw the bigger picture and thought a little more long-term. The #Wikimedia-dev channel usually has around 200 folks idling there, but you're lucky if you see two comments an hour normally. Perhaps there's an equally useful channel that has a few more welcoming denizens where we could do some preparatory work prior to the next round. Any suggestions?

The online conference call for new mentors was invaluable for me, although it would have been more useful earlier. I appreciate all the work the organisers did (thank you! if I haven't made that clear before) and I know everyone is a volunteer, but it would pay dividends to give new mentors the earliest possible opportunity to understand more precisely what they will be doing, and what their responsibilities will be.

If we want to motivate students to interact more on IRC,we have to offer an incentive. What's in it for them?

It's worth asking all mentors to encourage students to continue their engagement with WIkimedia after the event closes. Perhaps having a page on meta specifically targeting CGI students with leads, contacts, suggestions for how they can continue, etc. might be useful as well?

I was a little disappointed that I only interacted as a mentor with a few other mentors (thank you particularly, Derick). Having a sense of working as a team is something I'd like to see developed among the mentors. Do we need a regular comms channel? make use of a separate mailing list? have regular conference calls?

I think taking over a non-bot infested channel (either #wikimedia-tech or #mediawiki) would help with this too

I'm just thinking out loud here, and I understand that my style of teaching doesn't suit everybody, but I hope my reflections may be of some use to some mentors.

Aklapper closed this task as Resolved.Feb 19 2018, 12:36 PM

Thanks for all the feedback. I dropped and re-ordered feedback in https://www.mediawiki.org/wiki/Google_Code-in/Lessons_learned#2017 but that's often copy and paste; summarizing a bit more is welcome.