What we learned about the use of Zulip as a mentoring tool

• Most mentors didn't use the tool to communicate with students and org admins. Here are comments from some of them:
  • "I did not use it actively, but I got emails from it occasionally, which was useful"
  • "I greatly prefer phabricator and I won't use it again. But maybe worth keeping it as an option"
  • "I and our student never used or logged in to Zulip, so I can't really comment on this"
  • "I only checked it when I got email notifications. That worked fine, except I think my co-mentor once didn't get an email notification"

• One student wrote: "While it is open source and ethuses the community it is not a reliable source of communication"

• Analysis and next steps:
  • Most of the mentors didn't use the tool to communicate with students and mentors. It is not surprising as they were already on Phabricator group chat before the projects were accepted. Some of the mentors were using different synchronous communication channels to meet with their students. So, this is a fair assumption that switching to Zulip, an entirely new platform would have been extra work for them. Still, some of them who were familiar with the platform emphasized that email notifications they received were useful and that we could worth keeping Zulip as an option. As an organization administrator, the tool turned out to be extremely useful in keeping a close communication with students all throughout the program, specially in matters of urgent sync up. Zulip platform has a nice feature that allows threaded discussions. It was helpful that way to have organized and seperate discussions about each phase of the program. It might be worth to continue using the tool in the next round to be able to draw more concrete conclusions. One additional thing to try would be to introduce Zulip to mentors and students when the application period opens.

The summary above is my personal opinion, a second opinion would be helpful. Whatever we end up deciding as a next step, I will follow up with Zulip team accordingly.

Sidenote: Overall lessons learned related to GSoC/ Outreachy including the summary about Zulip added above is here https://www.mediawiki.org/w/index.php?title=Google_Summer_of_Code/2017/Lessons_Learned, also linked from the administrators pages of both GSoC and Outreachy https://www.mediawiki.org/wiki/Google_Summer_of_Code/Administrators

Is this still a thing that exists? I found this task by accident; I doubt many of the mentors for the Outreachy winter round are aware that there is a Zulip trial going on (if it is still ongoing). I've been using Conpherence which is very broken (especially due to T765, although that seems to be on the verge of being fixed now); an alternative would have been interesting. Although in the long term I would really, really like to see something with good IRC integration; any other approach will just fragment our mentor/helper community IMO.

Some of the mentors were using different synchronous communication channels to meet with their students.

Do we know which ones?

In T170356#3680069, @Qgil wrote:

Some of the mentors were using different synchronous communication channels to meet with their students.

Do we know which ones?

Google Hangouts, IRC, Slack, etc.

I'm intrigued at why Slack is considered synchronous and Zulip not. I guess because Slack has a mobile app?

Just a quick anecdote. I used zulip as part of GSOC 2017 mentoring for all the (weekly) meetings which we held, and we also used gitter to communicate occasionally for some of the more technical questions that needed an immediate response to avoid delays in the project. This is because gitter notifications worked better than zulip or IRC at ensuring a rapid response on specific chat rooms, because it also sends an email with any unread messages. OTOH, Zulip was better for scheduled meetings, and discussion threads which spanned large periods of time, and for keeping in contact with the other GSOC people (admins, mentors, students). Both clients were light weight, and low permissions, so it wasnt a huge drama for us to have multiple clients installed. We also both used IRC occasionally also, but not as a mechanism for having a discussion between mentor and mentee.

At the Mentor Summit, many GSOC orgs indicated they have migrated to, or are migrating to, either it or RocketChat for their GSOC programs. They are usually also keeping IRC, and often bridging everything together with Matrix behind the scenes. Zulip org is happily providing free hosting for any of these orgs so the other orgs can focus on their mission, and Zulip focus on its mission. I am happy to hear that Wikimedia will continue to keep a Zulip instance available for mentoring.

Zulip is rapidly improving with lots of GSOC and GCI fueled development in addition to the core team. As far as I can see, it is the fastest moving open source chat system. They have recently released an errbot connector, which enables many existing bots designed for gitter or Telegram to operate on Zulip also.

Worth noting that since the last evaluation of options, gitter has been 'purchased' by GitLab (who also 'owns' MatterMost) and was open sourced, so that is also another option on the table for self-hosting, if we wanted to go down that path. However development of gitter is moving slowly compared to Zulip.