Page MenuHomePhabricator

Developer Summit 2017: Work with TPG and RelEng on solution to event documenting
Closed, ResolvedPublic

Description

See https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit_2016/Lessons_Learned#Use_of_the_combination_of_etherpads.2C_Phabricator_and_wikis_to_coordinate_the_sessions

In general people were happy, although there is a whole lot of room for improvement....

Last year's process: T120693: Checklist for WikiDev16 session organizers

Action Items from discussion:

  • Week of Dec 4th: Recruit note-takers for pre-scheduled sessions (along with remote participant advocates for T146613) before the event @srishakatux
  • Week of Dec 11th: Once we have a final schedule, create a wiki page where summary for all sessions would exist @srishakatux
  • Week of Dec 18th: Share note-taking, and summarizing guidelines w/ note-takers and summarizers @srishakatux
  • Week of Jan 2nd: Send reminders to volunteers @srishakatux

Event Timeline

Rfarrand lowered the priority of this task from Medium to Low.Apr 14 2016, 6:58 PM
Rfarrand lowered the priority of this task from Low to Lowest.Apr 25 2016, 8:27 PM

Removing Team Practices Group. Please feel free to add us back when there is something for us to do.

Also, please see our draft of how to engage with us:

https://www.mediawiki.org/wiki/Team_Practices_Group/How_To_Engage_With_TPG

@Qgil and @Rfarrand: This isn't exactly in my wheelhouse, so I'm not volunteering to own it. But I would be happy to try to help whoever chooses to take it on.

@Rfarrand here's what we advised "scribes" to do in last year's checklist:

  1. Copy https://etherpad.wikimedia.org/p/WikiDev16-YourSession to https://etherpad.wikimedia.org/p/WikiDev16-[SESSION NAME]
  2. Fill out the information in the template, including
    1. Link to Phabricator task describing session
    2. Goal as described by facilitator
    3. Topics of discussion as laid out by facilitator
    4. Action items and who is responsible for them
  3. Create new Phabricator tasks as needed
  4. At end of session, copy meeting notes into Phabricator as appropriate

If we do something similar this year, I'd recommend https://etherpad.wikimedia.org/p/WikiDev17-[PHABTASK] (e.g. https://etherpad.wikimedia.org/p/WikiDev17-T132400 or maybe even https://etherpad.wikimedia.org/p/T132400). If we switch to using Phab events, it could be even shorter: (https://etherpad.wikimedia.org/p/E285) and still be reasonably unambiguous.

We probably shouldn't try to use Etherpad as an index like we did last year. Last year's notetaking template (WikiDev16-YourSession) pointed people to WikiDev16-AllNotes. The AllNotes link wasn't that hard to intuit, but the brand-new set of conventions made it hard to figure out.

It seems that we should encourage scribes to copy their notes to one of two places:

  • A designated place on MediaWiki.org
  • Someplace on Phabricator.

What TechCom has been doing for our IRC meetings has worked pretty well. The routine:

  1. Meeting is announced as a Phab event (e.g. E285)
  2. "Notes" are generated as plain text
  3. Plain text gets copied to a Phabricator Paste (e.g. P4132)
  4. (optional) Plain text can be embedded in a Phab comment or description (e.g. by entering "{P4132}" in the comment)

Anyone who has been following our meetings might even be used to the convention.

Thanks @RobLa-WMF! Very helpful.

I would like us to be able to easily refer to the notes in the future - so maybe getting the lost in a Phabricator task that will be closed is not ideal?

Worst case we can create a mediawiki page linking to all the tasks, best case we can store the notes another way.

I am going to meet with Kevin next week and then hopefully work with @ksmith and @MelodyKramer to come up with a good plan for session organizing, note taking, pre-session information & documentation which will be useful for the long run.

All thoughts and ideas welcome here! Keep them coming...

Rfarrand raised the priority of this task from Lowest to High.Oct 11 2016, 9:30 PM

I am going to meet with Kevin next week and then hopefully work with @ksmith and @MelodyKramer to come up with a good plan for session organizing, note taking, pre-session information & documentation which will be useful for the long run.

@Rfarrand: Did that meeting provide any 'results'/thoughts worth to share already?

Hello! We just had our meeting today.

We agree that while Phab is the best tool for most attendees, some attendees will have never used it previous to the event. So we don't want Phab to be mandatory. However it will still be used for pre-event discussion and post event summary.

We think that during the event the wiki page schedule and etherpad should be the main tools. Wiki for the schedule itself and also a collection of links to all the etherpads. Eitherpads for the note taking.

We still need more thought into the best place remote participation. Obviously a dedicated IRC channel.

We need to improve our etherpad template. The bottom of the etherpad can include all notes word for word, but we need multiple summary questions at the top which will be clear and interesting. Tentative examples: What were the questions asked during the session? What is tentatively agreed upon by the group? What is still a point of disagreement? What are next steps and who are the owners? Most unexpected outcome?

Our next steps from this meeting include:

  • Get program committee involved - we need at least one person from the program committee to help with this effort.
  • bounce ideas off participant - once we have a final plan we should run it individually by a few participants who are not involved in the organization and get feedback in advance of the event.
  • wiki pages where the schedule is going to be look nice, be the one point of truth. @TheDJ @Rfarrand and @srishakatux can work on this. We also have lots of people who volunteered in registration to help with this.
  • Update Eitherpad - we need someone to sit down and think about what we want our session either pads to look like. What should be filled out in advance of sessions. What should be filled out during sessions. What should be filled out after sessions. What should be moved to Phab. How can remote people be involved? ANY VOLUNTEERS TO TAKE THIS?

You can find more details from the meeting here:
https://docs.google.com/document/d/1bCZCyqpJFSFpkjwDo2-rEfhCUgkfdz9MjK4gQXmI46o/edit

Tagging people from the meeting so that they can review and comment or make changes to my summary as needed.

@srishakatux @MelodyKramer @ksmith and tagging @Qgil

@Qgil Can you please help us find one person who is active in the program committee to help get involved with the documentation plan?

@Rfarrand : Great summary. Thanks!

Perhaps IRC is not an obvious choice, as it is notoriously difficult for people to get it configured and start using it if they aren't already. Would it be possible to build a "chat channel" section into the etherpad itself?

The level of detail of the notes is going to be tricky. If people take notes at all, they tend to fall into a full-transcription mode. Robla was promoting a dual-notes structure, where one person would transcribe everything, while another would create a summary in real-time. That seems like an advanced technique that we shouldn't expect from speakers or attendees. Most(?) people probably don't have the skills to summarize a conversation effectively in the moment. But if the summary isn't created until later, it runs the risk of never getting done, or getting done less effectively due to lost context.

I don't have solutions, but wanted to raise some of the challenges I see in the area of "taking good notes".

I could not attend the meeting, but I had shared some thoughts about Phabricator tasks vs wiki pages vs etherpads at T150150#2779743. They are mostly in line of your conversation, but I think etherpads should have a shorter life, and be pasted to the Phabricator task descriptions at the end of the session. After that, Phab tasks can be edited and etherpads can be forgotten/ removed.

@Qgil Can you please help us find one person who is active in the program committee to help get involved with the documentation plan?

What is this person supposed to do, in more detail (any serious volunteer would ask). :)

I think etherpads should have a shorter life, and be pasted to the Phabricator task descriptions at the end of the session. After that, Phab tasks can be edited and etherpads can be forgotten/ removed.

I'm curious whether wiki pages would be better post-event archives than phab. Wikis tend to be more friendly and discoverable, I think. I'm imagining closing the phab task after the event, so any further discussion would happen on wiki. But I don't know if that's desirable, practical, etc.

I totally agree that the etherpad content should be copied out and stricken ASAP.

Wiki vs Phabricator, imho it depends on the type of outcome from the session. If the outcome are basically action points, Phabricator tasks will handle that good. If the outcome is some draft for an agreement, a guideline, etc, then the wiki is where those documents need to live.

While Phabricator tasks might be more difficult to discover, they are also more difficult to forget. This is why I think that phab tasks are a good default.

Sharing a key learning from Aspiration Tech that we could apply here: What I liked about the conference was the recommendation - not to use laptops during the sessions. Maybe due to the nature of our event, we can't do this, but this makes me wonder why do we want to encourage collaborative note-taking on Etherpads and encourage multi-tasking/ elements of distraction? Pardon me if I misunderstood this :P Maybe only a few people contribute to our Etherpads.

What seamlessly worked well at Aspiration was assigning just one note-taker before the start of a session. Only the note-takers were suggested to use their laptops for documentation work. With help from organizers, these notes were posted on event agenda afterward. See: https://devsummit.aspirationtech.org/index.php?title=2016_Agenda. So each of their session is linked now to a wiki page with the summary. I think this approach will work perfectly for T147057 as well.

If it's about Q&A after the talk, we could try some Q&A tools that help facilitate better discussions such as the Harvard's 'Question Tool Instance Chooser.' The tool allows posting questions and replying and voting on them. If you want to play around with it, visit http://cyber.harvard.edu/questions/chooser.php and choose "WikiDevSummit" from the list. Attaching a screenshot below:

Screen Shot 2016-11-23 at 1.46.35 AM.png (1×2 px, 242 KB)

@srishakatux : The Team Practices Group has long been advocates of closing devices during conversations. When there is someone outside the conversation who is willing to take notes, it allows all the participants to completely focus on the discussion.

In this setting, however, there are a couple challenges. First, whoever is the notetaker won't really be able to participate in the conversation. If the notetaker is drafted from among the people who wanted to participate in this unsession, that's not great for them. Second, a randomly drafted notetaker might not have great scribing skills. Even with best intentions and full attention, the notes might not be great. I have seen sessions where the volunteer scribe ended up taking hardly any notes at all.

Even if one person is the primary notetaker, having others with the ability to share that load tends to work really well. The main scribe can participate in the conversation for a few minutes, while others take notes. Or if the scribe misses some important nugget, someone else can fill in the gaps.

So it's finding the best balance between everyone getting to participate (not everyone minus one), and getting everyone to participate (not just the half who avoid laptop distraction), and ending up with solid notes. I don't know what would be optimal here.

Last year the assignment of roles before the beginning of the session (including scribes) worked quite well, and I think we should repeat it. I agree that laptops closed is better, but @ksmith is right that having just one person doing that is exhausting and prone to fail in most cases.

One solution to this problem could be to put more attention to the pre-scheduled sessions, and rely more on self-organization for the unconference, providing clear expectations and instructions for everybody. The pre-scheduled sessions will be known between December 12 and 23. This means that we have many days to recruit the different roles beforehand, including (say) 3 scribes. We could even put additional focus on the pre-scheduled session in Room 1 (which will have video streaming guaranteed), assuring there the best standards. This "premium quality" should help the Program committee deciding which sessions are pre-scheduled, and which make it to Room 1.

About Q&A's and voting questions... there is something inconsistent about asking participants to close their laptops at the beginning of the session, and then encourage everybody to post questions online and vote them. I'd rather put the emphasis on laptops (and mobile phones) in sleep mode during the entire session, for better conversations.

Working on this next week...
@srishakatux it would be helpful if you could post a hand-off here tomorrow before you leave for holidays as to what you have done so far.
Including any note taking or session templates, any instructions for how people should sign up for things and or record things.

@Rfarrand planning to help with you on this one :) not going anywhere for holidays :D will hopefully share more by tomorrow or Monday!

I hope we have those cards for each role (notetaker etc) printed and used in all rooms. I found them very useful in the last edition.

@Rfarrand @Qgil using the documentation from the previous year, here is a page on Session Guidelines: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines and on All Session Notes https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/All_Session_Notes.

If they look okay, then I will batch edit all sessions on Phabricator to add a "Session Notes" section explaining creation of an ether pad link following the guidelines.

@Rfarrand and I briefly discussed that it might be too much to ask from participants to volunteer as remote participant advocate / note-taker prior to the event and session owners could recruit different role players right at the beginning of their session.

The new roles you have created look good to me, Srishti! Thank you! Any reason I should not create and print the cards out now based on the information there?

We will print cards for all 4 roles and have them laminated and in every single room. I will also have some extras in case of loss.

I think I saw that you had created it yesterday, but can you also link the page with documentation instructions here as well? The clear step-by-step for session owners / facilitators to make sure they are following our process.

Related and recently updated documents are below:

Session Guidelines: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines
All Session Notes https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/All_Session_Notes
Etherpad notes template https://etherpad.wikimedia.org/p/devsummit17-YourSession
Wiki notes template https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Your_Session
Role cards that Rachel printed out: https://docs.google.com/document/d/1kYUTLFOPCYxRi4P1-fiuH_T6yUSXsQ573BmD1jTjZ_c/edit?usp=sharing

As per the plan, here is how the note-taking process would look like. Note-taker(s) will use the Etherpad to take notes during the session, and capture whatever is possible. We have recommended minimum 2 and maximum three notetakers for a session.
After the session is over, we've asked them to copy action items into the Phabricator task, and important notes and discussion points and not the entire conversation into a wiki page and link it from the 'All Session Notes' page. These instructions are contained in the guidelines and the role cards.

@ksmith @Qgil Does this workflow sound good to you? Will it be possible for you to take a quick look and give any feedback, easier to incorporate between now and the summit :P?
On the 'All Session Notes' page, we are considering to link each session with its brief summary (wiki) page and not add the etherpad link. Does this make sense?

If so, then the plan would be to include the link to the session guidelines in the last email to our participants. I would also batch edit pre-scheduled and unconference session tasks on Phabricator to include a 'Session notes' fields, and add a sentence explaining that it should contain links to brief (wiki) and detailed summary (etherpad) pages and that each session owner should recruit role players before their session following the guidelines.

If there is anything that is not clear, let us know!

@srishakatux : I don't have rights to view that google doc, but the list of roles I saw elsewhere seems good. The templates look fine, and I didn't see any problems with the instructions.

The only thing that tripped me up slightly was the "choose one" for the meeting type. It's probably fine to try to get people to narrow, but I could imagine sessions that wouldn't exactly fit into any of those. Especially the "one solution" limit, since sometimes 2 or 3 solutions might be discussed, agreed to, or drilled down on.

@ksmith Thank you so much for reviewing. Now that you mention it, I agree about the session choices. Sadly we have spent some hours printing and laminating roles cards which include this information. We can verbally help people understand that this is not a forced choice.

I am going to close this task, because I think we are completely finished with our documentation plan mostly thanks to @srishakatux 's hard work!

@srishakatux : I don't have rights to view that google doc

Me neither. I guess there is no reason not to make it public to anyone for View?

Hah, the reason not to share it is because the doc is a bit of a mess due to the way it needed to be formatting for printing. The text on the cards are word-for-word the same as the text here:
https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines#Key_Roleplayers

Ok, I just made the role cards doc public, and I will add a picture of our fancy printed cards for future reference here tomorrow :)