Based on initial feedback from the team, we need to discuss this synchronously. Will try to schedule a time for us to chat this week.

On 2016-06-16 @Awjrichards @ksmith @JAufrecht @MBinder_WMF and I met and devised plan for Q4 CSAT:

• keep Likert ques the same as Q3
• draft 3 new open-ended questions based on themes that emerged in answers to Q3 open-ended questions

Then on 2016-06-17 @Awjrichards and I paired to draft questions.

I submitted those questions to @Capt_Swing for review

I drafted Q4 CSAT:

https://docs.google.com/a/wikimedia.org/forms/d/1Mnt-iEgUkooxWD74VQElI5SbAh92eBVt3h5EqJXwhh4/edit

@Awjrichards @ksmith @JAufrecht @MBinder_WMF @ggellerman:

Looking good overall, thanks fro the work on this!

1. I noticed in the section header for the new open-ended questions, it states that: "We are considering adding new questions to future versions of this survey. Your answers to the following will help us understand areas we should focus on in future surveys".

I believe this is leftover language from last quarter, where we solicited feedback on themes to get the very focus areas that we are now asking questions about. To avoid a recursive loop of refinement into infinity, I was bold and changed the header to the following:

"Your feedback on the following optional questions will help us learn and improve how we support teams:" Note that I also made the questions in the section optional.

2. Two of the three open-ended questions solicit input along poles ("positively and/or negatively affected...communication", "strengthening and/or weakening trust"). I updated the other question to be consistent:

"Can you provide any examples of the TPGer assigned to your team improved and/or detracted from the development of team cohesion?"

Please chime in on changes if you disagree!

Consulted @Capt_Swing and made slight change to cohesion question:

""Can you provide any examples in which the TPGer assigned to your team improved and/or detracted from the development of team cohesion?""

Updated in Google form draft.

@ggellerman It would be useful for me, as I evaluate which teams have members that are and aren't participating, to have "Reading Apps" separated as "Android" and "iOS." Feasible?

The "and/or" reads funny to me because of instances like "improved and detracted." Suggest we change to just "or" or reword.

@MBinder_WMF

Good catch on the and/or- I checked with @Capt_Swing and have changed it to "or" in all 3 open-ended questions.

I've updated the dropdown in the form to replace Reading apps with separate iOS and AN

I think we should also remove "Reading Leadership" the drop-down list.

@KLans_WMF Any reason in particular? That list includes

Toby Negrin
Nirzar Pangarkar
Jon Katz
Adam Baso
Moushira Elamrawy
Anne Gomez
Joshua Minor

The only 2 on that list I could see removing are Toby (not on a team) and Moushira (scaling back from Reading and rarely works with TPG). Anne Gomez is not on any of the teams on which TPG is embedded in Reading, so if we did include her we'd have to find a home for her on the spreadsheet. The others could be folded into the teams they serve (Web, Android, iOS).

@ggellerman I responded to an email thread, but I'll post that here, too, if this is where we're having the conversation:

I updated "Reading Apps" to "Android" and "iOS" and updated the names respectively. I also attempted to clean up some of the formatting of other section for consistency.

I noticed that the Q4 recipients list that you attached did not include any of Reading Leadership, as was listed for Q2. I don't know if we intentionally left those folks out for Q3 (they are missing from that sheet, too), which would surprise me, but I've restored them to the new sheet, and updated according to the "Staff and contractors" page on wikimediafoundation.org. [1]

Clarification: Are we only surveying teams on which TPG is embedded? That is, would we exclude OIT, Design (those not on specific embed-teams but that worked with Grace, Arthur), Performance, etc? What about folks from other teams that attended, say, the Reading Web Offsite (Casey Dettinger and Elliott Eggleston, from FR-Tech, for instance)? What about C-Levels like Wes and Katherine?

TL;DR: How do we define the threshold of engagement that qualifies someone for receiving the CSAT?

[1] https://wikimediafoundation.org/wiki/Staff_and_contractors

In T131310#2404220, @MBinder_WMF wrote:

Clarification: Are we only surveying teams on which TPG is embedded?

Traditionally yes, and I would expect that to remain true for this cycle. The other cases will be handled by our new "light engagement" satisfaction survey, which we will start piloting next quarter.

TL;DR: How do we define the threshold of engagement that qualifies someone for receiving the CSAT?

There are 2 questions here: Which teams should we be surveying, and which members/affiliates of those teams should get the survey?

For the teams, I think it would be any that have an embedded TPGer. If there are teams where we are partially-embedded, that would require a case-by-case judgment.

Within a team, I think any "member" should be included. So, for example, I'm including Chris in Discovery, because although he is a CL, and is not full-time on Discovery, he is substantially involved. When Discovery had an ops liaison who only spent a couple hours each week interfacing with Discovery, I would not have expected him to participate in the CSAT.

@MBinder_WMF

The list discussion is going on in T138536 ;)

Thanks for updating the list. It was a copy of Q3 spreadsheet. Looks like @ksmith has answered some of your questions. Thanks, Kevin!

@KLans_WMF:

Would it make sense to include heads of audience verticals? Though the support we provided in Q4 changed, I believe that we included them in Q3 survey.

@MBinder_WMF My rationale for not including Reading Leadership is that they were previously a team that we intentionally and formally supported (when I was on Reading); I think all of them get benefit and support indirectly from the TPGer in the vertical, but no one is actively working with them as a team.

Per Kevin's response about who should be included as a team "member", I think many peeps on the Reading Leadership list could be included in that context.

@ggellerman thanks for clarifying the other task for discussing the list. I think I got confused by some of the existing comments in this ticket. There's not much going on in that ticket, so I'll comment here like the others. :)

@KLans_WMF I distributed the relevant managers who work with teams in Reading, and removed the Reading Leadership section.

@ksmith The Reading Offsite is one example of something that is outside of "embedded" and I believe we decided was outside of "light engagement", too. So which survey captures that experience for those people? Or are we deliberately excluding?

In T131310#2405479, @MBinder_WMF wrote:

@ksmith The Reading Offsite is one example of something that is outside of "embedded" and I believe we decided was outside of "light engagement", too. So which survey captures that experience for those people? Or are we deliberately excluding?

I don't think we have decided whether offsites are inside or outside of the "light engagement" bucket. Also, we haven't decided which light engagements would get surveys and which would not.

Personally, I would like to see offsite attendees fill out some kind of survey at the end of the event. I'm not sure if that should be our standard light engagement survey or something special for offsites. Right now, we don't have either one to offer.

I definitely would not offer an Android quarterly embedded CSAT to an external attendee like Deb or Casey. Since that's the only tool we have right now, the consequence is that they would be excluded for now.

Works for me, thanks for clarifying. In this case, folks like Toby and Anne Gomez would also be excluded, though I think that makes sense even though they are in Reading and work with people from the embed-teams (they just don't work directly with a TPG'er).

I'm cool to check the box in the description " Resolve open questions that exist in comments before resolving this task".