Changes incorporated into the Reading Offsite survey:

• Added a description to the Survey, working with the hypothesis that describing what the survey is for and how it will be used, as well as clearly establishing anonymity, will lead to a higher rate of engagement with the survey as well as open the door to more "constructive" feedback, rather than a pressure to provide positive feedback.

• Punctuation upgrade, working with the hypothesis that survey takers will respect the survey more if it is consistent in things like punctuation, grammar, spelling, etc (i.e. TPG is a details-oriented team and should reflect that in its public-facing work).

• Started numbering the questions earlier (and numbering un-numbered questions), working with a hypothesis that the survey is easier to engage with if the numbers start when the questions start (and don't start at a number that isn't "1").

• Added specifying text to the question "What skills were used" to focus the answer on how much easier or harder participants in the engagements could engage, working with a hypothesis that the question could be misinterpreted, otherwise, to mean "what skills did you yourself use" or "what skills did others who may or may not be TPG use" rather than asking specifically how TPG did (which I assume is the question).

• Moved "If you received other types of support not mentioned above, please describe" from a separate question, to "Other" as part of previous question, working with the hypothesis that the question structure makes more sense if all the answer options are in the same question (which is asking the same thing as having these 2 separated) rather than 2 different questions.

• Updated numbering to reflect earlier counting on questions

Other changes:

• Embedded the survey in the email directly, working with a hypothesis that it is more likely to receive a response if there are fewer layers (link clicks) with which to engage.

• Allowed editing of answers post-submission, working with a hypothesis that the feedback the user meant to convey is more important than the feedback they initially conveyed (assuming a discrepancy in some cases), with the risk that a user may provide valuable feedback that they then remove.

• Added a confirmation message upon completion of the survey, working with a hypothesis that it's friendlier and confirms to the user that they are done and can move on with their lives.

Suggested changes not reflected:

• A few times the survey asks "if yes then answer, if no then skip" and Google Forms actually has a mechanism for doing this on a paginated basis, rather than asking the user to manage the cognitive load themselves and navigate. However, I opted to leave it out because A) it's a little onerous to change and I was lazy, and B) I don't know how that affects embedding in email, which I thought was more important.
  • This would also allow for a progress bar, which also suffers from the open question around how that appears in email.

Almost all of these look like great changes to me.

I'm a bit nervous about injecting engagement-specific text into the survey at multiple places, because it substantially raises both the TPGer effort and the potential for error.

I didn't know you could embed the form into the email. We should test that it works cleanly with non-gmail (and especially non-HTML) email clients.

Allowing people to edit their answers also has the risk of throwing off our NPS calculations. But I still think it is a good change, and worth doing.

@JAufrecht Do these suggestions conflict with T162475: Update Light Engagement Survey with some tweaks?

I made the changes to the template, save for the "Other" option, which I found was less effective at getting actual answers from participants. @JAufrecht replicated the changes in the "ongoing engagement" equivalent survey (see T162475: Update Light Engagement Survey with some tweaks).

I have implemented these changes to the "ongoing engagement" version of the survey to keep them in sync.