Currently AFT5 uses 2*2 different ways to ask for improvement. It is fairly likely that a respondent doesn't notice both questions in either path, as one of them is in very large font, and another is in small gray text.

For the "Yes, I found what I'm looking for" path, the big headline reads "Great. Would you like to add a comment?" The smaller gray text in the textarea says: "How could this article be improved?"

For the "No, I didn't find what I'm looking for" path, the big headline reads "Sorry about that. Any suggestions for improvement?" The smaller gray text in the textarea reads: "What were you looking for?"

So respondents are answering four different questions:

1. "Would you like to add a comment?"
2. "How could this article be improved?"
3. "Any suggestions for improvement?"
4. "What were you looking for?"

Each of these surely has merits and drawbacks, but irrespective of that, the fact that we have four different questions, and no way for feedback reviewers to infer which question the feedback-giver answered, is seriously problematic.

To give an example: A feedback-giver writes "pulse valve" in the feedback form for the article "Air suction valve". If this response is in answer to question 4), it might indicate that the reader was looking for information about pulse valves and didn't find it. If it's in response to question 3), it might indicate that the reader has specific issues with that part of the article.

These four different questions may also make respondents seem less literate or willing to provide clear feedback than they actually are. As a respondent I might think I gave a perfectly clear answer ("I was looking for X, so I wrote X"), but the feedback-reviewer has no way to parse the response.

Likewise, the fact that the "Yes" path has such a prominent encouragement of "comments" (as opposed to suggestions for improvement) may bias the responses in favor of opinions and useless feedback.

In short, we should IMO ask a single question irrespective of the yes/no response. For example:

"Did you find what you were looking for? [Yes] [No]"

->

Yes: "Great. Do you have any suggestions that could make the article better?" Gray text: "Write your suggestion here, e.g.: 'The article would benefit from an illustration.'"

No: "Sorry about that. Do you have any suggestions that could make the article better?" Gray text: "Write your suggestion here, e.g.: 'The article would benefit from an illustration.'"

Asking a single consistent question with consistent instructions would help feedback-receivers parse the feedback, could improve feedback quality, and would make testing of alternatives easier.

Version: unspecified
Severity: normal