For the fourth running of the search relevance test, we'd like to try a few things differently:
- Do not show the same user the same query on the same article, ever.
- Increase the timeout to 60 seconds for survey to display (currently 30 sec)
- Be sure to only show survey on article pages (not diff's or anywhere else)
- Update wording for the question to ask
- "If you searched for ‘Naval flags’ would this article be relevant?"
- Update wording for answer text
- Yes, I don't know, No
- Add link to wiki page where the search relevance test is explained (and offer the talk page for discussion)
- "Why we are asking this?"
- Add opt-out for logged in users that would last forever
- "Opt out of search relevance questions"
- Add 'close' button in top right of survey display
- write wiki page for why we're testing and request feedback
- turn on test
- turn off test
- analyize results
- plan next test