Page MenuHomePhabricator

Plan release announcement for Discernatron, approx April 28th
Closed, ResolvedPublic

Description

Hoping to release the Discernatron (human judgment of search relevance) some time next week at https://discernatron.wmflabs.org. Anyways we should come up with some sort of plan to get some users onto the site and grading results for us. We plan to release a new set of 500 queries into it each month to be graded, and ideally we need each query graded by 10+ people.

For each query users will be provided the set of titles found by CirrusSearch, Bing, Google, and DuckDuckGo and asked to rank each title from 1 to 4, or leave the title unranked. We source up to 25 results per source, but due to duplicates there are probably 20 to 75 titles on each (not sure yet)

Possible avenues for getting users:

  • Village Pump Technical
  • Anywhere people have discussed how much search sucks
  • wikitech-l and discovery mailing lists
  • Moar?

Event Timeline

CKoerner_WMF added a subscriber: Ckoerner.
CKoerner_WMF removed a subscriber: Ckoerner.
CKoerner_WMF added a subscriber: CKoerner_WMF.

Looks good. I think it'd be good to have a screenshot or something there to liven it up a little.

I got the new oauth config so we can release with the better name, https://discernatron.wmflabs.org. Also made a tiny copy edit. Approved from my perspective.

Everything looks to be in place now, anything blocking sending out a release announcement?

Erik and I discussed this on IRC. Announcement sent to discovery-l and wikitech-l. Posted to Village Pump (technical) on English Wikipedia.