Page MenuHomePhabricator

Make a tool to test the quality of section recommendations
Closed, ResolvedPublic


Create a tool to test the quality of section recommendations by collecting binary feedback from experienced editors as whether a section recommended for an article is a good/correct section to be recommended or not.

Event Timeline

Michele Catasta built this tool and we tested in in Wikimania. The tool lived at I'm checking with Michele to see if we can keep it up, or get some screenshots of it, at the very least. (Michele is not on phab, so I assigned this task to myself, which is not correct.:)

@Pirroh can we have the tool up? If yes, I can claim victory on this task and move it to done. Having it up will also help us with further data collection that we should probably do in the coming weeks.

leila moved this task from Backlog to In Progress on the Research board.

The tool is now up and running here:
We might want to circulate it around only after we update the recommendations.

@Pirroh: perfect. thanks. I agree that we shouldn't circulate for now.

@Pirroh is the tool up? I can't access it. And while we're at it I'm wondering: this seems to be a kind of tool we need engineering help to maintain, expand, etc. Does it make sense to ask Baha to help with surfacing it somewhere on Wikimedia end?

@leila: unfortunately the Azure VM where we are running the tool is down again (i.e., we had some issues with the free credits).
In case of no further hiccups, we should be up and running again by Monday.

Re. engineering: as of now this is a very small hack, and the only reason why it went down is our bad luck with Azure.
I guess we can let Baha go full throttle on the extended GapFinder app -- we'll consider escalating this issue to engineering only if the tool will realistically get way more traffic than we expect now (i.e., just a few tens of editors).

@Pirroh I want to put it in the quarterly review slide deck, but if people go to it and doesn't work, it's not cool. :D

@Pirroh it's not loading for me again. Let's bring Baha in to help. I'll message off-thread.

@leila: it's loading for me. Can you try a force refresh / anonymous browser session? The DNS record was modified again, so that might be the issue on your side.

no. in incognito and with hard refresh it still demonstrates the same problem: it takes a long time to load the page and it eventually shows time out. Can it be that you're reading from cache?
And it works also for Tiziano. Could it be some networking issue from your side? I'm running out of options :(

/facepalm @Pirroh for archive happiness: I was using the old link. sorry about that.