Page MenuHomePhabricator

Make a tool to test the quality of section recommendations
Closed, ResolvedPublic

Description

Create a tool to test the quality of section recommendations by collecting binary feedback from experienced editors as whether a section recommended for an article is a good/correct section to be recommended or not.

Event Timeline

leila created this task.Aug 31 2017, 10:44 PM

Michele Catasta built this tool and we tested in in Wikimania. The tool lived at http://40.118.110.204/ I'm checking with Michele to see if we can keep it up, or get some screenshots of it, at the very least. (Michele is not on phab, so I assigned this task to myself, which is not correct.:)

leila added a subscriber: Pirroh.Sep 26 2017, 11:22 PM

@Pirroh can we have the tool up? If yes, I can claim victory on this task and move it to done. Having it up will also help us with further data collection that we should probably do in the coming weeks.

leila reassigned this task from leila to Pirroh.Sep 26 2017, 11:22 PM
leila moved this task from Staged to In Progress on the Research board.

The tool is now up and running here: http://sections.fooshed.net
We might want to circulate it around only after we update the recommendations.

Pirroh closed this task as Resolved.Sep 27 2017, 10:45 PM

@Pirroh: perfect. thanks. I agree that we shouldn't circulate for now.

leila added a comment.Oct 4 2017, 7:17 PM

@Pirroh is the tool up? I can't access it. And while we're at it I'm wondering: this seems to be a kind of tool we need engineering help to maintain, expand, etc. Does it make sense to ask Baha to help with surfacing it somewhere on Wikimedia end?

@leila: unfortunately the Azure VM where we are running the tool is down again (i.e., we had some issues with the free credits).
In case of no further hiccups, we should be up and running again by Monday.

Re. engineering: as of now this is a very small hack, and the only reason why it went down is our bad luck with Azure.
I guess we can let Baha go full throttle on the extended GapFinder app -- we'll consider escalating this issue to engineering only if the tool will realistically get way more traffic than we expect now (i.e., just a few tens of editors).

leila added a comment.Oct 4 2017, 11:26 PM

@Pirroh I want to put it in the quarterly review slide deck, but if people go to it and doesn't work, it's not cool. :D

Up and running again.

leila added a comment.Oct 11 2017, 7:00 PM

@Pirroh it's not loading for me again. Let's bring Baha in to help. I'll message off-thread.

@leila: it's loading for me. Can you try a force refresh / anonymous browser session? The DNS record was modified again, so that might be the issue on your side.

leila added a comment.Oct 11 2017, 7:37 PM

no. in incognito and with hard refresh it still demonstrates the same problem: it takes a long time to load the page and it eventually shows time out. Can it be that you're reading from cache?

https://isitdownorjust.me/sections-fooshed-net/
And it works also for Tiziano. Could it be some networking issue from your side? I'm running out of options :(

leila added a comment.Oct 12 2017, 4:14 PM

/facepalm @Pirroh for archive happiness: I was using the old link. sorry about that.