Create a tool to test the quality of section recommendations by collecting binary feedback from experienced editors as whether a section recommended for an article is a good/correct section to be recommended or not.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | leila | T171226 [Objective 9.1.3] Recommender system evaluation | |||
| Resolved | Pirroh | T174738 Make a tool to test the quality of section recommendations |
Event Timeline
Comment Actions
Michele Catasta built this tool and we tested in in Wikimania. The tool lived at http://40.118.110.204/ I'm checking with Michele to see if we can keep it up, or get some screenshots of it, at the very least. (Michele is not on phab, so I assigned this task to myself, which is not correct.:)
Comment Actions
@Pirroh can we have the tool up? If yes, I can claim victory on this task and move it to done. Having it up will also help us with further data collection that we should probably do in the coming weeks.
Comment Actions
The tool is now up and running here: http://sections.fooshed.net
We might want to circulate it around only after we update the recommendations.
Comment Actions
@Pirroh is the tool up? I can't access it. And while we're at it I'm wondering: this seems to be a kind of tool we need engineering help to maintain, expand, etc. Does it make sense to ask Baha to help with surfacing it somewhere on Wikimedia end?
Comment Actions
@leila: unfortunately the Azure VM where we are running the tool is down again (i.e., we had some issues with the free credits).
In case of no further hiccups, we should be up and running again by Monday.
Comment Actions
Re. engineering: as of now this is a very small hack, and the only reason why it went down is our bad luck with Azure.
I guess we can let Baha go full throttle on the extended GapFinder app -- we'll consider escalating this issue to engineering only if the tool will realistically get way more traffic than we expect now (i.e., just a few tens of editors).
Comment Actions
@Pirroh I want to put it in the quarterly review slide deck, but if people go to it and doesn't work, it's not cool. :D
Comment Actions
@Pirroh it's not loading for me again. Let's bring Baha in to help. I'll message off-thread.
Comment Actions
@leila: it's loading for me. Can you try a force refresh / anonymous browser session? The DNS record was modified again, so that might be the issue on your side.
Comment Actions
no. in incognito and with hard refresh it still demonstrates the same problem: it takes a long time to load the page and it eventually shows time out. Can it be that you're reading from cache?
Comment Actions
https://isitdownorjust.me/sections-fooshed-net/
And it works also for Tiziano. Could it be some networking issue from your side? I'm running out of options :(