Page MenuHomePhabricator

[AOI] Write browser automation script to detect which wikis HotCat is broken on
Closed, ResolvedPublic

Description

Write an automated browser test so we can generate a list of wikis where HotCat is broken.

Probably the easiest way to do this is to write a Cucumber script (https://cucumber.io/) for Selenium (an automated browser driver) and run the test on your local machine. The script would go to a given wiki and do the following:

  • Log in as Community Tech bot (or any user that doesn't care about their gadget preferences being changed)
  • Load https://xxx.xxxxxx.org/wiki/Special:Preferences#mw-prefsection-gadgets
  • Look for a checkbox input with id "mw-input-wpgadgets-HotCat" (I've already confirmed that the id is the same on the vast majority of wikis with HotCat)
  • If the checkbox exists, check it, and click Save
  • Go to Special:Random
  • See if there is any element of class "hotcatlink". If so, output that wiki as working, if not, output that wiki as broken.

You can just have it write the output to your console or a text file.

Event Timeline

DannyH created this task.Sep 25 2015, 6:05 PM
DannyH updated the task description. (Show Details)
DannyH raised the priority of this task from to High.
DannyH added a project: Community-Tech-Sprint.
DannyH moved this task to Ready on the Community-Tech-Sprint board.
DannyH added a subscriber: DannyH.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 25 2015, 6:05 PM
kaldari updated the task description. (Show Details)Sep 25 2015, 10:05 PM
kaldari renamed this task from Write browser automation script to detect which wikis HotCat is broken on to [AOI] Write browser automation script to detect which wikis HotCat is broken on.
kaldari updated the task description. (Show Details)Sep 25 2015, 10:07 PM
kaldari edited a custom field.Sep 29 2015, 5:38 PM

@NiharikaKohli: I checked in some stuff to the community-tech-tools repo that should give you a head-start on this: https://github.com/wikimedia/community-tech-tools/tree/master/hotcatfix. I'd love to go through all of it with you tomorrow if you have a chance and explain what is what, and what else needs to be built (and how to run the tests).

Posted the results on https://github.com/wikimedia/community-tech-tools/tree/master/hotcatfix along with a README for easy navigation. :)

@NiharikaKohli: It looks like there are some false positives due to the fact that hits.txt includes some wikis that don't actually have the HotCat gadget, but have some variation of it like HotCatMulti. HotCatMulti seems to be a significantly different gadget and it fails our browser test even if it's working perfectly well, like on French Wikipedia (as it uses different class names for the UI components). I would suggest limiting hits.txt to only wikis that have more than 1000 bytes of code at MediaWiki:Gadget-HotCat.js or MediaWiki:Gadget-hotcat.js, but not other variations like MediaWiki:Gadget-LiveRC.js or MediaWiki:Gadget-HotCatsMulti.js. We need to be conservative on what wikis we flag for automated fixing. I would rather miss a few broken ones than accidentally overwrite some locally customized versions that are running fine.

Actually, it looks like we can determine which ones don't actually have HotCat available based on the fact that they fail the browser test early on. So if we just take the wikis that fail with "1 failed, 8 passed", those should be safe to assume that they have HotCat, but it's broken.

kaldari closed this task as Resolved.Oct 8 2015, 7:22 PM

Final list at P2167.

DannyH edited a custom field.