Page MenuHomePhabricator

Adjust UploadWizard API test suite to match commons.wikimedia.org policy for bots
Closed, DeclinedPublic

Description

The UploadWizard monitoring job on commons has been falling because the 'Selenium user' has been blocked.

We should:

  • use {{subst:User:Selenium user/Filedesc}} on the filedescription page (it will subst the template for test uplads)
  • rename the account to a meaningful name such as User:Selenium user (WMF QA bot) or `User:UploadWizardSmokeTest (WMF QA)
  • Edit the user page to explain what the account is used for

Reference: https://commons.wikimedia.org/w/index.php?title=Commons:Administrators%27_noticeboard&oldid=175658630#Unblocking_User:Selenium_user

Event Timeline

hashar raised the priority of this task from to Medium.
hashar updated the task description. (Show Details)
hashar added subscribers: Krenair, Matanya, Multichill and 3 others.
hashar added subscribers: zeljkofilipin, dduvall.

rename the account to [[User:Selenium user (WMF QA bot)]]

I can rename the account, let me know when you are reedy :)

Some thoughts

  • https://commons.wikimedia.org/w/index.php?title=File:Test-image-rosa-mx-15x15.png&action=edit&redlink=1 has over 5000 deleted revisions.
  • I had to nuke many many test files some so old I couldn't use Special:Nuke and had to use a bot
  • I'm not very happy with a bot uploading test junk and having our users waste time on cleaning it up.
  • You shouldn't have been running an unauthorized bot in the first place
  • Isn't it possible to upload without actually completing the process? If I upload using the upload wizard, I first upload and after that fill out info. In the meantime the file is stashed. Isn't that enough for testing?
  • Second best is that you clean up your own junk
  • I don't care how to user is named, I'm fine with the current name
  • The user page should contain a very clear explanation what the account is doing with some pointers and links to one or more operators.
  • Do you have more bots like this running around?

The smoke test suite has been made as part of T60555: Create API Upload Wizard Smoke Tests. UploadWizard was then unstable either because of the extension code or Wikimedia backend infrastructure (Thumbnailling system, Swift?). Since as I understand it it is an important part of Commons, it makes sense to closely monitor it by running a test that exercise a good part of the stack.

I have not been personally involved beside the Jenkins integration. I can imagine that people reached out to commons originally or maybe it has been missed. @Steinsplitter suggest some enhancements that are the subject of this task (rename the account, add a specific text to the file description).

For what it is worth, the code is in mediawiki/extensions/UploadWizard.git under /tests/api.


At first, thank you @Multichill to have blocked the user. That has let us identify which script was causing the spam on Commons and now we can work on improving it :-}

Responding point by point with slight reordering:

https://commons.wikimedia.org/w/index.php?title=File:Test-image-rosa-mx-15x15.png&action=edit&redlink=1 has over 5000 deleted revisions.
I had to nuke many many test files some so old I couldn't use Special:Nuke and had to use a bot
Second best is that you clean up your own junk

Ideally the script would delete the file by itself, but I don't think MediaWiki can grant delete right to the user for a single page :-/

I'm not very happy with a bot uploading test junk and having our users waste time on cleaning it up.
You shouldn't have been running an unauthorized bot in the first place

Sorry, I have nothing much to say there, I am assuming good faith think developers/UploadWizard team did reach out to Commons community when the job was first deployed (aka 2 years ago?).

If you could describe the task description and add a checkbox step to get the bot formally authorized, I am willing to fill in the application process and clarify whatever is needed.

Isn't it possible to upload without actually completing the process? If I upload using the upload wizard, I first upload and after that fill out info. In the meantime the file is stashed. Isn't that enough for testing?

The job validates API calls used by UploadWizard for the whole workflow. Looking at the code, it does test that the filekey and token from stash can successfully be used to publish the file then assert the file is actually corresponding to what we sent. That validate the whole system (including the file and cache backends).

I don't care how to user is named, I'm fine with the current name
The user page should contain a very clear explanation what the account is doing with some pointers and links to one or more operators.

Both points are the subject of this task. If we get the user clearly identified that will surely help. Maybe we can come up with UploadWizardSmokeTest (WMF). Editing the user page is definitely a good idea.

Do you have more bots like this running around?

Beside the browser tests which should have very limited write activity (if at all), I am only aware of the UploadWizard smoke test.

Do you have more bots like this running around?

Beside the browser tests which should have very limited write activity (if at all), I am only aware of the UploadWizard smoke test.

See also:

From it's userpage:
"This account is used to ensure timely metrics for the WMF Operations staff when those metrics require physical actions on the wiki (edit save speed, etc). It will not edit outside it's own user space."

Steinsplitter raised the priority of this task from Medium to High.Oct 15 2015, 10:11 AM

Please fix this ASAP: The Sellenium account is editing again.

T115370 is about this problem too.

The bug seems about beta stuff. Likely production stuff (where this bug is about) should be fixed first. If users complaining again it is likely that the bot gets blocked again (and i won't unblock it a second time).
The main problem seems to be filedescription page and the name (for what i have seen - per the community complains on AN and VP).

hashar claimed this task.

The job has been deleted following discussion T115465. In effect, the UploadWizard smoke test no more hit commons.wikimedia.org.

Hence there is nothing to tweak.