Page MenuHomePhabricator

[BUG?] FileImporter does not mark source files with NowCommons even though I selected it
Closed, ResolvedPublic2 Estimated Story PointsBUG REPORT

Description

I just made these three imports from enwp to commons. I selected marking source files but it's not working.

https://commons.wikimedia.org/wiki/File:Arbor_Lodge_2008_3.jpg ‎ Imported with FileImporter from https://en.wikipedia.org/wiki/File:Wiki_photos_092.jpg
https://commons.wikimedia.org/wiki/File:Arbor_Lodge_2008_2.jpg ‎ Imported with FileImporter from https://en.wikipedia.org/wiki/File:Wiki_photos_091.jpg
https://commons.wikimedia.org/wiki/File:Arbor_Lodge_2008.jpg Imported with FileImporter from https://en.wikipedia.org/wiki/File:Wiki_photos_090.jpg

(I confirm I am logged in on enwp and I could edit.)

Event Timeline

@Roy17 Yes, I noticed this too. The same thing happened with File:Anvil_Publish_logo.svg and other files. I remember it used to work well. Probably a Regression.

Masumrezarock100 changed the subtype of this task from "Task" to "Bug Report".Oct 19 2019, 8:19 PM

A few days ago it went back to normal, but now it's not functioning again.

It was functioning yesterday (22 Oct), but as of 17:18, 23 October 2019, it's not.

Some kind of seasonal bug???

Masumrezarock100 triaged this task as Unbreak Now! priority.Oct 24 2019, 11:29 AM

Ah, seeing this bug's nature, marking this as Unbreak now! for now.

thiemowmde lowered the priority of this task from Unbreak Now! to Needs Triage.Oct 24 2019, 2:06 PM
thiemowmde added a project: WMDE-TechWish.
thiemowmde subscribed.

The "unbreak now" priority is for issues that would cause unfixable data loss otherwise, and should stop people from doing anything else. This is certainly not the case here. Adding the NowCommons template is an optional convenience feature. Nothing gets lost if it doesn't work.

I'm not saying this isn't a relevant bug – it clearly is. We will fix it – if we understand what causes it. At the moment it appears this won't be easy. We might need to add more logging first.

For later reference:

It was working at 18:22, 30 October 2019.
Right now at 10:32, 1 November 2019‎, it's not working again.

It was working at 18:22, 30 October 2019.
Right now at 10:32, 1 November 2019‎, it's not working again.

It seems so. I was able to reproduce it today. See File:Downtownstatesborocourthouse.jpg on English Wikipedia. @thiemowmde how weird and unusual this bug can be! You guys really need to fix this ASAP.

And the same thing happened with https://en.wikipedia.org/wiki/File:Dowth2014_1.JPG . It didn't mark it with NowCommons. I can export some more files to Commons to test it if you want.

Lena_WMDE set the point value for this task to 2.

Check log of errors that we log now.

Error messages to look for:

  • "user can't use CentralAuth"
  • "Remote API responded with an error"
  • "failed to do post import edit"
  • "failed to do post import delete"

I played around with https://logstash.wikimedia.org and found:

Looking at the log messages surrounding these errors:

  • The failed delete is because of failed to decode response from https://en.wikisource.org/w/api.php?centralauthtoken=…. Hard to tell. Unfortunately we don't log the full API request here, but I don't think this would tell us much.
  • The error at 2020-06-09T02:17 is: Failed to get centralauthtoken: Failed to discover API location from: https://en.wikipedia.org/wiki/File:Angus_the_bulldog.jpg. HTTP status code 0. Error fetching URL: Failed to connect to url-downloader.eqiad.wikimedia.org port 8080: Connection refused. Looks like a temporary network error or something.
  • All others are the same abusefilter-disallowed: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do. A brief description of the abuse rule which your action matched is: anti abuso bot.

My suggestions:

  • Check if we show the expected yellow warning message about the failed remote edit in the AbuseFilter situation. (I believe we do.)
  • We currently don't forward the AbuseFilter error message, and don't display it in the yellow warning box. It might be worth changing this.
  • We should create a new sub-ticket of T223288: [Epic] Better handling of abuse filter + FileImporter interaction for this. Note this is not the same as T253872. The later is about filters on Commons, while this investigation here is about filters on the source wiki.

The investigation looks good and I agree with the next steps. I'd also suggest that someone try to adjust the anti abuso bot AbuseFilter rule to allow this specific edit. Moving to "Demo" to get product owner review.

Why is the option of automatically marking source file with nowcommons removed? Now every single file has to be tagged manually? Is that related to this?

@Roy17, this is a temporary bug, see T256428. We hope to get it fixed on Monday.

awight claimed this task.

Should be fixed, please re-open if it happens again. Thanks for the report!