Page MenuHomePhabricator
Create Task
Maniphest T165153

arkivurl= should contain actual URL, not a Wikimedia-related test URL
Closed, ResolvedPublic
Actions

Description

Forwarded from:

The snippet:

arkivurl=https://web.archive.org/web/20131026082010/https://wikimediafoundation.org/wiki/Press_releases/Sue_Gardner_statement_paid_advocacy_editing

is repeatedly added to multiple articles. (This URL does not relate to the original content used in the citations):

Ideally the bot should remain disabled until:

  1. The bug has been identified
  2. The bug has been patched
  3. Clean-up: the introduced errors are found and repaired
  4. The new code has been deployed, and
  5. The new code has been carefully tested + reviewed in manual mode

Event Timeline

sladen created this task.May 12 2017, 12:15 PM
Restricted Application added a project: Internet-Archive. · View Herald TranscriptMay 12 2017, 12:15 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Cyberpower678 added a subscriber: Green_Cardamom.May 12 2017, 12:26 PM

Add Green who helps me with cleanup.

sladen updated the task description. (Show Details)May 12 2017, 12:34 PM
Green_Cardamom added a comment.May 12 2017, 1:44 PM

There are no other articles containing the link.

sladen updated the task description. (Show Details)May 12 2017, 7:41 PM
Cyberpower678 triaged this task as Medium priority.May 13 2017, 7:56 PM

This is an issue with the URL resolver, resolving it to "https:///". Since they are all seen as the same URL, it's inserting the same garbage URL into the sources. Since the problem is only limited to these articles, the bot can safely operate without fear of additional disruption. If it continues to make this errors, the run page for the bot can be found at https://en.wikipedia.org/wiki/User:InternetArchiveBot

Cyberpower678 moved this task from Inbox to v1.3 on the InternetArchiveBot board.May 13 2017, 7:56 PM
Cyberpower678 edited projects, added InternetArchiveBot (v1.3); removed InternetArchiveBot.
Cyberpower678 moved this task from Unsorted to Bugs on the InternetArchiveBot (v1.3) board.
Cyberpower678 added a comment.May 18 2017, 1:14 PM

I fixed what was causing the first diff to happen. It wasn't resolving the template's URL properly.

Cyberpower678 added a comment.EditedMay 18 2017, 1:17 PM

I will point out that the URL in the second diff is broken, not to be confused with dead. The URLs are improperly formatted, and not even a web browser can read them right. I'm just getting a blank tab when I click on it. They need to be fixed on wiki.

Cyberpower678 closed this task as Resolved.May 18 2017, 1:22 PM

I'm not going to fix the issue in the second diff. If a browser can't open it, then IABot can't be expected to handle it correctly either.

Cyberpower678 added a comment.May 18 2017, 1:23 PM

This issue is fixed in v1.3.2.

sladen added a comment.EditedMay 18 2017, 1:27 PM

@Cyberpower678: the bot does not need to fix broken stuff; the bot just needs to not to make it worse.

(FWIW, the second URL is double-escaped, and has repeated '://' at the front;

urllib.unquote(URL).replace('://://','://')

gives:

Please can the revision control (Git) link to the proposed fix be posted here?

Cyberpower678 added a comment.May 18 2017, 1:28 PM

I did however add a check to the routine to ignore instances where it can't read the URL correctly.

Cyberpower678 added a comment.May 18 2017, 1:37 PM
In T165153#3273396, @sladen wrote:

@Cyberpower678: the bot does not need to fix broken stuff; the bot just needs to not to make it worse.

(FWIW, the second URL is double-escaped, and has repeated '://' at the front;

urllib.unquote(URL).replace('://://','://')

gives:

Please can the revision control (Git) link to the proposed fix be posted here?

The git link isn't there yet as it's not committed yet, nor is it deployed.

https://sv.wikipedia.org/w/index.php?title=Studenthuset,_Stockholms_universitet&diff=39866435&oldid=39835828

Is an edit with the new code, and as you can see it ignored the badly formatted URLs.

sladen added a comment.May 18 2017, 3:46 PM

@Cyberpower678: The combined diff for "v1.3.2" is:

Which change is the "Added a sanity check to ignore URLs it can't process correctly." part?

Cyberpower678 added a comment.May 18 2017, 3:50 PM
In T165153#3273548, @sladen wrote:

@Cyberpower678: The combined diff for "v1.3.2" is:

Which change is the "Added a sanity check to ignore URLs it can't process correctly." part?

https://github.com/cyberpower678/Cyberbot_II/commit/258a365788db2e440ee9fdc313b6e942692a570f#diff-fba33131ff6de8f6a92bd61f8873e950R1248

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL