Page MenuHomePhabricator
Create Task
Maniphest T403209

SPIKE: Investigate why linksearchtotal_collect.py is not working
Closed, ResolvedPublic
Actions

Description

python manage.py linksearchtotal_collect

has not run since Aug. 18, 2025, and when I manually try to run it, it just hangs.

This spike is to investigate why and fix it so that we can run the command again.

Details

Other Assignee
• Kgraessle

Event Timeline

• Kgraessle created this task.Aug 28 2025, 6:55 PM
• Kgraessle assigned this task to jsn.sherman.
jsn.sherman changed the task status from Open to In Progress.EditedAug 29 2025, 7:35 PM
jsn.sherman moved this task from Ready (Maintenance) to QA on the Moderator-Tools-Team (Kanban) board.

hotfix pushed:
https://github.com/WikipediaLibrary/externallinks/commit/5391e1bcb3f6f3e9a33623886060f0af57a309a9
I tested this directly in production, so it's known working. It looks like we're currently caught up. I reverted the server-side change upon committing my work, so we're waiting on the deployment for it to be in for real. Note that the hotfix has print statements to allow CLI debugging; we'll replace with proper logging before closing

jsn.sherman added a comment.Aug 29 2025, 8:23 PM

and we're running the hotfix now; I have a small pile of notes for followup, but let's leave this in qa for now while we see what happens on the next scheduled run (9/1)

• Kgraessle added a comment.Aug 29 2025, 8:34 PM

Sounds good; thanks for getting this fixed!

• Kgraessle added a comment.Aug 29 2025, 8:35 PM

Was the query just running too long?

jsn.sherman added a comment.Aug 29 2025, 8:38 PM

Yep, and I'll have some follow-up to make this more robust.

jsn.sherman moved this task from QA to In Progress on the Moderator-Tools-Team (Kanban) board.Sep 2 2025, 11:57 AM

I verified that we have linksearch totals for 9/1, so I'll go ahead and flesh this out the rest of the way.

jsn.sherman moved this task from In Progress to Eng review on the Moderator-Tools-Team (Kanban) board.Sep 2 2025, 8:17 PM

followup patch:
https://github.com/WikipediaLibrary/externallinks/pull/467

• Scardenasmolinar moved this task from Eng review to QA on the Moderator-Tools-Team (Kanban) board.Sep 2 2025, 10:09 PM
• Scardenasmolinar subscribed.

Merged! We can QA this ticket at our engineering meeting tomorrow.

• Kgraessle closed this task as Resolved.Sep 4 2025, 6:14 PM
• Kgraessle moved this task from QA to Done on the Moderator-Tools-Team (Kanban) board.
jsn.sherman reopened this task as In Progress.Sep 12 2025, 4:02 PM
jsn.sherman moved this task from Done to Eng review on the Moderator-Tools-Team (Kanban) board.

updated pr here:
https://github.com/WikipediaLibrary/externallinks/pull/470

jsn.sherman moved this task from Eng review to In Progress on the Moderator-Tools-Team (Kanban) board.Sep 12 2025, 5:03 PM

note that I'll need to update retry/error handling a bit for swift uploads

• Kgraessle updated Other Assignee, added: • Kgraessle.Sep 12 2025, 5:11 PM
• Kgraessle moved this task from In Progress to Eng review on the Moderator-Tools-Team (Kanban) board.Sep 12 2025, 7:03 PM
Samwalton9-WMF moved this task from Eng review to QA on the Moderator-Tools-Team (Kanban) board.Sep 16 2025, 4:02 PM
• Scardenasmolinar closed this task as Resolved.Sep 17 2025, 6:22 PM
• Scardenasmolinar moved this task from QA to Done on the Moderator-Tools-Team (Kanban) board.

Checked the admin, and we can see results from the 15th.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits