Page MenuHomePhabricator
Create Task
Maniphest T314703

Structured data for deleted files on Commons still visible in SPARQL engine after deletion
Closed, ResolvedPublic5 Estimated Story PointsBUG REPORT
Actions

Description

Steps to replicate the issue (include links if applicable):

What happens?:
It returns data that shouldn't be visible anymore

What should have happened instead?: The query shouldn't return anything at all

Pointed out at https://commons.wikimedia.org/w/index.php?title=Commons:Village_pump&oldid=680579554#Structured_data_on_deleted_images . Tagged it with privacy too because this has the potential of exposing deleted data.

Details

SubjectRepoBranchLines +/-
wcqs: enable proper URI schemes for commonsoperations/puppetproduction+17 -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

Multichill created this task.Aug 6 2022, 11:07 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 6 2022, 11:07 AM
Maintenance_bot added a project: Wikidata.Aug 6 2022, 11:29 AM
HenkvD subscribed.Aug 6 2022, 11:44 AM
Bugreporter subscribed.Aug 6 2022, 6:44 PM

Compare T302189: Regularly purge orphaned sitelink, value and reference nodes

• JFishback_WMF added a project: Privacy Engineering.Aug 9 2022, 4:47 PM
• JFishback_WMF moved this task from Incoming to Watching on the Privacy Engineering board.
RP88 subscribed.Aug 10 2022, 10:33 AM
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.Aug 10 2022, 10:39 AM
• MPhamWMF moved this task from Incoming to Current work on the Wikidata-Query-Service board.Aug 15 2022, 3:30 PM
• MPhamWMF added a project: Discovery-Search (Current work).
• MPhamWMF set the point value for this task to 5.Aug 22 2022, 3:32 PM
• MPhamWMF moved this task from Incoming to Ready for Dev -- SWE on the Discovery-Search (Current work) board.
dcausse claimed this task.Aug 25 2022, 2:18 PM
dcausse moved this task from Ready for Dev -- SWE to In Progress on the Discovery-Search (Current work) board.
gerritbot added a comment.Aug 25 2022, 2:59 PM

Change 826589 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/puppet@production] wcqs: enable proper URI schemes for commons

https://gerrit.wikimedia.org/r/826589

gerritbot added a project: Patch-For-Review.Aug 25 2022, 2:59 PM
dcausse mentioned this in T316236: Reload WCQS from dumps.Aug 25 2022, 3:51 PM
dcausse added a subtask: T316236: Reload WCQS from dumps.Aug 25 2022, 3:54 PM
gerritbot added a comment.Aug 25 2022, 4:32 PM

Change 826589 merged by Bking:

[operations/puppet@production] wcqs: enable proper URI schemes for commons

https://gerrit.wikimedia.org/r/826589

Maintenance_bot removed a project: Patch-For-Review.Aug 25 2022, 5:30 PM
Bugreporter added a comment.EditedAug 26 2022, 4:53 PM

Note suppress delete should need special handling. Compare T105427: Need a way for WDQS updater to become aware of suppressed deletes / T256949: The streaming updater should support suppressed deletes

dcausse added a comment.Aug 29 2022, 6:59 AM
In T314703#8189526, @Bugreporter wrote:

Note suppress delete should need special handling. Compare T105427: Need a way for WDQS updater to become aware of suppressed deletes / T256949: The streaming updater should support suppressed deletes

WCQS is using the streaming updater so barring any regression this type of delete should be handled.

dcausse mentioned this in T316496: WCQS does not report proper lag information.Aug 29 2022, 7:25 AM
dcausse moved this task from In Progress to Waiting on the Discovery-Search (Current work) board.Aug 29 2022, 3:10 PM
Gehel subscribed.Sep 5 2022, 3:10 PM

The root cause is fixed, but we need a full data reload to actually fix this issue. We'll close the ticket once this is done.

Gehel closed subtask T316496: WCQS does not report proper lag information as Resolved.Sep 26 2022, 9:53 AM
HenkvD added a comment.Oct 26 2022, 2:19 PM

This particular example is now OK, after T316236 "Reload WCQS from dumps has performed" has updated data on several hosts, last Tuesday, Oct 25

• MPhamWMF moved this task from Waiting to Blocked/Waiting on the Discovery-Search (Current work) board.Oct 31 2022, 5:08 PM
bking closed subtask T316236: Reload WCQS from dumps as Resolved.Nov 18 2022, 3:42 PM
Gehel moved this task from Blocked/Waiting to Needs Reporting on the Discovery-Search (Current work) board.Nov 28 2022, 4:18 PM
Gehel closed this task as Resolved.Dec 2 2022, 2:08 PM

Data reload completed (T316236) and confirmation that those specific entries have been fixed.

RKemper reopened subtask T316236: Reload WCQS from dumps as Open.Dec 6 2022, 8:47 PM
Gehel closed subtask T316236: Reload WCQS from dumps as Resolved.Dec 9 2022, 4:00 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits