Page MenuHomePhabricator
Create Task
Maniphest T178079

Implement a (conservative) usage aspect deduplicater (days: 4)
Closed, ResolvedPublic
Actions

Description

While we for now probably don't want to deduplicate all redundant usages, we should have a usage deduplicater that removes specified usages of a type if there are unspecified usages of the same type.

This would look like this (left side is input, right side is output)

  • L.de, L → L
  • D.de, D, L, C.P12 → D, L, C.P12

As said above, other redundancies should not be tackled:

  • X, D.de → X, D.de

Patch-For-Review:

Details

SubjectRepoBranchLines +/-
Rearrange UsageDeduplicator implementationmediawiki/extensions/Wikibasemaster+39 -45
Fix smaller mistakes in UsageDeduplicatormediawiki/extensions/Wikibasemaster+27 -12
Deduplicate entity usages when returning from ParserOutputUsageAccumulatormediawiki/extensions/Wikibasemaster+156 -1
Customize query in gerrit

Related Objects
Search...

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
· · ·
ResolvedLydia_PintscherT49288 [Epic] Track Wikidata entity usage on client pages
ResolvedLadsgroupT178079 Implement a (conservative) usage aspect deduplicater (days: 4)
· · ·

Event Timeline

hoo created this task.Oct 12 2017, 3:25 PM
daniel added a comment.Oct 12 2017, 3:42 PM

Perhaps this can be implemented directly in the UsageAggregator.

eranroz mentioned this in T178114: Many Wikipedia's Wikidata module iterate over all entity claims if a Statement is searched for by property label.Oct 13 2017, 8:20 AM
eranroz mentioned this in T184322: Enable fine grained lua tracking gradually in client wikis.Jan 7 2018, 6:02 PM
Ladsgroup moved this task from incoming to consider for next sprint on the Wikidata board.Jan 10 2018, 9:08 AM
Ladsgroup claimed this task.Jan 16 2018, 8:07 PM
Ladsgroup added a project: Wikidata-Sprint-2018-01-17.
Ladsgroup moved this task from consider for next sprint to in progress on the Wikidata board.
Ladsgroup subscribed.

I do it.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJan 16 2018, 8:07 PM
WMDE-leszek triaged this task as Medium priority.Jan 17 2018, 11:33 AM
gerritbot subscribed.Jan 18 2018, 4:39 PM

Change 405013 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/extensions/Wikibase@master] Deduplicate entity usages when returning from ParserOutputUsageAccumulator

https://gerrit.wikimedia.org/r/405013

gerritbot added a project: Patch-For-Review.Jan 18 2018, 4:39 PM
Ladsgroup moved this task from Backlog to In Progress on the Wikidata-Sprint-2018-01-17 board.Jan 19 2018, 9:25 AM
Ladsgroup moved this task from In Progress to Review on the Wikidata-Sprint-2018-01-17 board.Jan 19 2018, 10:13 AM
Ladsgroup moved this task from Incoming to Blocked on others on the User-Ladsgroup board.Jan 20 2018, 5:35 PM
gerritbot added a comment.Jan 23 2018, 4:13 PM

Change 405895 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/Wikibase@master] Fix smaller mistakes in UsageDeduplicator

https://gerrit.wikimedia.org/r/405895

gerritbot added a comment.Jan 23 2018, 4:13 PM

Change 405896 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/Wikibase@master] Rearrange UsageDeduplicator implementation

https://gerrit.wikimedia.org/r/405896

gerritbot added a comment.Jan 23 2018, 4:30 PM

Change 405013 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Deduplicate entity usages when returning from ParserOutputUsageAccumulator

https://gerrit.wikimedia.org/r/405013

WMDE-leszek renamed this task from Implement a (conservative) usage aspect deduplicater to Implement a (conservative) usage aspect deduplicater (days: 3).Jan 23 2018, 4:52 PM
ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-02-06 (1.31.0-wmf.20)).Jan 23 2018, 5:00 PM
gerritbot added a comment.Jan 24 2018, 1:23 PM

Change 405895 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fix smaller mistakes in UsageDeduplicator

https://gerrit.wikimedia.org/r/405895

WMDE-leszek renamed this task from Implement a (conservative) usage aspect deduplicater (days: 3) to Implement a (conservative) usage aspect deduplicater (days: 4).Jan 24 2018, 3:48 PM
gerritbot added a comment.Jan 24 2018, 9:54 PM

Change 405896 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Rearrange UsageDeduplicator implementation

https://gerrit.wikimedia.org/r/405896

thiemowmde closed this task as Resolved.Jan 25 2018, 12:36 PM
thiemowmde removed a project: Patch-For-Review.
thiemowmde updated the task description. (Show Details)
thiemowmde moved this task from Review to Done on the Wikidata-Sprint-2018-01-17 board.
thiemowmde updated the task description. (Show Details)
Ladsgroup mentioned this in T185693: Implement a (more liberal) usage aspect deduplicater (days: 3).Jan 25 2018, 12:55 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL