Page MenuHomePhabricator
Create Task
Maniphest T184680

Update duplicate handling in reading lists API
Closed, ResolvedPublic
Actions

Description

  • prevent duplicate list names
    • stretch task: prevent duplicate list names in a case-insensitive manner
  • when duplicate lists or list entries are created, pretend to succeed but return the ID of the existing item and flag such IDs. (In case of list creation, still update the description.) List updates can fail as usual.
    • the response for creating new lists or items would change from { id: <id> } to { id: <id>, duplicate: true } when the requested new entry is "merged into" an existing one
    • for modifying lists, the current empty response would change to {}, and { id: <id>, duplicate: true } when there is a merge.

Details

SubjectRepoBranchLines +/-
Enforce list name uniqueness on the schema levelmediawiki/extensions/ReadingListsmaster+25 -14
Return merge flag when adding lists / entriesmediawiki/extensions/ReadingListswmf/1.31.0-wmf.23+96 -11
Update duplicate handlingmediawiki/extensions/ReadingListswmf/1.31.0-wmf.23+254 -97
Return merge flag when adding lists / entriesmediawiki/extensions/ReadingListsmaster+96 -11
Update duplicate handlingmediawiki/extensions/ReadingListsmaster+254 -97
Customize query in gerrit

Related Objects
Search...

Event Timeline

Tgr created this task.Jan 11 2018, 6:23 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 11 2018, 6:23 AM
Tgr claimed this task.Jan 11 2018, 6:23 AM
Tgr mentioned this in T182052: Batch reading list operations.
Tgr updated the task description. (Show Details)Jan 11 2018, 6:26 AM
Tgr updated the task description. (Show Details)Jan 17 2018, 6:01 PM
Tgr updated the task description. (Show Details)Jan 30 2018, 5:51 PM
• Fjalapeno mentioned this in T186041: Return full objects from POST and PUT endpoints for reading lists and reading list entries.Jan 30 2018, 7:42 PM
Tgr added a project: DBA.Jan 31 2018, 1:37 AM

This will involve creating a new unique index on reading_list.rl_name (no real data yet, just a few test rows). I imagine that does not require DBA involvement but asking just in case.

Also if we want name deduplication to be case-insensitive then either rl_name needs to be an utf8mb4 column (despite everything else being binary), or we need a separate rl_name_lowercase column that's computed by the PHP code (I suppose it's better to avoid SQL generated columns) and index that instead. Which of those options is better?

jcrespo subscribed.Feb 7 2018, 2:56 PM

We are currently experiencing a bit of a crisis with several emergencies on core systems, we may have some delay on attending features requests due to that- but site reliability is taking precedence-, I ask you to be patient, but contact us louder (through SOS or management) if that will block you a lot (but please only for unbreak now or similar).

Tgr moved this task from To Do to Doing on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Feb 8 2018, 11:17 PM
gerritbot subscribed.Feb 20 2018, 6:16 AM

Change 412844 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/ReadingLists@master] Update duplicate handling

https://gerrit.wikimedia.org/r/412844

gerritbot added a project: Patch-For-Review.Feb 20 2018, 6:16 AM
Tgr mentioned this in rERLSd6a71b256645: Update duplicate handling.Feb 20 2018, 6:22 AM
Tgr mentioned this in rERLScab6c9277eb1: Update duplicate handling.
Tgr moved this task from Doing to Code Review on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Feb 20 2018, 4:49 PM
Tgr mentioned this in rERLSbdf8b60e5f5a: Update duplicate handling.Feb 21 2018, 8:05 AM
Tgr mentioned this in T188048: Deploy ReadingLists schema change for efficient count(*) handling.Feb 22 2018, 9:44 PM
Tgr mentioned this in rERLSbb942ee5568e: Update duplicate handling.Feb 23 2018, 1:16 AM
Tgr mentioned this in T188120: Deploy ReadingLists schema change for unique list names.Feb 23 2018, 5:42 PM
Tgr moved this task from Code Review to Doing on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Feb 23 2018, 5:48 PM
gerritbot added a comment.Feb 27 2018, 1:59 AM

Change 414887 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/ReadingLists@master] Enforce list name uniqueness on the schema level

https://gerrit.wikimedia.org/r/414887

Tgr mentioned this in rERLSf91eecad009e: [WIP] Update duplicate handling.Feb 27 2018, 1:59 AM
Tgr mentioned this in rERLS49e20de6d1fd: Enforce list name uniqueness on the schema level.
Tgr mentioned this in rERLS1b548910c3fe: Enforce list name uniqueness on the schema level.Feb 28 2018, 7:34 PM
Tgr mentioned this in rERLSe8cd6abd8014: Update duplicate handling.Mar 5 2018, 1:09 AM
Tgr mentioned this in rERLS1ed6cc97a7c7: Enforce list name uniqueness on the schema level.Mar 5 2018, 1:15 AM
Tgr added a subtask: T188120: Deploy ReadingLists schema change for unique list names.Mar 5 2018, 2:10 AM
gerritbot added a comment.Mar 5 2018, 5:06 AM

Change 416378 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/ReadingLists@master] Return merge flag when adding lists / entries

https://gerrit.wikimedia.org/r/416378

Tgr mentioned this in rERLSb0bea81238cc: Update duplicate handling.Mar 5 2018, 5:07 AM
Tgr mentioned this in rERLS8b5342b408c2: Return merge flag when adding lists / entries.
Tgr mentioned this in rERLS56b0fc281925: Return merge flag when adding lists / entries.Mar 5 2018, 7:22 AM
Tgr mentioned this in rERLSee3ae73db095: Return merge flag when adding lists / entries.Mar 5 2018, 7:30 AM
Tgr moved this task from Doing to Code Review on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Mar 5 2018, 7:36 AM
• Mholloway mentioned this in rERLS3d076d9cb1ee: Update duplicate handling.Mar 5 2018, 7:37 PM
• Mholloway mentioned this in rERLS5b027fb9295c: Enforce list name uniqueness on the schema level.
• Mholloway mentioned this in rERLS603a2134b058: Return merge flag when adding lists / entries.Mar 5 2018, 7:40 PM
gerritbot added a comment.Mar 5 2018, 7:47 PM

Change 412844 merged by jenkins-bot:
[mediawiki/extensions/ReadingLists@master] Update duplicate handling

https://gerrit.wikimedia.org/r/412844

gerritbot added a comment.Mar 5 2018, 8:14 PM

Change 416378 merged by jenkins-bot:
[mediawiki/extensions/ReadingLists@master] Return merge flag when adding lists / entries

https://gerrit.wikimedia.org/r/416378

Tgr moved this task from Code Review to To Deploy on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Mar 5 2018, 9:59 PM
gerritbot added a comment.Mar 6 2018, 12:28 AM

Change 416611 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/ReadingLists@wmf/1.31.0-wmf.23] Update duplicate handling

https://gerrit.wikimedia.org/r/416611

gerritbot added a comment.Mar 6 2018, 12:28 AM

Change 416614 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/ReadingLists@wmf/1.31.0-wmf.23] Return merge flag when adding lists / entries

https://gerrit.wikimedia.org/r/416614

Tgr mentioned this in rERLSe9e847005e89: Update duplicate handling.Mar 6 2018, 12:30 AM
Tgr mentioned this in rERLSdc73b2862150: Return merge flag when adding lists / entries.
gerritbot added a comment.Mar 6 2018, 1:19 AM

Change 416611 merged by jenkins-bot:
[mediawiki/extensions/ReadingLists@wmf/1.31.0-wmf.23] Update duplicate handling

https://gerrit.wikimedia.org/r/416611

gerritbot added a comment.Mar 6 2018, 1:30 AM

Change 416614 merged by jenkins-bot:
[mediawiki/extensions/ReadingLists@wmf/1.31.0-wmf.23] Return merge flag when adding lists / entries

https://gerrit.wikimedia.org/r/416614

Tgr moved this task from To Deploy to Sign off on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Mar 6 2018, 2:23 AM
ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)).Mar 6 2018, 3:01 PM
jcrespo moved this task from Triage to Blocked external/Not db team on the DBA board.Mar 6 2018, 6:14 PM

Moving to blocked until we are pinged or subtask is ready for deploy.

Tgr moved this task from Sign off to To Do on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.Mar 10 2018, 12:55 AM
Tgr closed this task as Resolved.Apr 2 2018, 8:20 PM
Tgr closed subtask T188120: Deploy ReadingLists schema change for unique list names as Declined.

Let's call this done, probably fine without the schema change.

gerritbot added a comment.Apr 2 2018, 8:22 PM

Change 414887 abandoned by Gergő Tisza:
Enforce list name uniqueness on the schema level

Reason:
per task

https://gerrit.wikimedia.org/r/414887

• 238482n375 removed Tgr as the assignee of this task.Jun 15 2018, 8:03 AM
• 238482n375 triaged this task as Lowest priority.
• 238482n375 moved this task from Next Up to In Code Review on the Analytics-Kanban board.
• 238482n375 added projects: Analytics-Kanban, acl*security, Wikimedia-VE-Campaigns (S2-2018), Scap (Scap3-Adoption-Phase2), AbuseFilter, Data-release, Hashtags, LabsDB-Auditor, Ladies-That-FOSS-MediaWiki, Language-2018-Apr-June, Language-2018-Jan-Mar, HHVM, HAWelcome.
• 238482n375 edited subscribers, added: • 238482n375; removed: Aklapper.

SG9tZVBoYWJyaWNhdG9yCk5vIG1lc3NhZ2VzLiBObyBub3RpZmljYXRpb25zLgoKICAgIFNlYXJjaAoKQ3JlYXRlIFRhc2sKTWFuaXBoZXN0ClQxOTcyODEKRml4IGZhaWxpbmcgd2VicmVxdWVzdCBob3VycyAodXBsb2FkIGFuZCB0ZXh0IDIwMTgtMDYtMTQtMTEpCk9wZW4sIE5lZWRzIFRyaWFnZVB1YmxpYwoKICAgIEVkaXQgVGFzawogICAgRWRpdCBSZWxhdGVkIFRhc2tzLi4uCiAgICBFZGl0IFJlbGF0ZWQgT2JqZWN0cy4uLgogICAgUHJvdGVjdCBhcyBzZWN1cml0eSBpc3N1ZQoKICAgIE11dGUgTm90aWZpY2F0aW9ucwogICAgQXdhcmQgVG9rZW4KICAgIEZsYWcgRm9yIExhdGVyCgpUYWdzCgogICAgQW5hbHl0aWNzLUthbmJhbiAoSW4gUHJvZ3Jlc3MpCgpTdWJzY3JpYmVycwpBa2xhcHBlciwgSkFsbGVtYW5kb3UKQXNzaWduZWQgVG8KSkFsbGVtYW5kb3UKQXV0aG9yZWQgQnkKSkFsbGVtYW5kb3UsIEZyaSwgSnVuIDE1CkRlc2NyaXB0aW9uCgpPb3ppZSBqb2JzIGhhdmUgYmVlbiBmYWlsaW5nIGF0IGxlYXN0IGEgZmV3IHRpbWVzIGVhY2guIE1vcmUgaW52ZXN0aWdhdGlvbiBuZWVkZWQuCkpBbGxlbWFuZG91IGNyZWF0ZWQgdGhpcyB0YXNrLkZyaSwgSnVuIDE1LCA3OjIxIEFNCkhlcmFsZCBhZGRlZCBhIHN1YnNjcmliZXI6IEFrbGFwcGVyLiC3IFZpZXcgSGVyYWxkIFRyYW5zY3JpcHRGcmksIEp1biAxNSwgNzoyMSBBTQpKQWxsZW1hbmRvdSBjbGFpbWVkIHRoaXMgdGFzay5GcmksIEp1biAxNSwgNzoyMiBBTQpKQWxsZW1hbmRvdSB1cGRhdGVkIHRoZSB0YXNrIGRlc2NyaXB0aW9uLiAoU2hvdyBEZXRhaWxzKQpKQWxsZW1hbmRvdSBhZGRlZCBhIHByb2plY3Q6IEFuYWx5dGljcy1LYW5iYW4uCkpBbGxlbWFuZG91IG1vdmVkIHRoaXMgdGFzayBmcm9tIE5leHQgVXAgdG8gSW4gUHJvZ3Jlc3Mgb24gdGhlIEFuYWx5dGljcy1LYW5iYW4gYm9hcmQuCkNoYW5nZSBTdWJzY3JpYmVycwpDaGFuZ2UgUHJpb3JpdHkKQXNzaWduIC8gQ2xhaW0KTW92ZSBvbiBXb3JrYm9hcmQKQ2hhbmdlIFByb2plY3QgVGFncwpBbmFseXRpY3MtS2FuYmFuCtcKU2VjdXJpdHkK1wpXaWtpbWVkaWEtVkUtQ2FtcGFpZ25zIChTMi0yMDE4KQrXClNjYXAK1wpTY2FwIChTY2FwMy1BZG9wdGlvbi1QaGFzZTIpCtcKQWJ1c2VGaWx0ZXIK1wpEYXRhLXJlbGVhc2UK1wpIYXNodGFncwrXCkxhYnNEQi1BdWRpdG9yCtcKTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2kK1wpMYW5ndWFnZS0yMDE4LUFwci1KdW5lCtcKTGFuZ3VhZ2UtMjAxOC1KYW4tTWFyCtcKSEhWTQrXCkhBV2VsY29tZQrXCkJvbGQKSXRhbGljcwpNb25vc3BhY2VkCkxpbmsKQnVsbGV0ZWQgTGlzdApOdW1iZXJlZCBMaXN0CkNvZGUgQmxvY2sKUXVvdGUKVGFibGUKVXBsb2FkIEZpbGUKTWVtZQpQcmV2aWV3CkhlbHAKRnVsbHNjcmVlbiBNb2RlClBpbiBGb3JtIE9uIFNjcmVlbgoyMzg0ODJuMzc1IGFkZGVkIHByb2plY3RzOiBTZWN1cml0eSwgV2lraW1lZGlhLVZFLUNhbXBhaWducyAoUzItMjAxOCksIFNjYXAgKFNjYXAzLUFkb3B0aW9uLVBoYXNlMiksIEFidXNlRmlsdGVyLCBEYXRhLXJlbGVhc2UsIEhhc2h0YWdzLCBMYWJzREItQXVkaXRvciwgTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2ksIExhbmd1YWdlLTIwMTgtQXByLUp1bmUsIExhbmd1YWdlLTIwMTgtSmFuLU1hciwgSEhWTSwgSEFXZWxjb21lLlBSRVZJRVcKMjM4NDgybjM3NSBtb3ZlZCB0aGlzIHRhc2sgZnJvbSBJbiBQcm9ncmVzcyB0byBJbiBDb2RlIFJldmlldyBvbiB0aGUgQW5hbHl0aWNzLUthbmJhbiBib2FyZC4KMjM4NDgybjM3NSByZW1vdmVkIEpBbGxlbWFuZG91IGFzIHRoZSBhc3NpZ25lZSBvZiB0aGlzIHRhc2suCjIzODQ4Mm4zNzUgdHJpYWdlZCB0aGlzIHRhc2sgYXMgTG93ZXN0IHByaW9yaXR5LgoyMzg0ODJuMzc1IHJlbW92ZWQgc3Vic2NyaWJlcnM6IEFrbGFwcGVyLCBKQWxsZW1hbmRvdS4KQ29udGVudCBsaWNlbnNlZCB1bmRlciBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLVNoYXJlQWxpa2UgMy4wIChDQy1CWS1TQSkgdW5sZXNzIG90aGVyd2lzZSBub3RlZDsgY29kZSBsaWNlbnNlZCB1bmRlciBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSAoR1BMKSBvciBvdGhlciBvcGVuIHNvdXJjZSBsaWNlbnNlcy4gQnkgdXNpbmcgdGhpcyBzaXRlLCB5b3UgYWdyZWUgdG8gdGhlIFRlcm1zIG9mIFVzZSwgUHJpdmFjeSBQb2xpY3ksIGFuZCBDb2RlIG9mIENvbmR1Y3QuILcgV2lraW1lZGlhIEZvdW5kYXRpb24gtyBQcml2YWN5IFBvbGljeSC3IENvZGUgb2YgQ29uZHVjdCC3IFRlcm1zIG9mIFVzZSC3IERpc2NsYWltZXIgtyBDQy1CWS1TQSC3IEdQTApZb3VyIGJyb3dzZXIgdGltZXpvbmUgc2V0dGluZyBkaWZmZXJzIGZyb20gdGhlIHRpbWV6b25lIHNldHRpbmcgaW4geW91ciBwcm9maWxlLCBjbGljayB0byByZWNvbmNpbGUu

Aklapper raised the priority of this task from Lowest to Needs Triage.Jun 15 2018, 12:44 PM
Aklapper removed projects: HAWelcome, HHVM, Language-2018-Jan-Mar, Language-2018-Apr-June, Ladies-That-FOSS-MediaWiki, LabsDB-Auditor, Hashtags, Data-release, AbuseFilter, Scap (Scap3-Adoption-Phase2), Wikimedia-VE-Campaigns (S2-2018), acl*security, Analytics-Kanban.
Aklapper edited subscribers, added: Aklapper; removed: • 238482n375.
Aklapper assigned this task to Tgr.Jun 15 2018, 12:47 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits