Page MenuHomePhabricator
Create Task
Maniphest T184322

Enable fine grained lua tracking gradually in client wikis
Closed, ResolvedPublic
Actions

Description

The plan is to enable it for hewiki first, then other Wikis that have fineGrainedUsageTracking enabled (cawiki, trwiki, elwiki, etc.) and then gradually move forward from there.

Details

SubjectRepoBranchLines +/-
Enable x-kill feature everywhereoperations/mediawiki-configmaster+1 -31
Enable lua fine grained usage tracking in cawiki, cewiki, elwiki, kowiki, trwikioperations/mediawiki-configmaster+5 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

Ladsgroup triaged this task as High priority.Jan 6 2018, 1:02 AM
Ladsgroup created this task.
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJan 6 2018, 1:02 AM
hoo added a comment.Jan 6 2018, 1:02 PM

The description is a bit unclear: Shall it be enabled for hewiki only initially (and if so for how long) or hewiki + cawiki, trwiki, …?

daniel added a subscriber: Joe.Jan 6 2018, 5:39 PM
eranroz added a comment.Jan 7 2018, 6:02 PM

This is likely to have tiny aspects on hewiki, as many modules are already adopted to avoid getEntity() but rather use entity ID and fetch only the specific aspect (this have additional benefit of lower footprint of serializations).
Hence wbc_entity_usage is probably not going to change drastically, nor recentchanges (where rc_source=wb), and hewiki may be a good starting point. This is mainly to validate this setting works well and doesn't break anything.

As for other wikis:

  • cawiki could be the next one, where we can evaluate how it affects the wbc_entity_usage table size and recentchanges
  • for wider deployment, I believe https://phabricator.wikimedia.org/T178114 should be considered as a blocker. (possibly requires something similar to T178079)
eranroz added a comment.Jan 7 2018, 6:11 PM

One more point for consideration: fineGrainedUsageTracking can be deployed with or without tracking specific claims (T151717), where we mask C.Px to O in SqlUsageTracker::handleBlacklistedUsages. AFAIK cawiki already tracks specific claims.

IKhitron subscribed.Jan 8 2018, 9:56 PM
Ladsgroup updated the task description. (Show Details)Jan 10 2018, 6:09 AM
Ladsgroup mentioned this in T172914: [Tracking] Fine-grained change notifications based on tracking from Lua getters via __index.
Ladsgroup moved this task from Incoming to In progress on the User-Ladsgroup board.Jan 12 2018, 12:15 PM
gerritbot added a comment.Jan 15 2018, 1:59 PM

Change 404298 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Enable lua fine grained usage tracking in cawiki, cewiki, elwiki, kowiki, trwiki

https://gerrit.wikimedia.org/r/404298

gerritbot added a project: Patch-For-Review.Jan 15 2018, 1:59 PM
gerritbot added a comment.Jan 15 2018, 2:05 PM

Change 404298 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable lua fine grained usage tracking in cawiki, cewiki, elwiki, kowiki, trwiki

https://gerrit.wikimedia.org/r/404298

Stashbot added a comment.Jan 15 2018, 2:08 PM

Mentioned in SAL (#wikimedia-operations) [2018-01-15T14:08:53Z] <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:404298|Enable lua fine grained usage tracking in some wikis (T184322)]] (duration: 01m 14s)

Ladsgroup added a comment.Jan 15 2018, 2:14 PM

I made this to check in a few days: P6587

IKhitron mentioned this in T184985: Wikidata changes on local wikis do not coordinate well between css classes and bold for unseen changes.Jan 16 2018, 11:01 AM
Lydia_Pintscher closed subtask T185032: Enable lua fine grained usage tracking - Late January 2018 batch as Resolved.Jan 30 2018, 3:04 PM
matej_suchanek changed the status of subtask T184985: Wikidata changes on local wikis do not coordinate well between css classes and bold for unseen changes from Open to Stalled.Feb 8 2018, 11:30 AM
IKhitron changed the status of subtask T184985: Wikidata changes on local wikis do not coordinate well between css classes and bold for unseen changes from Stalled to Open.Feb 8 2018, 1:26 PM
Ladsgroup added a subtask: T186645: Enable lua fine grained usage tracking -Early February 2018 batch (days: 2).Feb 8 2018, 1:28 PM
matej_suchanek removed a subtask: T184985: Wikidata changes on local wikis do not coordinate well between css classes and bold for unseen changes.Feb 8 2018, 1:54 PM
Ladsgroup added a subtask: T187265: Enable xkill on top wikis that use x aspect.Feb 13 2018, 11:18 PM
Krinkle mentioned this in T173710: Job queue is increasing non-stop.Feb 13 2018, 11:19 PM
RP88 subscribed.Feb 14 2018, 12:49 AM
Agabi10 subscribed.Feb 14 2018, 3:43 PM
gerritbot added a comment.Feb 20 2018, 8:42 PM

Change 412664 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Enable x-kill feature everywhere

https://gerrit.wikimedia.org/r/412664

Ladsgroup added a subtask: T186714: enable x-kill feature on Commons.Feb 20 2018, 8:43 PM
gerritbot added a comment.Feb 21 2018, 12:09 AM

Change 412664 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable x-kill feature everywhere

https://gerrit.wikimedia.org/r/412664

Stashbot mentioned this in T186714: enable x-kill feature on Commons.Feb 21 2018, 12:14 AM

Mentioned in SAL (#wikimedia-operations) [2018-02-21T00:14:05Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:412664|Enable x-kill feature everywhere]] T186714 T184322 (duration: 01m 13s)

Lydia_Pintscher closed subtask T186714: enable x-kill feature on Commons as Resolved.Feb 27 2018, 1:23 PM
Ladsgroup closed this task as Resolved.Mar 2 2018, 8:06 AM
Ladsgroup removed a project: Patch-For-Review.

It's enabled everywhere now.

Liuxinyu970226 unsubscribed.Mar 4 2018, 4:15 AM
• 238482n375 removed Ladsgroup as the assignee of this task.Jun 15 2018, 8:01 AM
• 238482n375 lowered the priority of this task from High to Lowest.
• 238482n375 moved this task from Next Up to In Code Review on the Analytics-Kanban board.
• 238482n375 added projects: Analytics-Kanban, acl*security, Wikimedia-VE-Campaigns (S2-2018), Scap (Scap3-Adoption-Phase2), AbuseFilter, Data-release, Hashtags, LabsDB-Auditor, Ladies-That-FOSS-MediaWiki, Language-2018-Apr-June, Language-2018-Jan-Mar, HHVM, HAWelcome.
• 238482n375 edited subscribers, added: • 238482n375; removed: Aklapper.

SG9tZVBoYWJyaWNhdG9yCk5vIG1lc3NhZ2VzLiBObyBub3RpZmljYXRpb25zLgoKICAgIFNlYXJjaAoKQ3JlYXRlIFRhc2sKTWFuaXBoZXN0ClQxOTcyODEKRml4IGZhaWxpbmcgd2VicmVxdWVzdCBob3VycyAodXBsb2FkIGFuZCB0ZXh0IDIwMTgtMDYtMTQtMTEpCk9wZW4sIE5lZWRzIFRyaWFnZVB1YmxpYwoKICAgIEVkaXQgVGFzawogICAgRWRpdCBSZWxhdGVkIFRhc2tzLi4uCiAgICBFZGl0IFJlbGF0ZWQgT2JqZWN0cy4uLgogICAgUHJvdGVjdCBhcyBzZWN1cml0eSBpc3N1ZQoKICAgIE11dGUgTm90aWZpY2F0aW9ucwogICAgQXdhcmQgVG9rZW4KICAgIEZsYWcgRm9yIExhdGVyCgpUYWdzCgogICAgQW5hbHl0aWNzLUthbmJhbiAoSW4gUHJvZ3Jlc3MpCgpTdWJzY3JpYmVycwpBa2xhcHBlciwgSkFsbGVtYW5kb3UKQXNzaWduZWQgVG8KSkFsbGVtYW5kb3UKQXV0aG9yZWQgQnkKSkFsbGVtYW5kb3UsIEZyaSwgSnVuIDE1CkRlc2NyaXB0aW9uCgpPb3ppZSBqb2JzIGhhdmUgYmVlbiBmYWlsaW5nIGF0IGxlYXN0IGEgZmV3IHRpbWVzIGVhY2guIE1vcmUgaW52ZXN0aWdhdGlvbiBuZWVkZWQuCkpBbGxlbWFuZG91IGNyZWF0ZWQgdGhpcyB0YXNrLkZyaSwgSnVuIDE1LCA3OjIxIEFNCkhlcmFsZCBhZGRlZCBhIHN1YnNjcmliZXI6IEFrbGFwcGVyLiC3IFZpZXcgSGVyYWxkIFRyYW5zY3JpcHRGcmksIEp1biAxNSwgNzoyMSBBTQpKQWxsZW1hbmRvdSBjbGFpbWVkIHRoaXMgdGFzay5GcmksIEp1biAxNSwgNzoyMiBBTQpKQWxsZW1hbmRvdSB1cGRhdGVkIHRoZSB0YXNrIGRlc2NyaXB0aW9uLiAoU2hvdyBEZXRhaWxzKQpKQWxsZW1hbmRvdSBhZGRlZCBhIHByb2plY3Q6IEFuYWx5dGljcy1LYW5iYW4uCkpBbGxlbWFuZG91IG1vdmVkIHRoaXMgdGFzayBmcm9tIE5leHQgVXAgdG8gSW4gUHJvZ3Jlc3Mgb24gdGhlIEFuYWx5dGljcy1LYW5iYW4gYm9hcmQuCkNoYW5nZSBTdWJzY3JpYmVycwpDaGFuZ2UgUHJpb3JpdHkKQXNzaWduIC8gQ2xhaW0KTW92ZSBvbiBXb3JrYm9hcmQKQ2hhbmdlIFByb2plY3QgVGFncwpBbmFseXRpY3MtS2FuYmFuCtcKU2VjdXJpdHkK1wpXaWtpbWVkaWEtVkUtQ2FtcGFpZ25zIChTMi0yMDE4KQrXClNjYXAK1wpTY2FwIChTY2FwMy1BZG9wdGlvbi1QaGFzZTIpCtcKQWJ1c2VGaWx0ZXIK1wpEYXRhLXJlbGVhc2UK1wpIYXNodGFncwrXCkxhYnNEQi1BdWRpdG9yCtcKTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2kK1wpMYW5ndWFnZS0yMDE4LUFwci1KdW5lCtcKTGFuZ3VhZ2UtMjAxOC1KYW4tTWFyCtcKSEhWTQrXCkhBV2VsY29tZQrXCkJvbGQKSXRhbGljcwpNb25vc3BhY2VkCkxpbmsKQnVsbGV0ZWQgTGlzdApOdW1iZXJlZCBMaXN0CkNvZGUgQmxvY2sKUXVvdGUKVGFibGUKVXBsb2FkIEZpbGUKTWVtZQpQcmV2aWV3CkhlbHAKRnVsbHNjcmVlbiBNb2RlClBpbiBGb3JtIE9uIFNjcmVlbgoyMzg0ODJuMzc1IGFkZGVkIHByb2plY3RzOiBTZWN1cml0eSwgV2lraW1lZGlhLVZFLUNhbXBhaWducyAoUzItMjAxOCksIFNjYXAgKFNjYXAzLUFkb3B0aW9uLVBoYXNlMiksIEFidXNlRmlsdGVyLCBEYXRhLXJlbGVhc2UsIEhhc2h0YWdzLCBMYWJzREItQXVkaXRvciwgTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2ksIExhbmd1YWdlLTIwMTgtQXByLUp1bmUsIExhbmd1YWdlLTIwMTgtSmFuLU1hciwgSEhWTSwgSEFXZWxjb21lLlBSRVZJRVcKMjM4NDgybjM3NSBtb3ZlZCB0aGlzIHRhc2sgZnJvbSBJbiBQcm9ncmVzcyB0byBJbiBDb2RlIFJldmlldyBvbiB0aGUgQW5hbHl0aWNzLUthbmJhbiBib2FyZC4KMjM4NDgybjM3NSByZW1vdmVkIEpBbGxlbWFuZG91IGFzIHRoZSBhc3NpZ25lZSBvZiB0aGlzIHRhc2suCjIzODQ4Mm4zNzUgdHJpYWdlZCB0aGlzIHRhc2sgYXMgTG93ZXN0IHByaW9yaXR5LgoyMzg0ODJuMzc1IHJlbW92ZWQgc3Vic2NyaWJlcnM6IEFrbGFwcGVyLCBKQWxsZW1hbmRvdS4KQ29udGVudCBsaWNlbnNlZCB1bmRlciBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLVNoYXJlQWxpa2UgMy4wIChDQy1CWS1TQSkgdW5sZXNzIG90aGVyd2lzZSBub3RlZDsgY29kZSBsaWNlbnNlZCB1bmRlciBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSAoR1BMKSBvciBvdGhlciBvcGVuIHNvdXJjZSBsaWNlbnNlcy4gQnkgdXNpbmcgdGhpcyBzaXRlLCB5b3UgYWdyZWUgdG8gdGhlIFRlcm1zIG9mIFVzZSwgUHJpdmFjeSBQb2xpY3ksIGFuZCBDb2RlIG9mIENvbmR1Y3QuILcgV2lraW1lZGlhIEZvdW5kYXRpb24gtyBQcml2YWN5IFBvbGljeSC3IENvZGUgb2YgQ29uZHVjdCC3IFRlcm1zIG9mIFVzZSC3IERpc2NsYWltZXIgtyBDQy1CWS1TQSC3IEdQTApZb3VyIGJyb3dzZXIgdGltZXpvbmUgc2V0dGluZyBkaWZmZXJzIGZyb20gdGhlIHRpbWV6b25lIHNldHRpbmcgaW4geW91ciBwcm9maWxlLCBjbGljayB0byByZWNvbmNpbGUu

• 238482n375 set Security to Software security bug.Jun 15 2018, 8:04 AM
• 238482n375 changed the visibility from "Public (No Login Required)" to "Custom Policy".

SG9tZVBoYWJyaWNhdG9yCk5vIG1lc3NhZ2VzLiBObyBub3RpZmljYXRpb25zLgoKICAgIFNlYXJjaAoKQ3JlYXRlIFRhc2sKTWFuaXBoZXN0ClQxOTcyODEKRml4IGZhaWxpbmcgd2VicmVxdWVzdCBob3VycyAodXBsb2FkIGFuZCB0ZXh0IDIwMTgtMDYtMTQtMTEpCk9wZW4sIE5lZWRzIFRyaWFnZVB1YmxpYwoKICAgIEVkaXQgVGFzawogICAgRWRpdCBSZWxhdGVkIFRhc2tzLi4uCiAgICBFZGl0IFJlbGF0ZWQgT2JqZWN0cy4uLgogICAgUHJvdGVjdCBhcyBzZWN1cml0eSBpc3N1ZQoKICAgIE11dGUgTm90aWZpY2F0aW9ucwogICAgQXdhcmQgVG9rZW4KICAgIEZsYWcgRm9yIExhdGVyCgpFVzZSC3IERpc2NsYWltZXIgtyBDQy1CWS1TQSC3IEdQTApZb3VyIGJyb3dzZXIgdGltZXpvbmUgc2V0dGluZyBkaWZmZXJzIGZyb20gdGhlIHRpbWV6b25lIHNldHRpbmcgaW4geW91ciBwcm9maWxlLCBjbGljayB0byByZWNvbmNpbGUu

eranroz raised the priority of this task from Lowest to High.Jun 15 2018, 9:57 AM
eranroz removed projects: HAWelcome, HHVM, Language-2018-Jan-Mar, Language-2018-Apr-June, Ladies-That-FOSS-MediaWiki, LabsDB-Auditor, Hashtags, Data-release, AbuseFilter, Scap (Scap3-Adoption-Phase2), Wikimedia-VE-Campaigns (S2-2018), acl*security, Analytics-Kanban.
eranroz edited subscribers, added: Aklapper; removed: • 238482n375.
Restricted Application added a project: acl*security. · View Herald TranscriptJun 15 2018, 9:57 AM
Halfak removed a project: acl*security.Jun 15 2018, 1:40 PM
Halfak changed the visibility from "Custom Policy" to "Public (No Login Required)".
Ladsgroup closed subtask T187265: Enable xkill on top wikis that use x aspect as Resolved.Jun 15 2018, 3:08 PM
matej_suchanek added a subtask: T186716: enable fine grained usage tracking on Commons.Jun 29 2018, 2:26 PM
matej_suchanek removed a subtask: T186716: enable fine grained usage tracking on Commons.Jun 29 2018, 3:19 PM
matej_suchanek added a subtask: T188730: [C-DIS][SW] Enable statement usage tracking on Commons and Co.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL