Page MenuHomePhabricator
Create Task
Maniphest T274175

Control instruments with Stream Configuration
Closed, ResolvedPublic
Actions

Description

This task tracks work for a demonstration of the new capabilities for creating datasets that the client libraries enable. The demo will cover combinations of basic operations using examples drawn from past experience, to illustrate how it can save time. Operations covered would be

  1. Placing instrumentation in code
  2. Defining a stream configuration
  3. Deploying a stream configuration
  4. At least the following stream configuration operations
    1. Multiple streams consuming the same instrument
    2. One stream consuming multiple instruments
    3. Streams requesting metadata and having it automatically included in the event
    4. Streams specifying targeting using value-based sampling

Specifics to be added in near future.

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

• jlinehan created this task.Feb 8 2021, 6:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 8 2021, 6:56 PM
• jlinehan triaged this task as Medium priority.Feb 8 2021, 7:28 PM
• jlinehan moved this task from Inbox to Doing on the Better Use Of Data board.
gerritbot added a comment.Feb 10 2021, 2:53 PM

Change 663211 had a related patch set uploaded (by Jason Linehan; owner: Jason Linehan):
[mediawiki/extensions/EventLogging@master] [WIP] Streams subscribe to one or more instruments

https://gerrit.wikimedia.org/r/663211

gerritbot added a project: Patch-For-Review.Feb 10 2021, 2:53 PM
• Mholloway mentioned this in T271456: Enable 'skin' dimension using stream configuration.Feb 10 2021, 7:07 PM
gerritbot added a comment.Feb 11 2021, 5:32 PM

Change 663616 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/EventLogging@master] MEP: Add stream config support for filtering by value

https://gerrit.wikimedia.org/r/663616

gerritbot added a comment.Feb 11 2021, 5:34 PM

Change 661975 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/EventLogging@master] MEP: Demo support for adding login state and wiki db name via config

https://gerrit.wikimedia.org/r/661975

awight subscribed.Mar 9 2021, 9:32 AM

I would love to use the mechanism described in 4.C:

Streams requesting metadata and having it automatically included in the event

For T210106, I would add the ability to transparently fill a user_edit_count_bucket field. But maybe it's too early, the deployed code can only write to a test object?

Also, the existing schemas have a variety of property paths into which this data should land, for example TemplateWizard wants the data in event.performer.user_edit_count_bucket, but QuickSurveysResponses wants it in event.editCountBucket. Will the final design include this kind of flexibility?

• jlinehan moved this task from Doing to To Do on the Better Use Of Data board.Mar 10 2021, 7:19 PM
kzimmerman moved this task from To Do to Doing on the Better Use Of Data board.Mar 10 2021, 7:19 PM
kzimmerman moved this task from Doing to To Do on the Better Use Of Data board.
gerritbot added a comment.Mar 18 2021, 4:32 PM

Change 663616 abandoned by Mholloway:
[mediawiki/extensions/EventLogging@master] MEP: Add stream config support for filtering by value

Reason:
There was a big structural change in core.js, and also the syntax for this is now completely different. I'll start a new patch.

https://gerrit.wikimedia.org/r/663616

gerritbot added a comment.Mar 18 2021, 4:34 PM

Change 661975 abandoned by Mholloway:
[mediawiki/extensions/EventLogging@master] MEP: Demo support for adding login state and wiki db name via config

Reason:
Will start a new patch for this rather than rebasing.

https://gerrit.wikimedia.org/r/661975

gerritbot added a comment.Mar 18 2021, 7:35 PM

Change 673345 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/EventLogging@master] [WIP] Metrics Platform: Filter events by value

https://gerrit.wikimedia.org/r/673345

gerritbot added a comment.Mar 24 2021, 2:26 PM

Change 674620 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/EventLogging@master] [WIP] Metrics platform: Add context values requested in stream config

https://gerrit.wikimedia.org/r/674620

• Mholloway added a project: Product-Data-Infrastructure.Apr 16 2021, 5:35 PM
• Mholloway moved this task from Inbox to Doing on the Product-Data-Infrastructure board.
• Mholloway moved this task from To Do to Doing on the Better Use Of Data board.Apr 16 2021, 5:38 PM
ldelench_wmf moved this task from Doing to To Do on the Better Use Of Data board.Apr 21 2021, 6:34 PM
• Mholloway added a project: Metrics Platform Backlog.Jun 2 2021, 4:27 PM
• Mholloway mentioned this in T284161: Update Java metrics platform client to support Kotlin.
gerritbot added a comment.Jun 3 2021, 7:05 PM

Change 698018 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] Allow curating stream data sets based on event data values

https://gerrit.wikimedia.org/r/698018

gerritbot added a comment.Jun 3 2021, 7:08 PM

Change 673345 abandoned by Mholloway:

[mediawiki/extensions/EventLogging@master] Metrics Platform: Filter events by value

Reason:

See I2afae4799cb0fdcf26bdf75f4c3621c80a95c069

https://gerrit.wikimedia.org/r/673345

gerritbot added a comment.Jun 4 2021, 8:08 PM

Change 674620 abandoned by Mholloway:

[mediawiki/extensions/EventLogging@master] Metrics Platform: Add context values requested in stream config

Reason:

See I8ab62c8ac63bda9e91c8dacba18fd805de655c77

https://gerrit.wikimedia.org/r/674620

• DAbad moved this task from Backlog to Ready/Groomed on the Metrics Platform Backlog board.Jun 30 2021, 3:03 PM
gerritbot added a comment.Jul 14 2021, 9:12 PM

Change 704630 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] [Java] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/704630

gerritbot added a comment.Jul 14 2021, 9:12 PM

Change 698241 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] [JS] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/698241

gerritbot added a comment.Jul 19 2021, 5:54 PM

Change 705435 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] [PHP] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/705435

gerritbot added a comment.Aug 3 2021, 8:24 PM

Change 709820 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] [WIP] [Java] Support curating stream data sets based on event properties

https://gerrit.wikimedia.org/r/709820

• Mholloway renamed this task from [Client libraries] Demonstrate new capabilities to Control instruments with Stream Configuration.Aug 11 2021, 1:43 PM
• Mholloway updated the task description. (Show Details)Aug 11 2021, 6:58 PM
gerritbot added a comment.Aug 19 2021, 7:04 PM

Change 713927 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] [WIP] [Swift] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/713927

• jlinehan moved this task from Ready/Groomed to Discussed on the Metrics Platform Backlog board.Aug 23 2021, 1:45 PM
gerritbot added a comment.Sep 1 2021, 10:30 PM

Change 716057 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] [VERY WIP] [Swift] Support curating stream data sets based on event properties

https://gerrit.wikimedia.org/r/716057

Ottomata subscribed.Sep 3 2021, 3:18 PM
• Mholloway added a subtask: T259708: MEP Client MediaWiki JS (non-essential features).Sep 7 2021, 1:41 PM
gerritbot added a comment.Sep 7 2021, 10:28 PM

Change 719378 had a related patch set uploaded (by Mholloway; author: Michael Holloway):

[mediawiki/libs/metrics-platform@master] [PHP] Support curating stream data sets based on event properties

https://gerrit.wikimedia.org/r/719378

gerritbot added a comment.Sep 13 2021, 7:38 PM

Change 704630 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [Java] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/704630

• Mholloway mentioned this in rMLMP684baac60322: [Java] Support adding context values based on stream config.Sep 13 2021, 7:38 PM
gerritbot added a comment.Sep 13 2021, 7:43 PM

Change 709820 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [Java] Support curating stream data sets based on event properties

https://gerrit.wikimedia.org/r/709820

• Mholloway mentioned this in rMLMP314f78df9def: [Java] Support curating stream data sets based on event properties.Sep 13 2021, 7:44 PM
gerritbot added a comment.Sep 13 2021, 8:12 PM

Change 713927 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [Swift] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/713927

• Mholloway mentioned this in rMLMP37a857a9d47d: [Swift] Support adding context values based on stream config.Sep 13 2021, 8:12 PM
gerritbot added a comment.Sep 13 2021, 8:18 PM

Change 716057 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [Swift] Support curating stream data sets based on event properties

https://gerrit.wikimedia.org/r/716057

• Mholloway mentioned this in rMLMP53d768e33955: [Swift] Support curating stream data sets based on event properties.Sep 13 2021, 8:18 PM
gerritbot added a comment.Sep 13 2021, 8:31 PM

Change 698241 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [JS] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/698241

• Mholloway mentioned this in rMLMP2a7f65a1fab9: [JS] Support adding context values based on stream config.Sep 13 2021, 8:32 PM
gerritbot added a comment.Sep 13 2021, 8:33 PM

Change 698018 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [JS] Support curating stream data sets based on event properties

https://gerrit.wikimedia.org/r/698018

• Mholloway mentioned this in rMLMPf9baa8a07d2d: [JS] Support curating stream data sets based on event properties.Sep 13 2021, 8:33 PM
gerritbot added a comment.Sep 13 2021, 8:41 PM

Change 705435 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [PHP] Support adding context values based on stream config

https://gerrit.wikimedia.org/r/705435

• Mholloway mentioned this in rMLMP90e59363f89e: [PHP] Support adding context values based on stream config.Sep 13 2021, 8:41 PM
gerritbot added a comment.Sep 13 2021, 8:42 PM

Change 719378 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] [PHP] Support curating stream data sets based on event properties

https://gerrit.wikimedia.org/r/719378

• Mholloway mentioned this in rMLMPb188bd2a175a: [PHP] Support curating stream data sets based on event properties.Sep 13 2021, 8:43 PM
• Mholloway closed subtask T287892: Stream config: Refine value-based curation config syntax as Resolved.Sep 17 2021, 3:47 PM
• DAbad changed the task status from Open to Stalled.Oct 6 2021, 2:10 PM
• DAbad moved this task from Discussed to To be prioritised on the Metrics Platform Backlog board.
• DAbad subscribed.

Moving to blocked until we are ready to deploy for end-to-end testing

• EChetty reassigned this task from • jlinehan to phuedx.Apr 28 2022, 10:47 AM
gerritbot added a comment.May 4 2022, 3:37 PM

Change 663211 abandoned by Phuedx:

[mediawiki/extensions/EventLogging@master] Streams subscribe to one or more instruments

Reason:

Superseded by I0badcfb5b52a7981c5e697ff7f3c5d70278684ad

https://gerrit.wikimedia.org/r/663211

• EChetty moved this task from To be prioritised to Discussed on the Metrics Platform Backlog board.May 4 2022, 7:01 PM
• EChetty changed the task status from Stalled to In Progress.May 5 2022, 11:05 AM
• EChetty moved this task from Discussed to QA/Review on the Metrics Platform Backlog board.
• EChetty removed a project: Patch-For-Review.May 12 2022, 11:10 AM
• EChetty moved this task from QA/Review to Done on the Metrics Platform Backlog board.
• EChetty closed this task as Resolved.Oct 1 2022, 4:16 PM
phuedx closed subtask T259708: MEP Client MediaWiki JS (non-essential features) as Resolved.Mar 16 2023, 1:38 PM
phuedx closed subtask T289608: Review names of each attribute & patch as needed as Declined.Jul 31 2023, 2:24 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL