meeting notes: @aude to reach out to metrics platform to verify how the PHP SKD works with the same front-end schema.

@jwang @Jdrewniak is it necessary that these add / remove page (to reading list) events have the session_id?

@jwang I am looking at the contextual attributes:

https://phabricator.wikimedia.org/T402320

For the contextual attributes, should we also include "performer_id"? I think this is the central auth global id of the user.

https://wikitech.wikimedia.org/wiki/Experimentation_Lab/Contextual_attributes

Would including performer_id allow us to do hybrid backend (php) and client side instrumentation? or is having session id mandatory.

Change #1181192 had a related patch set uploaded (by Aude; author: Aude):

[mediawiki/extensions/ReadingLists@master] POC - experiment instrumentation for reading lists add/remove page

https://gerrit.wikimedia.org/r/1181192

In my LocalSettings.php, I have:

$wgEventStreams = [
    'product_metrics.web_base_reading_list' => [
        'schema_title' => '/analytics/product_metrics/web/base/1.4.2',
        'destination_event_service' => 'eventgate-analytics-external',
        'producers' => [
            'metrics_platform_client' => [
                'provide_values' => [
                    'agent_client_platform',
                    'agent_client_platform_family',
                    'page_content_language',
                    'page_namespace_id',
                    'mediawiki_skin',
                    'mediawiki_database',
                    'performer_is_logged_in',
                    'performer_is_temp',
                    'performer_is_bot',
                    'performer_edit_count_bucket',
                    'performer_session_id'
                ],
            ],
        ],
    ],
];

After discussion with the experiment platform team and technical investigation, we think it is best to keep all instrumentation on the client side.

The reading_listdatabase table has a column rl_size which keeps track of the size of the reading list. One possible solution could be to return this in the api response when a user adds or removes a page from their reading list. Then we can send an event for the experiment after the reading list add or removal action.

We are not supporting batch add/remove at this time on web, but that could be a consideration in the future with the instrumentation.

Here is a proof of concept of how this could work:

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReadingLists/+/1181192

By keeping instrumentation on the client-side, this brings more simplicity to the implementation and analysis. With php instrumentation, we do not have the session_id and would need another way to join the event data with the client side events. In the future, this will be possible for experiments with logged-in users.

LocalSettings.php

$wgMetricsPlatformExperiments = [
    'rwe-3-3-4-reading-list-test1' => [
        'name' => 'rwe-3-3-4-reading-list-test1',
        'sample' => [
            'unit' => 'user',
            'rate' => 1
        ],
        'groups' => [
            'control',
            'treatment'
        ]
    ]
];

I am seeing the event in eventlogging:

"event": {
    "action": "reading_list_change_xlab",
    "action_subtype": "remove_article_from_reading_list",
    "action_source": "reading_list_api",
    "action_context": "{\"article_count\":0}",
    "experiment": {
        "enrolled": "rwe-3-3-4-reading-list-test1",
        "assigned": "treatment",
        "subject_id": "fcaa63b95334e5fc2397ad74bab68aa3a2dec4245b477f0644649da22b69ce09",
        "sampling_unit": "mw-user",
        "coordinator": "xLab"
    }
}

Change #1183160 had a related patch set uploaded (by Aude; author: Aude):

[mediawiki/extensions/WikimediaEvents@master] POC - Add ReadingLists experiment

https://gerrit.wikimedia.org/r/1183160

Change #1181192 merged by jenkins-bot:

[mediawiki/extensions/ReadingLists@master] Add data-mw-list-page-count data attribute for add/remove page instrumentation

https://gerrit.wikimedia.org/r/1181192

Change #1183160 abandoned by Aude:

[mediawiki/extensions/WikimediaEvents@master] Add ReadingLists experiment, instrument page add/remove

Reason:

done in another patch

https://gerrit.wikimedia.org/r/1183160

@Jdrewniak, is this something testable in prod? I did check out the events but I don't recall how to get bucketed.

@Edtadros We won't have prod testing abilities until the test is configured with xlab in T402320. I think maybe we can wait for that and then do a round of dedicated QA for the event instrumentation.