T258183 will instrument the MediaSearch results page for measurement, but it will not include the file name and wikitext link copy functionality in T261699 because that is coming later. Once T261699 is done, this ticket is to instrument the following using the same method in T258183:
Change 661273 had a related patch set uploaded (by Eric Gardner; owner: Eric Gardner):
[schemas/event/secondary@master] Update schema to handle quickview copy events
Change 661274 had a related patch set uploaded (by Eric Gardner; owner: Eric Gardner):
[mediawiki/extensions/WikibaseMediaInfo@master] Instrument QuickView copy events
I've opened two patches for this task.
Let me know if anything should change here, otherwise this is ready for review. @Nettrom if anyone else on Analytics should review the schema, feel free to add them to the patch.
Change 661274 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Instrument QuickView copy events
Change 661273 merged by Mholloway:
[schemas/event/secondary@master] Update schema to handle quickview copy events
Checked in betalabs - both events are present
quickview_filename_copy
{action: "quickview_filename_copy", $schema: "/analytics/mediawiki/mediasearch_interaction/1.0.0", web_pageview_id: "ceb16b08947a98206ee9", language_code: "en", ui_mw_skin: "vector", …}
$schema: "/analytics/mediawiki/mediasearch_interaction/1.0.0"
action: "quickview_filename_copy"
dt: "2021-02-17T01:03:19.379Z"
language_code: "en"
meta: {stream: "mediawiki.mediasearch_interaction", domain: "commons.wikimedia.beta.wmflabs.org"}
ui_mw_skin: "vector"
web_pageview_id: "ceb16b08947a98206ee9"quickview_wikitext_link_copy
{action: "quickview_wikitext_link_copy", $schema: "/analytics/mediawiki/mediasearch_interaction/1.0.0", web_pageview_id: "ceb16b08947a98206ee9", language_code: "en", ui_mw_skin: "vector", …}
$schema: "/analytics/mediawiki/mediasearch_interaction/1.0.0"
action: "quickview_wikitext_link_copy"
dt: "2021-02-17T01:05:44.545Z"
language_code: "en"
meta: {stream: "mediawiki.mediasearch_interaction", domain: "commons.wikimedia.beta.wmflabs.org"}
ui_mw_skin: "vector"
web_pageview_id: "ceb16b08947a98206ee9"Thanks for verifying this @Etonkovidova. This actually makes me realize that I missed something – the logger needs to be told to use version 1.1.0 of the schema to properly capture these events (soon we'll have to bump it to 1.2.0 once the second follow-up schema patch goes through).
Fortunately this is simple to fix.
Sorry for this, I'm still getting the hang of how to use the new analytics platform. I'll post one last (small) patch shortly.
Change 664693 had a related patch set uploaded (by Eric Gardner; owner: Eric Gardner):
[mediawiki/extensions/WikibaseMediaInfo@master] Update Logger to use version 1.1.0 of mediasearch schema
Change 664693 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Update Logger to use version 1.1.0 of mediasearch schema
@egardner Yes, I wasn't sure about 1.0.0 - some schemas have been migrated, some haven't. There was a case with HomepageModule schema when events were rejected because of old version number, so I made a note to re-check in production. Good thing that you caught the incorrect versioning!
Unfortunately, currently there is no way to catch validation errors in beta cluster.
Checked production commons wmf.31 - logstash eventgate-validation dashboard predictably shows
'.action' should be equal to one of the allowed values
{
"_index": "logstash-2021.02.18",
"_type": "_doc",
"_id": "7Ou6sncBXM-H9NFXRhKp",
"_version": 1,
"_score": null,
"_source": {
"errored_schema_uri": "/analytics/mediawiki/mediasearch_interaction/1.0.0",
"tags": [
"input-kafka-eventgate-analytics-external-validation-error-codfw",
"kafka",
"es",
"eventgate",
"normalized_message_untrimmed"
],
"message": "'.action' should be equal to one of the allowed values",
"@version": "1",
"$schema": "/error/1.0.0",
"type": "eventgate_validation_error",
"@timestamp": "2021-02-18T01:21:26.212Z",
"errored_stream_name": "mediawiki.mediasearch_interaction",
"emitter_id": "eventgate-analytics-external-canary",
"raw_event": "{\"action\":\"quickview_wikitext_link_copy\",\"$schema\":\"/analytics/mediawiki/mediasearch_interaction/1.0.0\",\"web_pageview_id\":\"8a7c1318e8652ed0ed0e\",\"language_code\":\"en\",\"ui_mw_skin\":\"vector\",\"meta\":{\"stream\":\"mediawiki.mediasearch_interaction\",\"domain\":\"commons.wikimedia.org\",\"id\":\"93f7c844-3d9a-4fa1-890a-a34c60ee5286\",\"dt\":\"2021-02-18T01:21:25.078Z\",\"request_id\":\"1b3ac0d3-55ee-4f2c-8484-dd3533039e65\"},\"dt\":\"2021-02-18T01:21:22.741Z\",\"http\":{\"request_headers\":{\"user-agent\":\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36\"}}}",
"normalized_message": "'.action' should be equal to one of the allowed values",
"meta": {
"uri": "unknown",
"request_id": "1b3ac0d3-55ee-4f2c-8484-dd3533039e65",
"id": "f526705c-c483-4f86-a4ea-eb7b3a710083",
"domain": "commons.wikimedia.org",
"stream": "eventgate-analytics-external.error.validation",
"dt": "2021-02-18T01:21:25.080Z"
}
},
"fields": {
"@timestamp": [
"2021-02-18T01:21:26.212Z"
]
},
"highlight": {
"errored_stream_name.keyword": [
"@kibana-highlighted-field@mediawiki.mediasearch_interaction@/kibana-highlighted-field@"
],
"type": [
"@kibana-highlighted-field@eventgate_validation_error@/kibana-highlighted-field@"
]
},
"sort": [
1613611286212
]
}Checked commons wmf.34 - no validation errors; the events are properly recorded and the stream has updated schema.
e.g.
{action: "quickview_wikitext_link_copy", $schema: "/analytics/mediawiki/mediasearch_interaction/1.1.0", web_pageview_id: "61e0549e52bb297d5d91", language_code: "en", ui_mw_skin: "vector", …}
$schema: "/analytics/mediawiki/mediasearch_interaction/1.1.0"
action: "quickview_wikitext_link_copy"
dt: "2021-03-12T00:15:12.658Z"
language_code: "en"
meta: {stream: "mediawiki.mediasearch_interaction", domain: "commons.wikimedia.org"}
ui_mw_skin: "vector"
web_pageview_id: "61e0549e52bb297d5d91"