End to end test of the Java MPC
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• EChetty
	Jan 16 2023, 3:25 PM

Description

Verify that EventLogging can receive and validate events submitted by the MP Java client.

Approach

Before the Java library is packaged/published, we can perform end-to-end testing by embedding the MP Java Client into a sample project and submitting an event to a local EventLogging stream using the available Java methods for dispatching and enqueueing events.

Acceptance criteria

A test project embeds the MP Java Client library as a dependency
When the MetricsClient::dispatch() and EventProcessor::sendEnqueuedEvents() are invoked, a local EventLogging stream accepts and validates a test event.

Details

	Subject	Repo	Branch	Lines +/-
	Add end-to-end test to emulate producing event to local eventlogging	mediawiki/libs/metrics-platform	master	+2 K -11

Customize query in gerrit

Related Objects

Mentioned In: T331501: [Java] Enable pattern matching for event name prefixes
T330460: Fix context values in Java client to pass EventLogging validation
T330459: Add wikidata_qid property to metrics_event schema
T281773: Publish librarized Metrics Platform Java client to Maven

Event Timeline

• EChetty created this task.Jan 16 2023, 3:25 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 16 2023, 3:25 PM

cjming claimed this task.Feb 1 2023, 3:54 PM

cjming moved this task from Next Up to In Progress on the Metrics Platform Backlog (Metrics Platform Kanban) board.Feb 8 2023, 3:09 PM

cjming renamed this task from End to end test of the Java MPC in android app to End to end test of the Java MPC.Feb 22 2023, 2:12 AM

cjming triaged this task as Medium priority.

cjming updated the task description. (Show Details)

The following sample project embeds a packaged version of the MP Java Client as a dependency:
https://github.com/cjming/MetricsPlatform-testJavaClient

Instructions are included in the README file.

Running TestJava::main produces an event that is accepted and validated by local EventLogging:
{F36864494}{F36864495}

The stream config used for this test is a modified version of https://github.com/wikimedia/mediawiki-libs-metrics-platform/blob/master/tests/consistency/stream_configs.json and the test ClientMetadata is a direct copy of https://github.com/wikimedia/mediawiki-libs-metrics-platform/blob/master/tests/consistency/integration_data.json

The packaged jar file of the MP Java Client has been slightly modified to expedite submitting an event to local EventLogging. The following changes were made:

StreamConfig::getDestinationEventService() simply returns DestinationEventService.LOCAL.
- StreamConfigTest.java assertion was updated to match the LOCAL enum.
The SamplingController and SessionController classes are made public.

Some notes/questions:

I am not sure if I correctly sent events but since the event was produced in my local EventLogging, I assume some implementation of the EventSender interface should be included in the Java MPC?

I might have uncovered two possible bugs during this end-to-end testing related to the follow properties:
- wikidata_id
  - The jsonschema for this property indicates that it should be an integer however in the Java MPC, it is currently a string
- registration_dt
  - The jsonschema for this property indicates it should be a string in date-time format but in the Java MPC, it is currently of type Long

The above properties did not validate in my local EventLogging:

mediawiki-eventlogging-1  | [2023-02-22T02:37:35.171Z] TRACE: service-runner/27 on c53e3f496c66: Set meta.dt in event 5b2d35d8-ce1f-4647-b643-d6ca876a2942 of schema at /analytics/mediawiki/client/metrics_event/1.1.0 destined to stream test.stream
mediawiki-eventlogging-1  | [2023-02-22T02:37:35.171Z] TRACE: service-runner/27 on c53e3f496c66: Set meta.dt in event 5b2d35d8-ce1f-4647-b643-d6ca876a2942 of schema at /analytics/mediawiki/client/metrics_event/1.1.0 destined to stream test.stream
mediawiki-eventlogging-1  | [2023-02-22T02:37:35.171Z] TRACE: service-runner/27 on c53e3f496c66: Set meta.request_id in event 5b2d35d8-ce1f-4647-b643-d6ca876a2942 of schema at /analytics/mediawiki/client/metrics_event/1.1.0 destined to stream test.stream
mediawiki-eventlogging-1  | [2023-02-22T02:37:35.171Z] TRACE: service-runner/27 on c53e3f496c66: Set http.request_headers.user-agent in event 5b2d35d8-ce1f-4647-b643-d6ca876a2942 of schema at /analytics/mediawiki/client/metrics_event/1.1.0 destined to stream test.stream
mediawiki-eventlogging-1  | [2023-02-22T02:37:35.172Z]  WARN: service-runner/27 on c53e3f496c66: event failed schema validation.
mediawiki-eventlogging-1  |     ValidationError: Event failed validation with schema at /analytics/mediawiki/client/metrics_event/1.1.0
mediawiki-eventlogging-1  |         at validatorFor.then (/opt/eventlogging/node_modules/eventgate/lib/EventValidator.js:285:23)
mediawiki-eventlogging-1  |         at tryCatcher (/opt/eventlogging/node_modules/bluebird/js/release/util.js:16:23)
mediawiki-eventlogging-1  |         at Promise._settlePromiseFromHandler (/opt/eventlogging/node_modules/bluebird/js/release/promise.js:547:31)
mediawiki-eventlogging-1  |         at Promise._settlePromise (/opt/eventlogging/node_modules/bluebird/js/release/promise.js:604:18)
mediawiki-eventlogging-1  |         at Promise._settlePromiseCtx (/opt/eventlogging/node_modules/bluebird/js/release/promise.js:641:10)
mediawiki-eventlogging-1  |         at _drainQueueStep (/opt/eventlogging/node_modules/bluebird/js/release/async.js:97:12)
mediawiki-eventlogging-1  |         at _drainQueue (/opt/eventlogging/node_modules/bluebird/js/release/async.js:86:9)
mediawiki-eventlogging-1  |         at Async._drainQueues (/opt/eventlogging/node_modules/bluebird/js/release/async.js:102:5)
mediawiki-eventlogging-1  |         at Immediate.Async.drainQueues [as _onImmediate] (/opt/eventlogging/node_modules/bluebird/js/release/async.js:15:14)
mediawiki-eventlogging-1  |         at runCallback (timers.js:705:18)
mediawiki-eventlogging-1  |         at tryOnImmediate (timers.js:676:5)
mediawiki-eventlogging-1  |         at processImmediate (timers.js:658:5)
mediawiki-eventlogging-1  |     --
mediawiki-eventlogging-1  |     err.errors: [
mediawiki-eventlogging-1  |       {
mediawiki-eventlogging-1  |         "keyword": "type",
mediawiki-eventlogging-1  |         "dataPath": ".page.wikidata_id",
mediawiki-eventlogging-1  |         "schemaPath": "#/properties/page/properties/wikidata_id/type",
mediawiki-eventlogging-1  |         "params": {
mediawiki-eventlogging-1  |           "type": "integer"
mediawiki-eventlogging-1  |         },
mediawiki-eventlogging-1  |         "message": "should be integer"
mediawiki-eventlogging-1  |       },
mediawiki-eventlogging-1  |       {
mediawiki-eventlogging-1  |         "keyword": "type",
mediawiki-eventlogging-1  |         "dataPath": ".performer.registration_dt",
mediawiki-eventlogging-1  |         "schemaPath": "#/properties/performer/properties/registration_dt/type",
mediawiki-eventlogging-1  |         "params": {
mediawiki-eventlogging-1  |           "type": "string"
mediawiki-eventlogging-1  |         },
mediawiki-eventlogging-1  |         "message": "should be string"
mediawiki-eventlogging-1  |       }
mediawiki-eventlogging-1  |     ]
mediawiki-eventlogging-1  |     --
mediawiki-eventlogging-1  |     err.errorsText: '.page.wikidata_id' should be integer, '.performer.registration_dt' should be string
mediawiki-eventlogging-1  | [2023-02-22T02:37:35.172Z]  WARN: eventgate-devserver/27 on c53e3f496c66: 1 out of 1 events were invalid and not accepted. (levelPath=warn/events, request_id=dd393220-b259-11ed-91a1-013038519d35)
mediawiki-eventlogging-1  |     request: {
mediawiki-eventlogging-1  |       "url": "/v1/events",
mediawiki-eventlogging-1  |       "headers": {
mediawiki-eventlogging-1  |         "content-type": "application/json",
mediawiki-eventlogging-1  |         "user-agent": "Java/1.8.0_345",
mediawiki-eventlogging-1  |         "content-length": "989",
mediawiki-eventlogging-1  |         "x-request-id": "dd393220-b259-11ed-91a1-013038519d35"
mediawiki-eventlogging-1  |       },
mediawiki-eventlogging-1  |       "method": "POST",
mediawiki-eventlogging-1  |       "params": {
mediawiki-eventlogging-1  |         "0": "/v1/events"
mediawiki-eventlogging-1  |       },
mediawiki-eventlogging-1  |       "query": {},
mediawiki-eventlogging-1  |       "remoteAddress": "172.18.0.1",
mediawiki-eventlogging-1  |       "remotePort": 60794
mediawiki-eventlogging-1  |     }

I'll follow up with tickets to update these properties in the Java MPC pending discussion with @phuedx.

For the sake of expediency, I removed these two context values from the provide_values key in stream_configs.json of the test repo. I will update the end-to-end test to re-include them once their types are sorted out.

cjming added a subscriber: phuedx.Feb 22 2023, 2:43 AM

cjming moved this task from In Progress to In Review on the Metrics Platform Backlog (Metrics Platform Kanban) board.Feb 22 2023, 2:50 AM

cjming mentioned this in T281773: Publish librarized Metrics Platform Java client to Maven.Feb 22 2023, 11:44 PM

cjming mentioned this in T330459: Add wikidata_qid property to metrics_event schema.Feb 24 2023, 12:56 AM

cjming mentioned this in T330460: Fix context values in Java client to pass EventLogging validation.Feb 24 2023, 1:00 AM

Latest version of the test repo leverages the following upstream changes:

Along with the minor changes to the packaged MP jar file mentioned in T327083#8635819

The result is that the 2 properties that were not passing validation previously in local EventLogging are now passing:

Screen Shot 2023-02-23 at 9.13.59 PM.png (2×4 px, 1 MB)

In T327083#8635819, @cjming wrote:

The following sample project embeds a packaged version of the MP Java Client as a dependency:
https://github.com/cjming/MetricsPlatform-testJavaClient

🎉

Some notes/questions:

I am not sure if I correctly sent events but since the event was produced in my local EventLogging, I assume some implementation of the EventSender interface should be included in the Java MPC?

The JS MPC includes a default implementation, which only uses stable APIs. All MPCs probably should to ease integration testing and initial setup in first-party projects like the portals and third-party projects.

As we discussed, wherever possible we should be using a UA that identifies the Metrics Platform, e.g. Metrics Platform/$metricsPlatformVersion Java/$javaVersion (per https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent#syntax).

I might have uncovered two possible bugs during this end-to-end testing related to the follow properties:

wikidata_id

The jsonschema for this property indicates that it should be an integer however in the Java MPC, it is currently a string

That's a bug in the schema. Could you create a Phab task?

registration_dt

The jsonschema for this property indicates it should be a string in date-time format but in the Java MPC, it is currently of type Long

This is a bug in the Java MPC. Like the dt and client_dt properties, the registration_dt property should be a serialised according to ISO 8601 in the UTC timezone (and the resulting string should end with a Z).

phuedx moved this task from In Review to Sign Off on the Metrics Platform Backlog (Metrics Platform Kanban) board.Feb 24 2023, 10:48 AM

Change 894750 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/libs/metrics-platform@master] DO NOT MERGE: end-to-end test using local event logging

https://gerrit.wikimedia.org/r/894750

gerritbot added a project: Patch-For-Review.Mar 7 2023, 12:25 AM

cjming mentioned this in T331501: [Java] Enable pattern matching for event name prefixes.Mar 8 2023, 4:03 AM

Change 894750 merged by jenkins-bot:

[mediawiki/libs/metrics-platform@master] Add end-to-end test to emulate producing event to local eventlogging

https://gerrit.wikimedia.org/r/894750

Maintenance_bot removed a project: Patch-For-Review.Mar 17 2023, 2:11 PM

Being bold.

SNowick_WMF subscribed.Apr 6 2023, 4:49 PM

	F36867472: Screen Shot 2023-02-23 at 9.13.59 PM.png
	Feb 24 2023, 4:29 AM

	F36864483: JavaClientEndToEndTesting.png
	Feb 22 2023, 2:16 AM

End to end test of the Java MPCClosed, ResolvedPublicActions