User story
As a product manager of contributor-facing features, I want to learn and iterate on ideas/interventions quickly. I need to test hypotheses with experiments (A/B tests) where username and/or user ID must be collected with interaction data so that the analysis results can include insights about the impact of the tested idea/intervention on editors' experience and their productivity.
Background
Under the data collection guidelines, each experiment we do that involves usernames and/or user IDs is classified as a Medium Risk data collection activity and requires Legal & Security review and approval, which can take 1-2 weeks. This is because as an ongoing data collection activity, Low Risk classification requires that all low risk criteria are met.
The one low risk criteria we fail is "personal information + username/user ID" because the system automatically collects User-Agent data with every event and it gets stored in http.request_headers['user-agent']. There is no UA-specific field/fragment that we can keep out of schemas like we can with IP addresses (which are only collected when http.client_ip exists in the schema).
If we made UA data optional, it would make it possible for us to run editor-focused A/B tests that record usernames/user IDs for analysis, without elevating the data collection activity to Medium Risk.
- Done is
- It is possible to disable collection of User-Agent header in for event streams that have the http.request_headers field.
- Wikitech documentation updated:
Chosen implementation
- Ownership of configuration to enrich event stream data with stream headers has moved to EventStreamConfig. The new producers.eventgate.enrich_fields_with_http_headers config setting is a map of field name to header name. It has a default value that is applied to all streams, and can be overridden per stream.
- More work will be needed for T385180: Implement agent.ua_string as contextual attribute
- EventLogging PHP submitted events. These manually set the http.request_headers['user-agent'] field, and eventgate will not override what the client sets.
- Experiment Platform 'mono streams'. These are streams that have the same schema, that are shared by many instruments. If it is needed to vary collection of user-agent per instrument, then the client will have to handle conditionally setting this in the event.
Implementation context
eventgate-wikimedia has a http_request_headers_to_fields option, which defaults to
http_request_headers_to_fields: { 'x-request-id': 'meta.request_id', 'user-agent': 'http.request_headers.user-agent', },
This option is used to automatically set event fields with http request headers. E.g. if a user-agent header is set in the http request to EventGate, and if the event's schema has a http.request_headers map field, http.request_headers['user-agent'] will be set with the value of the User-Agent header.
We need a way for this code to explicitly enable or disable collection of this (and possible any?) request header like this. In this case, doing this based on the presence of a field name in a schema won't work, because this field is a map field, and the keys of the map (e.g. 'user-agent') are not pre-defined in the schema.
Implementation options
Option 1a: Use an EventStreamConfig setting to configure http_request_headers_to_fields per stream.
A StreamConfigs is initialized for each eventgate instance. This instance is currently only used in a couple of places when receiving an event to produce.
We should be able to use it (via options.streamConfigs) in the makeSetWikimediaDefaults function to get the http_request_headers_to_fields setting from EventStreamConfig for a stream.
The eventgate-wikimedia code default will be used for any streams that don't specifically override this setting.
In wgEventStreams (EventStreamConfig), this probably belongs in a producers.eventgate specific setting. Something like
'mediawiki.web_ui_actions' => [ // ... 'producers' => [ 'metrics_platform_client' => [ 'provide_values' => [ // ... ], ], 'eventgate' => [ 'http_request_headers_to_fields' => [ 'x-request-id' => 'meta.request_id' ] ], ], // ...
Here, because 'http_request_headers_to_fields' is overridden, 'user-agent' will not be collected.
Pros:
- http_request_headers_to_fields makes things more configurable, and allows us to explicitly configure the setting of request_headers.
Cons:
- verbose configuration just to disable user-agent collection.
Option 1b: Variation: disable user-agent collection by default, and explicitly enable it
This would be the same implementation of Option 1a, except that we disable user-agent collection or perhaps even all http_request_headers_to_fields collection, by default in eventgate.
We then use EventStreamConfig to opt-in to collection of any of these things.
In EventStreamConfig defaults, we could set the headers we want to collect by default, and add any overrides in specific per-stream config in the same producers.eventgate.http_request_headers_to_fields setting.
Pros:
- http_request_headers_to_fields makes things more configurable, and allows us to explicitly configure the setting of request_headers.
- header collection setting defaults handing is in one place, instead of in both EventGate service config and EventStreamConfig.
- Assuming most streams don't need user-agent, per stream configuration is less verbose.
- Using config to configure data collection is better than using schemas.
Cons:
- If most streams do need user-agent, then per stream configuration is more verbose.
A slight tweak would be to manage defaults fully in EventStreamConfig settings (removing them from EventGate service configs), but still enable user-agent collection by default. This has the pro of managing the default for this in one place, and possibly less verbose per-stream configuration.
Option 2: Make user_agent its own fragment schema field
This is how http.client_ip is collected. If http.user_agent is a concrete field (not in a map), we can vary its collection just like we do for client_ip: only collect it if the field is present in the schema.
Pros:
- less producer code to change
- more(?) explicit collection
- http.user_agent will show up in datahub and SQL UIs.
Cons:
- Not backwards compatible. Existent consumers of http.request_headers['user-agent'] will have to change the way they access this information.
- More code to change in e.g. Refine in order to parse user agent.
Option 3. Declare user-agent property field in http.request_headers map
Event Platform has a convention for declaring map type fields, and specific fields in a map can be pre-declared.
We could remove support from eventgate-wikimedia for setting undeclared fields in a JSON object, and always rely on presence of declared field in schema to decide if a header value should be set in the event.
Pros:
- Less stream config to deal with
- Setting of value is explicitly managed by schema
Cons:
- Less specific configuration support
- Requires schema changes to enable/disable collection, and only backwards compatible changes are allowed. I.e. it would not be allowed to disable collection after it is enabled (since technically removing fields is backwards incompatible). Or, we could add compatibility exception for removing fields from a map type, since this is practically fine.
- Using schemas for 'configuration' is weird.