Good idea! https://wikitech.wikimedia.org/w/index.php?title=Test_Kitchen%2FConduct_an_experiment&diff=2468029&oldid=2465909

• An announcement of support for the attribute is written and posted

Thanks, @Sfaci. That all makes sense.

For completeness, there are no on-wiki mentions: https://global-search.toolforge.org/?q=mw.%28testKitchen%7Ctk%29.compat&regex=1&namespaces=2%2C4%2C8&title=%28Gadgets-definition%7C.*%5C.%28js%7Ccss%7Cjson%7Cvue%29%29

Also, we can also inspect the image we have built via docker run -it --entrypoint=bash <name_of_the_docker_image> to check that required changes/files are where they should.

At the moment, the above patch sets up the ExperimentManager instance returned by ExperimentManagerFactory#newForUser() to honour overrides for local development and QA. I think this makes sense 🤔

Same here. EventDataProducts is descriptive without being overly constraining.

In T434137#12383583, @KCVelaga_WMF wrote:

I see an error in browser console when trying to trigger events on https://developer.wikimedia.org/ - not sure if events are reaching the EventGate:
POST https://developer.wikimedia.org/ins-502b/v2/events?hasty=true 404 (Not Found)

There haven't been any event validation errors since 5 PM yesterday 👍

https://logstash.wikimedia.org/goto/0ff4c6732423a8631a1f6768321fa717 shows event validation errors for events flowing on the product_metrics.web_base stream, the stream that the instrumentation is sending events to. If there's a problem with the event creation code, then we'll find out quickly.

Per https://gerrit.wikimedia.org/g/operations/mediawiki-config/+/fd1d7e89f585a5174cf8413dc3e8e61119648ee6/wmf-config/CommonSettings.php#4613:

In T438350#12369515, @brouberol wrote:

There might be a world in which the fundraising events live in the data lake and all of this is moot.

In T437414#12360816, @JVanderhoop-WMF wrote:

Proposed implementation: follow the same pattern in T439137: editSaved should record editing interface: create a client-side hook that fires when an edit is successfully published via wikitext editor, and editSaved should pick that up.

I'll create a DNM patch that will allow folks to test this during their review.

Copy over the custom fields and attributes for now.

In T438351#12333855, @brouberol wrote:

@phuedx do you have any idea/preference in terms of project name? The one project we have atm is called "Wikimedia". We could call this one "Fundraising" I guess?

The actual data pipeline importing FR data into the database should probably be left to the XP team. WDYT?

Perfect 😍

In T437330#12303823, @Daimona wrote:

I think that would be either a hook handler anyway, or a conditional in ServiceWiring, no? Which would leave us with a similar problem to overriding the config setting.

In T435898#12326690, @Sfaci wrote:

In T435898#12325828, @cjming wrote:

Pre-register all Custom Hooks, manually via GrowthBook UI, as empty snippets to get their corresponding id
Include as a comment that id in the first line of every snippet in the repo: something like // id: hook_2CesebMuSr7WL752FDRszU

Wouldn't ids be different between production and staging since they are different GrowthBook instances? I suspect each hook would get a different id in each environment. Should we do lookups by name instead? GB API has a get all custom hooks list endpoint (https://growthbook-ea15456d.mintlify.app/api/custom-hooks/operation/listCustomHooks). Could CI do a lookup by name/slug and skip lookup by ids? This way too we wouldn't have to embed ids in comments - this seems kinda fragile to me.

Then I think CI can patch the pre-UI-registered hooks?

You are right! Putting that comment is pretty fragile and also, as you have mentioned, the id will be different depending on the environment. That wasn't a good idea!. Also, I think it would be great if we were able to manage the configuration for every custom hooks also via MR, for example to be able to enable/disable them when needed.

Using that GET endpoint to get all the custom hooks and lookup by name/id sounds good, but I wonder if we will be overcomplicating the thing considering that the endpoint list all hooks from all projects. Based on T436792: Update Test Kitchen UI API to allow querying by project and some related discussion we had already, it seems we will have different projects in GrowthBook and I guess that we could have different Custom Hooks or validation depending on that project. That GET endpoint doesn't filter by project and the value used by the endpoint is the project id (not user friendly) so we should keep that in mind to filter explicitly in the CI.

That's why I was wondering if we could go with a different approach by adding a config file, per project, to the repository: config.yaml, for example, with all the needed configuration for all hooks. This approach also assume that we will create all custom hooks first as empty things in GrowthBook UI. Each file could be as follows:

<snip />

@cjming: Maybe we should sync on this? I wrote T437240: Add Test Kitchen functionality to WikimediaDebug with this in mind but maybe there's other ways

@brouberol: We've been bold and assigned this task for you for work breakdown.

In T392997#12313492, @JVanderhoop-WMF wrote:

@phuedx is this to say if we have Kotlin and Swift SDKs, or if we want to continue supporting Android and iOS experiments with their current enrollment methods?