@VirginiaPoundstone It seems that there has been some activity on this ticket recently. Can you please give me an update on what was discussed and what the outcome was?

@mforns, @Ottomata, @phuedx, @VirginiaPoundstone This ticket is stalled. Is there anything we can do to move it forward?

@kai.nissen I pinged the Metrics Platform team to get an update.

Unless they advise otherwise, you could move forward and create your event schema and stream as needed.

@kai.nissen so sorry for having missed all the pings for so long.

About 18 months ago, we decided to change the existing metrics platform "monoschema", to get rid of the custom_data field.
I believe that having such a field is not a good data engineering practice, because of the reasons detailed in this document.
We designed the new set of fragments and schemas to offer an alternative to the custom_data field.

I understand that the new fragments and schemas do not fulfill your requirements.
In this case, my advice would be to:

1. Create a fragment for your project. In it, you will be able to add whichever extra fields you need in different experiments. Whenever you create a new experiment, you can add new fields there if necessary, without the need to create another schema. That said, please do not use a custom_data field, but rather create explicit fields with types and descriptions for each concept you want to store.
2. Create an "empty" schema for your project that just references the fragment that you created in step 1, and also references the corresponding base fragments. This way you get all the standard structure from the base fragments, and the customization from your own fragment.

That said, I'm not sure this is the preferred way? Maybe @phuedx has different ideas?

In T336842#10377159, @mforns wrote:

That said, I'm not sure this is the preferred way? Maybe @phuedx has different ideas?

(1) is my preference.

Hehe, 1 and 2 were intended to be a sequence, rather than 2 options. Sorry, I think my question in the end was misleading.
But knowing that you are OK with (1), 2 is just the application of that in the form of a schema (not a fragment).
And to clarify, the reason to structure the fields in a fragment and not in a schema directly, would be to be able to re-use the fragment for app vs web base schemas.

@kai.nissen we also have some documentation about how to make a custom schema fragment which might also help: https://wikitech.wikimedia.org/wiki/Metrics_Platform/Custom_schemas

MR for schema: https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-secondary/-/merge_requests/57

Event Stream PR: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1148322

Documentation PR: https://github.com/wmde/fundraising-banners/pull/779

@mforns Could you take a look at Abban's change/merge requests on Gerrit and Gitlab, or suggest someone I could assign as a reviewer?

@kai.nissen Hi!
I've reviewed the changes.
I think at this point, it would be good that someone from the Experimentation Platform team reviewed this change, since I'm not fully aware of the latest changes in that system.
I added @Sfaci as reviewer 🙏

Hello! I'm our new PM on the experiment platform team, and I'm just back from vacation. I'll take a look at this and the conversation on the patch this week. Any context you'd like to provide is helpful!

@kai.nissen I'd love to understand more about the context behind this new schema and what you're looking to achieve in the bigger picture, so that we can best support or advise. Would you be available for a quick 15-30 minute chat in the next few days? If you're not the right person to discuss this with, I'd appreciate being connected with whoever is.

Thank you!

Hello @kai.nissen @AbbanWMDE bumping a request to connect! jvanderhoop@wikimedia.org :)

@mforns Can you give this PR another look over? I previously changed it to fit into the xLab platform, but the fundraising team decided to not use xLab for now so I reverted it again. I'd love to finally get this off my plate.

Hi! I added a few nits to your schema MR. :)

@AbbanWMDE OH NO! I never submitted my review comments. They were stuck in Pending on Gitlab. I'm so sorry!

I just submitted them.