This definitely needs a review from security before we can proceed.
This will also likely need specific triggers/views.

But first, we need to get an indication on:

• tables that can be replicated entirely as they are
• tables tha cannot be replicated at all
• tables that need some redactions
• tables that require some specific views

we need to get an indication on

Who is best placed to provide the answers here? I'm looping in Data-Engineering as well.

In T395881#10878832, @fnegri wrote:

we need to get an indication on

Who is best placed to provide the answers here? I'm looping in Data-Engineering as well.

Privacy subteam of security.

@fnegri can you remind me what initiative this supports? Better yet add the context to the description. If there are relevant deadlines or plans, please include those as well. Thank you!

@VirginiaPoundstone I updated the task description with more context. I will also ask in the user-created parent tasks if they have specific use cases why they would like to have this section, or if it's just for the sake of completeness.

A use case for having x1 in Wiki Replicas was discussed in T387419: Create wiki replicas views for globaljsonlinks tables: being able to "generate statistics and usage information about charts". The tables used by the Chart extension are in the x1 section.

Tagging Data-Platform-SRE to get their input.

I created T407485 to track the work required to add this section to an-redacteddb1001 and set up the initial replication.

I'm conscious that @Marostegui has said this, back in June.

In T395881#10878825, @Marostegui wrote:

This definitely needs a review from security before we can proceed.
This will also likely need specific triggers/views.

But first, we need to get an indication on:

• tables that can be replicated entirely as they are
• tables that cannot be replicated at all
• tables that need some redactions
• tables that require some specific views

When you say:

tables that cannot be replicated at all

...do you mean that certain table might be entirely filtered out of the MariaDB replication so that it never leaves the upstream x1 replica?
Or do you just mean that there is no view of these tables defined at all?

Is it safe for me to start defining a puppet resource for this x1 section on an-redacteddb1001 now, or do we still need to wait for a privacy/security review before taking any action?

It's worth noting that because of the table catalog (T363581), now all tables that should be fully filtered or are fully public can be easily determined and already will be filtered when someone sets up the replication (We did catalog x1 tables too: T399302). That leaves only updating maintain-views for partially public tables.

In T395881#11280914, @BTullis wrote:

I created T407485 to track the work required to add this section to an-redacteddb1001 and set up the initial replication.

I'm conscious that @Marostegui has said this, back in June.

In T395881#10878825, @Marostegui wrote:

This definitely needs a review from security before we can proceed.
This will also likely need specific triggers/views.

But first, we need to get an indication on:

• tables that can be replicated entirely as they are
• tables that cannot be replicated at all
• tables that need some redactions
• tables that require some specific views

When you say:

tables that cannot be replicated at all

...do you mean that certain table might be entirely filtered out of the MariaDB replication so that it never leaves the upstream x1 replica?

Correct

Or do you just mean that there is no view of these tables defined at all?

We've had both cases, tables that we may be okay at some point and tables that we are 100% sure will never be allowed to have views on and hence we even exclude them in the replication thread.

Is it safe for me to start defining a puppet resource for this x1 section on an-redacteddb1001 now, or do we still need to wait for a privacy/security review before taking any action?

I think we can start puppetizing this, but we definitely need a security review before proceeding with any data movement.