As I understand this task, we are aiming to build sound AI governance through open discussions with various stakeholders. I think this is much needed to allow us to scale our AI services and continue to be world leaders in community-driven AI.

I do have a few questions regarding this process.

1. Will these discussions be asynchronous?
2. Will there be a public record of these discussions?
3. How technical are these discussions going to be?
4. Are we planning to hold a discussion with various wiki communities that have deployed or wish to deploy Machine Learning solutions in their wiki?

I think the answers to these questions can help push this task forward.

Currently there are no set of policies in place that candidate models (internally and externally) must meet in order to be deployed. This is highly problematic.

I think it's equally if not more important to have a maintenance protocol for identifying and addressing issues post-deployment. Most important issues don't show up in fitness statistics. They are hard to understand pre-deployment. Yet we have found that deploying the model and purposefully trialing the model with users allows us to catch even subtle issues.

So I might encourage us to not just think about what qualities we want to ensure before deployment but rather to focus on how issues will be identified and addressed in collaboration with users and issues will be addressed post deployment.

This has strong implication for product process. E.g. before a new product is deployed coupled with some ML model, the model should be deployed in a trial fashion so that users (usually volunteer contributors, but obviously readers might be another) can review the behavior and highlight issues. There also would need to be a process (and funding) for addressing issues that appear post-deployment. Who is responsible for fixing bugs with a model? The ML team? A researcher who developed it? The product team whose feature it supports?

I think it would be helpful to determine what the end result might look like here. The outcomes I would like to see include:

1. A process for determining the suitability of any given model. (checklist or risk-assessment app)
2. A matrix that contains all models against governance considerations. Each cell would identify measures required for deployment/maintainance/etc.

Here are some high-level governance considerations that we should, at the very least, be able to answer about every model on our new infrastructure:

Reproducibility & traceability

• is there a full snapshot (vm + data?) for model re-instantiation?
• provide ability to recreate environment and retrain w/ sample data?

Audit & Documentation

• full log of changes during development (experiments run, rationale for decisions made, etc.)

Human-in-the-loop sign off

• multiple stakeholders should sign-off when going from dev->test->deploy (not just engineers)
• if the model is for a wiki community, we need a local contact and proof of community consensus before deployment.

Preprod verification

• automated checks
• unit tests
• e2e tests

Transparency & Explainability

• Can we provide an explanation for non-experts on how it works?
• Trees can give maximum explainability (LIME etc.)
• SHAP etc. for regression models
• neural networks are still opaque - can we give any explanation?
• what features are we using? why are they important?

Bias and harm testing

• Adversarial "red team" testing
• automated bias checking on specific models?

Production deployment reqs

• containerized deployment with scalable high-availability.
• automated stress/load testing prior to deployment
• what kind of maintenance/SLA/SLO needs are required?

Production monitoring

• real-time error alerting
• model balancing
• automated nightly retraining
• model evaluation w/ redeployment loop
• weekly drift monitoring
• also allow for manual retraining and basic alerting

Data quality and compliance

• PII considerations
• can we understand the source/quality/appropriateness of data?
• automated data quality checks?
• outlier detection/anomalies?

@ACraze completely agree on the parameters you have proposed.

I understand there will be an initial overhead of setting up these up for the various types of models we wish to deploy on the new infrastructure, but this is definitely worth investing in.

Could you also look into my previous comment and see if maybe we could get some answers to those questions? Now that we have an idea of what the end result looks like, we can start deliberating on the process we use to achieve it.

@Chtnnh - some answers for your questions

Will these discussions be asynchronous?

Yes, the community discussions should be async to allow everyone a chance to speak. The internal discussions are still informal at the moment, although we will update the sub-tasks as details begin to emerge.

Will there be a public record of these discussions?

Absolutely, we will be discussing in the sub-tasks (i.e. T276604) as the conversations progress, so do subscribe to them if you are interested in learning more. We will also have a talk page open for discussion when the first draft is ready.

How technical are these discussions going to be?

I think there will be different tiers of conversations to address different concerns. There should be high-level talks around process, procedures and implications. There will also need to be some more technical discussions around implementation and tooling.

Are we planning to hold a discussion with various wiki communities that have deployed or wish to deploy Machine Learning solutions in their wiki?

Yes, eventually we will need to do this, although currently we are discussing with internal teams, as they will be the first to deploy models on our new architecture. Going forward, we need to have a local contact and proof of community consensus before deploying any ML solutions for a wiki community. There is some question about how best to do this, do we dump information on each wiki's talk pages and hope they want to discuss with us in English? Do we setup a committee to oversee this work? This will be a large, yet important undertaking and we still need to think about how best to achieve this.