Change Details

WMF deploys models from a wide variety of sources, both internally and in the community. Currently there are no set of policies in place that candidate models (internally and externally) must meet in order to be deployed. This is highly problematic. We don't know who wants the models, how they have been trained, is their potential for data leaking PII, is their a risk of bias, how does a community contest a model's predictions etc. This poor model governance goes against our values of transparent, accountable, and ethical ML. We need to create a model an initial draft of model deployment guidelines, invite comment both from the foundation teams and the community, and then hold ourselves to the guidelines going forward.

WMF deploys models from a wide variety of sources, both internally and in the community. Currently there are no set of policies in place that candidate models (internally and externally) must meet in order to be deployed. This is highly problematic. We don't know who wants the models, how they have been trained, is their potential for data leaking PII, is there a risk of bias, how does a community contest a model's predictions etc. This poor model governance goes against our values of transparent, accountable, and ethical ML. We need to create a model an initial draft of model deployment guidelines, invite comment both from the foundation teams and the community, and then hold ourselves to the guidelines going forward.

WMF deploys models from a wide variety of sources, both internally and in the community. Currently there are no set of policies in place that candidate models (internally and externally) must meet in order to be deployed. This is highly problematic. We don't know who wants the models, how they have been trained, is their potential for data leaking PII, is theirre a risk of bias, how does a community contest a model's predictions etc. This poor model governance goes against our values of transparent, accountable, and ethical ML. We need to create a model an initial draft of model deployment guidelines, invite comment both from the foundation teams and the community, and then hold ourselves to the guidelines going forward.