Page MenuHomePhabricator

Create Model Card for Search MLR
Open, Needs TriagePublic

Description

The ML team is working on having model cards describing the various ML models we have in production. Those cards are all based on the same template and provide better transparency on how ML is used at WMF.

To increase the transparency of how ML is used in the context of Search, it might make sense to document the MLR models in the same way.

The template seems to be more oriented towards models that can be reused for other applications (which is the case of the models deployed on LiftWing). The Search MLR models are very specific to Search and make no sense to be reused outside of it. So the needs for documentation are slightly different in this case.

Note that while we have multiple language specific MLR models, they are all similar enough that they should be documented by a single Model Card.

AC:

  • decision: do MLR models benefit from the same Model Cards
  • if yes: a model card exists for the MLR models

Event Timeline

The template seems to be more oriented towards models that can be reused for other applications (which is the case of the models deployed on LiftWing). The Search MLR models are very specific to Search and make no sense to be reused outside of it. So the needs for documentation are slightly different in this case.

@calbon , do you have thoughts on the priority of doing a Search ML Ranking model card is given the above? (Not that we won't do it, more so about the timing of it)

I think the priority can be low for you all. We are just getting starting with some other model cards @kevinbazira is making. Once we have a better idea how we want to do it, then expanding to other models makes sense.

The goal isn't to make it possible for other people to reuse the models, the goal is for the community and other stakeholders to have a place where they can understand what AI is affecting their Wikipedia experience and have a venue to learn/discuss/understand/debate/critique that.

calbon raised the priority of this task from Low to Needs Triage.Nov 2 2023, 5:26 PM