Collect feedback from model developers on draft Human Rights AI Checklist. I'll aim to get feedback from Research, ML Platform, and at least one other model developer outside of these two teams.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | Isaac | T348328 Ethical AI Support | |||
| Resolved | Isaac | T348330 Human Rights AI Checklist Feedback |
Event Timeline
Comment Actions
Go-ahead from Human Rights folks to begin testing the model-specific aspects of the checklist. If I get a chance, I'll do some of this at the offsite as Diego and Aiko will both be there and are excellent candidates.
Comment Actions
Didn't have a chance at the offsite as we were too busy with other work to want to sit down and go through these but RG sent over a new version of the checklist that I'll be reviewing next week and begin scheduling meetings with individual researchers / ML developers to go through the model part of the checklist and collect feedback.
Comment Actions
- Began collecting feedback from Santhosh and Martin. Already good thoughts on interpretation of some of the questions and pieces that might be missing (around model continuity).
Comment Actions
Weekly update:
- Finished receiving feedback from Martin. I need to sit down and start synthesizing what has been shared and schedule a run-through with Aiko or someone from ML Platform.
Comment Actions
(Thank you for your work on this front. I'm moving this task to the Staged lane because I want to close Q2's lane and there is no recent update in the comments. Please move it to the right column.)
Comment Actions
Whoops thanks (I thought I'd closed this). Feedback passed back to team. Full feedback in [https://docs.google.com/document/d/1SOjFUkRClpssyZoBNNoAQaAksbvyz9RegY76i9Ni-jQ/edit?usp=sharing this google doc] but amounted to mixture of question-specific wording to improve clarity, raising the question of how to handle models that are a mixture of pre-trained language model + fine-tuning, and question about how to address maintainability of model in the checklist. I also have a TODO to propose guidance around what "open-access" means for ML models to Legal so we can give more specifics to folks there too -- i.e. what elements of a model (training data, training code, outputs, etc.) need to be explicitly licensed and how. That will happen separately.