Please provide all the following information:
Context
The Trust and Safety Product team has a KR (WE4.5) that assesses the risks and opportunities of ML/AI on Trust & Safety aspects of Wikimedia’s ecosystem. The recent "Human Rights Impact Assessment of the WMF’s AI/ML Products" highlights the potential marginal human rights risks of generative AI, which includes areas relating to Trust and Safety, such as harassment: “The primary threat is that GenAIs could be used at scale to generate harassing content targeting Wikimedia volunteers, readers and staff.” (p.31)
In Q1, we would like to gain qualitative insights into how bad actors are using generative AI for the purposes of abuse on the wikis, which will enable us to assess the risk to the wikis and generate ideas for mitigating genAI assisted abuse. This research would enable the team to run experiments in mitigating identified risks in Q2.
Description
Run qualitative research to identify wiki examples of abuse from bad actors assisted by generative AI. This might include spam, harassment, long term abusers, undisclosed paid editing, or disinformation campaigns. The outcome can be used to:
- Assess the risk of generative AI assisted abuse to our community models.
- Generate ideas to mitigate different types of generative AI assisted abuse.
Expected Deliverable
- This is a qualitative research project, e.g. talking to CheckUsers/functionaries to learn if highly motivated LTAs are using genAI to try to defeat administration sanctions, or incorporating genAI into their tactics.
- Output could be case studies, such as any examples of LTAs using genAI. Ideally the case studies would include a high-level assessment of the risk to wikis and recommendations of where to focus our mitigation ideas.
Estimated Effort
We can scope this work according to available resources. It could be a 3-4 week project. This is a lower-priority task and could be scoped down to only include descriptions of existing cases where genAI is negatively affecting community health, without a detailed risk assessment.
Priority
I need this task resolved in:
- 1 month.
- 3 months (Q1 Jul-Sep)
- 6 months.
- Whenever you get to it :-)
Ideally this task would be done in the first half of Q1, to allow us to act on the insights for the remainder of Q1 (i.e. formulate experiments of genAI abuse mitigation ideas).
For use by WMF Research team; please leave everything below as it is:
- Does the request serve one of the existing Research team's audiences? If yes, choose the primary audience. (1 of 4)
- What is the type of work requested?
- What is the impact of responding to this request?
- Support a technology or policy need of one or more WM projects
- Advance the understanding of the WM projects.
- Something else. If you choose this option, please explain briefly the impact below.