Wikipedia is increasingly receiving edits that contain AI-generated and blindly machine-translated content. For this there is need for methods for automatically identifying such content and edits. The task is suitable for multiple persons to do simultaneously and seek different solutions for the same problem.
**Examples from Finnish Wikipedia**
LLM-edited articles with hallucinated sources
- Example: https://fi.wikipedia.org/w/index.php?title=Koneoppiminen_bioinformatiikassa&diff=23412527&oldid=23260793
- Article was modified with LLM, including adding hallucinated citations
Machine-translated articles created simultaneously across multiple Wikipedias
- Example: https://fi.wikipedia.org/wiki/Harati-lukio
- Cross-wiki pattern indicating automated translation
- https://www.wikidata.org/wiki/Special:Contributions/5.124.168.171
LLM-generated text with unnatural writing style
- Example: https://fi.wikipedia.org/wiki/Isabel_Oakeshott
- Grammatically correct but lacking human writing characteristics
**Tracking categories**
- English Wikipedia: [[ https://en.wikipedia.org/wiki/Category:Articles_containing_suspected_AI-generated_texts | Category:Articles containing suspected AI-generated texts ]]
- Finnish Wikipedia: [[ https://fi.wikipedia.org/wiki/Luokka:Artikkelit,_jotka_saattavat_sisältää_tekoälyn_tuottamaa_sisältöä | Luokka:Artikkelit, jotka saattavat sisältää tekoälyn tuottamaa sisältöä ]]
**Task**
Investigate how LLM generated content detection could be done. Provide a proposal or results of the investigation in the comments, and we can write an actual task ticket based on the investigation.
Investigation can focus on single topics such as:
- Methods for detecting hallucinated citations
- Cross-wiki edit pattern analysis approaches
- Detecting if text is directly translated from another Wikipedia
Also note for implementation proposals: Only open source tools should be considered. Closed source LLMs are out of scope.