Page MenuHomePhabricator

LLM that specializes in assisting Wikimedia/MediaWiki technical contributors
Open, MediumPublic2 Estimated Story PointsFeature

Description

Feature summary (what you would like to be able to do and where):
Do you know if there are any plans for our machine learning team to train an LLM to assist Wikimedia technical contributors? This would involve training the LLM using MediaWiki code, gadget and userscript code, MediaWiki wiki, doc.wikimedia.org, Toolforge code, etc.

Use case(s) (list the steps that you performed to discover that problem, and describe the actual underlying problem which you want to solve. Do not describe only a solution):

Benefits (why should this be implemented?):
LLMs can be quite useful for software engineers since they are good at reading code and generating sample code, but LLMs on the market right now such as ChatGPT probably aren't very familiar with the MediaWiki framework.

And there is also a lack of MediaWiki questions and answers on StackOverflow and related sites, making google searches for MediaWiki-related topics difficult sometimes, and creating a niche that a MediaWiki-trained LLM could fill.

Event Timeline

calbon set the point value for this task to 2.
Susannaanas subscribed.

This is a really great suggestion! ChatGPT does not know enough and gives bad suggestions, and humans aren't available.

Update: Right now we don't have the resources to prioritize this. I'm moving it to the backlog.

Another possible feature - which we may want to split from this - is having an LLM that translated natural language questions into Wikidata,

This is an active research area, as of today (february 2025), here are some recent papers:

  • A dedicated dataset to train text-to-SPARQL models to be used on Wikidata. Strappazon, A., Granitzer, M., Egyed-Zsigmond, E., Mitrovic, J., & Amor, M. B. (2025, March). "Instruct-to-SPARQL: A text-to-SPARQL dataset for training Wikidata Agents." In ACM SIGIR Conference on Human Information Interaction And Retrieval. ACM. https://hal.science/hal-04918564/document
  • Emonet, V., Bolleman, J., Duvaud, S., de Farias, T. M., & Sima, A. C. (2024). "LLM-based SPARQL Query Generation from Natural Language over Federated Knowledge Graphs." arXiv preprint arXiv:2410.06062. https://arxiv.org/pdf/2410.06062
  • Meyer, L. P., Frey, J., Brei, F., & Arndt, N. (2024). "Assessing SPARQL capabilities of Large Language Models." arXiv preprint arXiv:2409.05925. https://arxiv.org/pdf/2409.05925
  • Brei, Felix, Johannes Frey, and Lars-Peter Meyer. "Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistance." arXiv preprint arXiv:2405.17076 (2024). https://arxiv.org/pdf/2405.17076