While now it needs an external LLM, there are plans to host LLMs internally T418350, this could be a great use case.

Btw it's def a good idea to add the article title and paragraph text to the context. One of the failure modes I encounter is when a claim "In 2026, she appeared on the game show Countdown in dictionary corner.[12]" is classified as "Not supported" because the model has no idea who "she" is.

Thanks for sharing the updates. Looking forward to the results!

Deploy the model-server in the LiftWing production namespace to provide an internal production endpoint for wider use.

Re the tensions between performance and nuance

Thanks!

@sbassett , could you whitelist the Anthropic, OpenAI and PublicAI endpoints? There are more scripts that stooped working, see https://en.wikipedia.org/wiki/Wikipedia_talk:WikiProject_AI_Tools#c-Phlsph7-20260308103900-AI_scripts_not_working_anymore_following_CSP_changes

Great news! Running it like Suggestion Mode would be amazing. Is it something that WMF infrastructure can support now? This would really move the needle. The reach of user scripts is limited (mine has 14 active users).

api.publicai.co (see the documentation here https://platform.publicai.co/docs)

specifically api.anthropic.com and api.openai.com

Thanks for the quick turnaround. I'm happy to confirm that the tool works now... partially. I forgot that I also made calls to Anthropic and OpenAI. Since @Polygnotus also mentioned them, I think it would be good to whitelist them too.

I think it's quite likely that many scripts have been affected. I understand that Wikipedia:WikiShield (an anti-vandalism tool with 70 active users) also no longer works, @LuniZunie - please confirm

I'd suggest renaming this task to something like "Support user scripts accessing external data". the current title frames whitelisting as *the* solution, while the right approach is still being discussed.

@Polygnotus thanks for you comment. On the second though I think that you're right and toolforge is not a good solution.

I think that hosting on toolforge could be a good medium-term solution as many other applications need to fetch website contents or access open-souece LLMs

I've fixed the logging so now we'll start collecting data.

@Isaac thanks for sharing the statistics and impressions. Good to get a confirmation that this is solving a real problem for editors reviewing recent changes or articles for creation. It's tempting to imagine an edit filter but it needs to be considered carefully - there will be false positives and errors and we don;t want to discourage new editors. Another modality is a bot that flags citations that fail verification by adding tags or by posting at the talk pages.

Thanks for the detailed response

@Isaac , the short answer to your second question is that everything that I couldn't classify as "Supported" or "Not Supported" ended up as "Partially supported"

Re 1. it's certainly feasible technically. I thought about it but didn't do it to avoid having to think about privacy. I only see the number of requests to my proxy, it's about 20-30 per day and I'm not entirely sure they are all real. I'd rather not make it opt-in since most people won't click on it, and given the total volumes we'll hardly get any data. Do you think that collecting data by default would be okay if I don't store any PII-adjacent stuff (no user names, only verification results) and add a disclaimer to the UI? Or maybe you have better ideas how to solve this.

Thanks for sharing the results, very interesting!

I've tested a few models and written up the results here: https://github.com/alex-o-748/citation-checker-script/blob/main/Citation%20Verification%20-%20LLM%20Benchmarking.md

I've been using AI source verification (one of the tools mentioned above) and also built a standalone version for non-wiki editors https://wiki-cite-checker.replit.app/

In T336905#11452620, @Sj wrote:

And we are making Apertus and Olmo-Think and other fully public models available via API for free (though with rate limits to prevent saturation) vua the Public AI Inference Facility https://chat.publicai.co

Speaking of use cases, there are quite a few tools here https://en.wikipedia.org/wiki/Wikipedia:WikiProject_AI_Tools#Projects. For now they require API keys and hence their adoption is very limited. If some kind of LLM credits were available to established editors that would've been awesome!

thanks!