Thanks for adding the task, I'd be happy to go for it!

Looking at the hits of Special:AbuseFilter/1346, it looks like only ChatGPT-generated links are getting caught here. @DLynch already looked into ChatGPT's behavior earlier today, finding that it would only generate UTM links through its web-search tool, but not through a regular "ask to write an article" query.

I tried myself with the other popular LLMs:

• Microsoft Copilot doesn't seem to ever generate links with a UTM tracking parameter.
• Gemini didn't either, but did generate one of its links in an odd format (https://www.google.com/url?sa=E&source=gmail&q=https://www.science.org/doi/10.1126/sciadv.1700715). Gemini's "Flash Thinking Experimental" model, which was connected to Google Search (and thus the closest equivalent to ChatGPT's web search tool that David mentioned), also didn't generate any UTM links.
• Grok didn't provide any links at all without explicitly asking it to do a web search, and, when doing a web search, still didn't generate anything with UTM parameters. I hit the 3 query limit so couldn't go much further with Grok 3. When asking Grok 2, the model answered that it wouldn't usually generate UTM parameters, but could add them to the links it generated previously on my request (and spontaneously gave me https://www.smithsonianmag.com/smart-news/enormous-stegosaurus-skeleton-called-apex-smashes-auction-records-and-sells-for-446-million-180984757/?utm_source=grok&utm_medium=response&utm_campaign=apex_info). So, it looks like Grok is technically capable of adding UTM parameters but just not trained to do it.
• Claude replied that it couldn't search the Internet, and was thankfully transparent on the fact that all the citations it generated were fictional (none were hyperlinked at all).
• Groq (not to be confused with Grok above!) was also linked in the edit filter. It isn't a model itself but a platform for inference using other models. Trying with Llama and Mistral, neither could perform a web search or return URLs.

TL;DR: Only ChatGPT seems to generate UTM links, as seen from our edit filter hits as well as from experimenting with other LLMs. Not sure if this can be marked as resolved yet or if anyone is interested in going deeper.