This is now done in citoid, however note that for outbound requests, fetch in node versions <= 24 does not use the correct protocol for http when contacting it through a proxy.

Marking this as declined as it doesn't seem any work is planned for this. Feel free to re-open if this changes.

In T427333#11973343, @Geekgecko wrote:

I did email Taylor and Francis about the error, and they said there wasn't anything they could do on their end

In T428420#11993112, @Lina_Farid_WMDE wrote:

@Mvolz I've added the link to the ticket.
https://www.pnas.org/doi/pdf/10.1073/pnas.2108146119

What url was this specifically?

This is a bug, and we do have some tests for it. (I.e. in no-zotero.js test file).

https://www.pnas.org/doi/pdf/10.1073/pnas.2108146119 is actually a regression unfortunately.

In T427333#11964039, @Geekgecko wrote:

Hm thanks for letting me know, what might the fix for this be then?

In T425924#11924318, @Lina_Farid_WMDE wrote:

Thanks! I see that there are many successful attempts (200s) for PDF sources, see below. Do we have an understanding why these result in a successufl retrieval of data ?
The mapping of data is definitely not great, though.

Sucessfully scraped resource at https://web.archive.org/web/20260515072614/https://s3-crs-prod.microplustimingservices.com/RHRU2025/static/_PDF/WUG2025/ATH/2025-07-27/ATHM4X400M------------FNL-000100--__C73E_1.0.pdf 200

Looking into this a bit more it's the website itself that's telling us it's python when we do a request with our particular user agent string. It's kind of a weird bug that happens when I guess something about our user-agent doesn't parse?

This was caused by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1281765 .

The successful applicant should claim the phabricator task I linked above, so no one else works on it in the meantime, thanks!

Note there is an existing phabricator task for wishlist item #3 :

We currently race for crossref but we might want to wait and return both options so people can choose whichever citation "turns out" better.

In T420397#11748180, @Elya wrote:

Update: the techies at the Hamburg Museum added exactly the stuff you commited upstream, and it works - thanks for the heads up! I hope it will be inherited by other local allow lists … 

Done upstream: https://github.com/TecharoHQ/anubis/commit/24857f430f5236edd770c7ae6377eeda21a21606

In T419903#11731179, @Soda wrote:

The tool is behind OAuth2 now; in theory, unintended requests should drop off a cliff. I'll keep monitoring :)

In T419903#11727748, @Soda wrote:

In T419903#11719778, @Samwalton9-WMF wrote:

In T419903#11719512, @Mvolz wrote:

I quickly had a look to see if we can find out where the extra traffic is coming from and it seems to be toolforge, specifically https://link-dispenser.toolforge.org/

Tagging @Soda - is this unusual level of activity or kind of of activity for your tool? (I.e. is this potentially abuse from a user? Or just normal activity?)

Regardless of the activity of the source, perhaps it would be a good idea for link-dispenser to have some internal list of known-good sources that it doesn't bother checking?

This does seem like usual levels of activity. That being said, the tool does not require Auth at the moment. I'll implement that ASAP and see if that helps.

In T419903#11719734, @Mvolz wrote:

This amount of activity is more than 5 times the amount we normally get from all sources and it's not just newspapers.com, this is all websites.

Can't see the image... but some of this should be expected traffic. The tool queries Citoid to resolve the title and determine whether a particular source is hallucinated. If the Citoid title and the title on the citation is dissimilar then we flag it as "potentially LLM generated".

Our IP range is: 208.80.152.0/22 for IPv4 and 2620:0:860::/46 for IPv6

This amount of activity is more than 5 times the amount we normally get from all sources and it's not just newspapers.com, this is all websites.

I quickly had a look to see if we can find out where the extra traffic is coming from and it seems to be toolforge, specifically https://link-dispenser.toolforge.org/

Tagging @Samwalton9-WMF and @sjvipin

In T242089#11717193, @Johannes_Richter_WMDE wrote:

There's currently a discussion about this on German Wikipedia: de:WP:FZW#Fehler bei Einfügen von Links im Visual Editor

How about showing some warning that the URL output doesn't match their entered URL, asking users to consider checking the reference to make sure if it's correct before saving their edit?

In T416471#11683293, @Jdforrester-WMF wrote:

Note that the Node 24 image will now be available from Monday; maybe just jump to that?

In T343020#11648798, @colewhite wrote:

In T343020#11583194, @Mvolz wrote:

However it has now received code review and is just waiting on response from the submitter, so it probably no longer needs to be in that column.

Hi! Looks like that submitter is me!

As mentioned on the patch, I'm not terribly comfortable writing JS and I don't have a clear idea of how to move forward based on David's comment. Further, I don't know who owns the code or is interested in preserving these metrics. The metrics ingest for these timers have been broken for about a year now - Should we consider switching tactics from a metrics migration to cleaning up old code?

Just FYI, but I had a quick look at the citoid mapping for Cite web (used as _default and some other ones instead of Template:Citation on en) and it's pretty barebones. You might want to consider adding some more mappings; zotero has a lot more fields than the templates do so for rare item types, they might not be mapped.

In T416080#11583504, @Johannnes89 wrote:

In T416080#11583286, @Mvolz wrote:

In T416080#11571730, @Johannnes89 wrote:

This doesn't seem to be limited to Safari or mobile browsers. Steps to replicate the issue (even in desktop browsers):

• Open any (enwiki) article in VisualEditor and use Citoid to automatically create a reference from any URL (e.g. https://www.bbc.com/sport/football/articles/cj6w3wr9x2lo)
• Observe that the wikitext output doesn't respect the parameter order (and the space between different parameters) defined in TemplateData of en:Template:Cite web

<ref>{{Cite web|title=Rafaela Pimenta talks transfer system, gender inequality and more|url=https://www.bbc.com/sport/football/articles/cj6w3wr9x2lo|website=BBC Sport|date=2026-02-01|access-date=2026-02-01|language=en-GB|first=Simon|last=Stone}}</ref>

• Switch to 2017 Wikitext editor and use Citoid to create the same citation
• Observe that the wikitext output matches the format defined in TemplateData

<ref>{{Cite web |last=Stone |first=Simon |date=2026-02-01 |title=Rafaela Pimenta talks transfer system, gender inequality and more |url=https://www.bbc.com/sport/football/articles/cj6w3wr9x2lo |access-date=2026-02-01 |website=BBC Sport |language=en-GB}}</ref>

Can you tell me which desktop browser you're using and ideally the version?

I've tried both Chrome and Firefox and I'm unable to replicate this using the 2010 editor.

I've used Chrome and Safari on MacOS and iOS. But I can no longer reproduce the issue.
Perhaps the backport in T415328 this Monday to fix T411238: Unexpected wikitext changes & whitespace removals by VisualEditor edits and T416074: VisualEditor capitalizes wikilinks in ref content when ISBN magic links are present also fixed the issue reported in this task?

Hi @Reeti, I think you've misunderstood the task.

In T416080#11571730, @Johannnes89 wrote:

This doesn't seem to be limited to Safari or mobile browsers. Steps to replicate the issue (even in desktop browsers):

• Open any (enwiki) article in VisualEditor and use Citoid to automatically create a reference from any URL (e.g. https://www.bbc.com/sport/football/articles/cj6w3wr9x2lo)
• Observe that the wikitext output doesn't respect the parameter order (and the space between different parameters) defined in TemplateData of en:Template:Cite web

<ref>{{Cite web|title=Rafaela Pimenta talks transfer system, gender inequality and more|url=https://www.bbc.com/sport/football/articles/cj6w3wr9x2lo|website=BBC Sport|date=2026-02-01|access-date=2026-02-01|language=en-GB|first=Simon|last=Stone}}</ref>

• Switch to 2017 Wikitext editor and use Citoid to create the same citation
• Observe that the wikitext output matches the format defined in TemplateData

<ref>{{Cite web |last=Stone |first=Simon |date=2026-02-01 |title=Rafaela Pimenta talks transfer system, gender inequality and more |url=https://www.bbc.com/sport/football/articles/cj6w3wr9x2lo |access-date=2026-02-01 |website=BBC Sport |language=en-GB}}</ref>

In T343020#11576351, @ppelberg wrote:

@Mvolz: I'm boldly un-assigning the Editing Team from this task.

Note: please let me know if you think the above is unwarranted. I propose the above, admittedly, no clear about the thinking that led to you placing this ticket on the Editing Team board in the first place (T343020#10803633).

I can confirm these urls work locally, but in production we're getting 504 gateway timeout errors. It's been ongoing as long as we have data (3 months).

In T361576#11562523, @HCoplin-WMF wrote:

Following up on this comment (a few months late, sorry):

The only think we're currently using *in* restbase right now is the swagger documentation; I don't think we've decided where to host that, ultimately.

We actually recently moved the Swagger docs to the new REST Sandbox and implemented a redirect from the RESTBase docs. Announcement was in Tech News, for more info.

The citation section is currently in the docs; is there still outstanding work here that needs to happen on that front? Or are we good from a docs migration perspective?