I should probably also point out that earlier versions of the citoid tool on wikidata for wikidata references used open refine and produced better results.

This looks like a mismatch between VisualEditor and Citoid - I had this issue locally too on master with Citoid until I updated VisualEditor to master. Some code was moved about between the two.

In T160845#6525484, @gerritbot wrote:

Change 632732 had a related patch set uploaded (by Mvolz; owner: Mvolz):
[mediawiki/services/citoid@master] [WIP] Divide authors by first name and last name

https://gerrit.wikimedia.org/r/632732

In T115224#6415888, @Cyberpower678 wrote:

I would like to emphasize that the availability API has seen improvements since 2017. Is it possible you can redo your investigation a bit?

In T108713#6434245, @iamjessklein wrote:

I agree with @Amire80 - the issue here is really the word "form". If we were in an universe where the term "template" wasn't such a loaded word, I'd suggest "Basic Template".

The hold-up seems to be eventstreams; it actually uses a fork of service runner, and the fork is missing the feature, so it's slightly more complicated to update. https://github.com/wikimedia/service-runner/tree/prometheus_metrics

It looks like they've released a v2 of their search api since I last looked that returns JSON: https://developer.api.oclc.org/wcv2 (instead of MarcXML or Dublin core, which is what we use now).

I think switching to that is a good idea, although even just using their sample input there's still a bit of junk left over in the authors field (a stray period!) But probably better than doing that processing ourselves!

In T160845#6500357, @kaldari wrote:

@Mvolz - I would be fine with that, but there are a couple caveats:

• The refToolbar code isn't foolproof, and in fact, there is no foolproof way to handle the WorldCat author data since its formatting is just too inconsistent.
• It currently handles "Jr." and "Sr.", but not suffixes or prefixes that may exist in other languages.

If you want to use it, the most recent version of the code is at https://github.com/alexz-enwp/reftoolbar/blob/master/lookup.php#L139.

In the meantime, I'll see if we can ask our contact at WorldCat if there are any better options.

In T160845#6459280, @kaldari wrote:

@Whatamidoing-WMF - It looks like the Library of Congress is using the same "weird" punctuation and capitalization as WorldCat:

• A world undone : the story of the Great War, 1914-1918
• How we get free : black feminism and the Combahee River Collective
• Ada, the enchantress of numbers : a selection from the letters of Lord Byron's daughter and her description of the first computer

Maybe it's some kind of library database standard or maybe the LOC is reusing WorldCat data for their own books.
@Mvolz - Can you report the titles that you get for the following ISBN numbers via Zotero's API: 9781608468553, 9780553803549, 9780912647180. Just want to confirm that they are similar to those returned by the LOC search.

In T219919#6478632, @fgiunchedi wrote:

It looks like citoid is now on k8s but still using gelf for logging, possibly the easiest at this point is switching to stdout logging like other node services on k8s. What do you think @Mvolz ?

It's likely a difference between Cite book on en wiki and the template that is used on ar wiki for books?

I'm unable to replicate this, it looks like it works: https://en.wikipedia.org/api/rest_v1/data/citation/mediawiki/10.1098%2Frsta.2014.0376

Just FYI but the reason we put maps above parameters is because there are often a lot of parameters, which means that in many cases the edit maps visible button isn't visible because it's "below the fold." This was an intentional design decision we made.

In T258572#6446378, @JMeybohm wrote:

In T258572#6446370, @Mvolz wrote:

I'm planning to deploy tomorrow, so I was wondering if I can have clarification on https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/622585

The eqiad file was renamed to just values, but I noticed in the examples folder there's both values.yaml and values-eqiad.yaml

So when I do a deploy, will I be able to deploy to eqiad, or is this a mistake and there should also be a separate eqiad file in this change in order to deploy to eqiad?

You're totally able to deploy to eqiad without a values-eqiad.yaml. Those environment specific values files can be used to overwrite settings (for that environment) but are not required. Please see https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/helmfile.d/services/README.MD for a description on which files are applied in which order (please tell if anything is unclear or missing! :))

I'm planning to deploy tomorrow, so I was wondering if I can have clarification on https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/622585

In T211411#4913374, @mobrovac wrote:

There have been no timeouts recorded by the automatic check scripts since the deploy, so looking good. Resolving for now, let's reopen if things change.

In T262129#6438611, @Mvolz wrote:

In T262129#6438603, @Mvolz wrote:

In T262129#6438601, @alanajjar wrote:

I've reverted these two changes with my work account Mvolz (WMF), but another user will need to approve the change since it's pending review.

Thanks @Mvolz. I approved all of them. And I can confirmed that Automatic cite works again. (I'll ask users with complaints to try it, then we can close this task).

What about the error message not substituted ?

citoid-citefromiddialog-mode-manual is the wrong message, I think... we refractored it. T219512. Looks like it didn't get retranslated with the right message name?

Yup, this should be citoid-citoiddialog-mode-manual instead.

https://translatewiki.net/w/i.php?title=Special:Translate&showMessage=citoid-citoiddialog-use-general-error-message&group=ext-citoid&language=ar&filter=&optional=1&action=translate needs to be fixed to use the correct dialog name, if you have an account on translatewiki?

In T262129#6438603, @Mvolz wrote:

In T262129#6438601, @alanajjar wrote:

I've reverted these two changes with my work account Mvolz (WMF), but another user will need to approve the change since it's pending review.

Thanks @Mvolz. I approved all of them. And I can confirmed that Automatic cite works again. (I'll ask users with complaints to try it, then we can close this task).

What about the error message not substituted ?

citoid-citefromiddialog-mode-manual is the wrong message, I think... we refractored it. T219512. Looks like it didn't get retranslated with the right message name?

In T262129#6438601, @alanajjar wrote:

I've reverted these two changes with my work account Mvolz (WMF), but another user will need to approve the change since it's pending review.

Thanks @Mvolz. I approved all of them. And I can confirmed that Automatic cite works again. (I'll ask users with complaints to try it, then we can close this task).

What about the error message not substituted ?

In T262129#6438421, @Mvolz wrote:

If you look at https://ar.m.wikipedia.org/wiki/%D8%AE%D8%A7%D8%B5:%D8%AA%D8%A7%D8%B1%D9%8A%D8%AE/%D9%82%D8%A7%D9%84%D8%A8:%D8%A7%D8%B3%D8%AA%D8%B4%D9%87%D8%A7%D8%AF_%D9%88%D9%8A%D8%A8/%D8%B4%D8%B1%D8%AD someone has deleted all the template data. That needs to be reverted on all the templates where it has been done as citoid will not work without template data.

If you look at https://ar.m.wikipedia.org/wiki/%D8%AE%D8%A7%D8%B5:%D8%AA%D8%A7%D8%B1%D9%8A%D8%AE/%D9%82%D8%A7%D9%84%D8%A8:%D8%A7%D8%B3%D8%AA%D8%B4%D9%87%D8%A7%D8%AF_%D9%88%D9%8A%D8%A8/%D8%B4%D8%B1%D8%AD someone has deleted all the template data. That needs to be reverted on all the templates where it has been done as citoid will not work without template data.

In T259928#6373100, @Sadads wrote:

Yep, I was using the getlink urls like:
*https://link.gale.com/apps/doc/A171034250/GPS?u=wikipedia&sid=GPS&xid=dbbf72df
https://link.gale.com/apps/doc/A171034250/GPS?u=wikipedia&sid=GPS&xid=dbbf72df
, but they weren't generating citations on Citoid so fell back on the
preformatted citations which are three buttons over. *

Declined because I think the adding of archive links is better left to bots.

Just an FYI, but I investigated adding this with a WIP patch here (a few years ago): https://gerrit.wikimedia.org/r/c/mediawiki/services/citoid/+/375810

In T255176#6376876, @kaldari wrote:

@Mvolz - Now that the code is merged locally, what happens next? Does someone need to deploy the new version to the production servers?

In T257809#6348472, @TmY_e12 wrote:

I couldn't find out what the problem was caused by. I would appreciate it if you review it at a convenient time.

Requested 10 get added to CI here upstream: https://github.com/zotero/translation-server/issues/118

In T160845#6357203, @kaldari wrote:

I've now deployed a fix that no longer splits the authors, and just puts them all in the "last" field. So in terms of how it's rendered with most citation templates, it looks a lot better. However this is a bit hacky, so I'm leaving this open for right now. It will continue to leave in parts of the author name that we don't necessarily want, like the birth date in parens.

Unfortunately, this entirely broke all author data for ISBN look-ups for RefToolbar, which excludes any author data that has a comma in it (under the assumption that this indicates the data is malformed). It took me a long time to track down where this regression came from. In the future, could you please post a notice at https://en.wikipedia.org/wiki/Wikipedia_talk:RefToolbar any time you make a significant change to what Citoid returns? Thanks!

@akosiaris is this feasible/desirable?

In T259469#6356479, @akosiaris wrote:

@Mvolz any ideas?

I think this is as resolved as it's going to get - it's been pretty much "as needed" and now we have a window.

In T257809#6344248, @TmY_e12 wrote:

I thought it was working but there is a problem. Some links work, some don't.

For example it works when I use this link: https://www.hurriyet.com.tr/kelebek/keyif/tiyatroya-adanmis-50-yil-tuncer-cucenoglu-41566796
But it doesn't work when I use it: https://www.nytimes.com/2020/07/28/science/nasa-jezero-perseverance.html

What did I do wrong?

https://doc.wikimedia.org/mediawiki-core/master/php/classStatusValue.html#a173df704be03a150626c89412f14fe04 looks like newFatal is a StatusValue fcn now? Maybe things just need to be updated?

I previously thought this was a TemplateData extension issue, but given that this is handled correctly by the 2017 wikitext editor, it might be a core problem? Not sure I tagged the right projects.

In T257809#6333142, @TmY_e12 wrote:

Despite all the mishaps, I finally save it. Can you verify that I'm doing it right?

https://tr.wikipedia.org/w/index.php?title=Şablon%3AKaynak%2Fbelge&type=revision&diff=22912043&oldid=17915365

And I also have to do the same to other templates from now on. Isn't it?

In T257809#6332989, @TmY_e12 wrote:

I have activated VisualEditor by adding ?veaction=edit to the end of the URL and deleted a few parameters that gave the error (like doi parameter) However, noticed some working paramaters also gave errors (like ad2). I'm sure the "ad2" parameter is working.

Error: "maps.citoid.podcaster" özelliği için geçersiz "ad2" parametresi.
Translation: Invalid "ad2" parameter for "maps.citoid.podcaster" feature.

Is this a bug too or something else?

This doesn't appear to be a problem in the 2017 wikitext editor, only the old wikitext editor. Unfortunately on some wikis these aren't enabled on Template pages (i.e. where the bug was first detected, Turkish Wikipedia: https://phabricator.wikimedia.org/T257809#6332885 )

In T257809#6329936, @TmY_e12 wrote:

I was trying to create my first map for the Kaynak template, but I cannot save the page. I review the code and found no errors. The code is here: https://pastebin.com/7uQvyj3x

Could you tell me why I couldn't save and what I did wrong? Thanks.

It's possible we might also be getting blocked or something because when I first ran it I was able to get the metadata, but the second time not. It's weird though, When I try to get the html of these locally, it's just a bunch of gibberish - basically a <script> tag in between an html tag and that's it.