Add ability to request base types in citoid, and offer use of both in extension for backwards compatibility until all templateData has been updated, and undo its use in extension, including use of templateData
Closed, ResolvedPublic0 Story Points

Description

Allow a query to request the base type (i.e. publicationTitle instead of bookTitle or websiteTitle, for example)

ToDo:

  • Add query parameter to request base types in citoid and temporarily include both basefield and specific fields to allow for gradual update of templatedata.
  • In extension, request basefields parameter.
  • Check templatedata everywhere : add synonyms in; leave in base fields like publicationTitle since there's no harm in having them in; add missing 'case' to itemType list, add in contributors & oclc field
    • en
    • fr
    • it
    • de
    • bs
    • el
    • es
    • fi
    • he
    • id
    • nl
    • no
    • pl
    • pt
    • sl
    • sv
    • uk - pending review
    • zh
  • Update extension to use mediawiki format (not mediawiki-basefields)
  • Remove backwards compatibility in citoid service that service puts basefields in the service
There are a very large number of changes, so older changes are hidden. Show Older Changes

@Mvolz: Yeah, I see what you mean. I wonder if it would be worth creating a Cite booksection template that remaps the parameters for Cite book.

Change 259499 merged by Mvolz:
Use basefields query parameter in citoid req

https://gerrit.wikimedia.org/r/259499

@Mvolz: In case you're interested, Citoid support in the WikiText editor is now live on English Wikipedia: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Adding_citations_now_easier. Thanks for your help on this. Adding the base types made things a lot easier.

Mvolz updated the task description. (Show Details)Jan 19 2016, 1:49 PM
Mvolz updated the task description. (Show Details)Jan 19 2016, 1:56 PM
Mvolz updated the task description. (Show Details)Jan 19 2016, 2:11 PM
Mvolz updated the task description. (Show Details)Apr 29 2016, 9:49 PM
Mvolz added a comment.Apr 29 2016, 9:51 PM

Is there a way to find out which wikis have citoid enabled? Or does anyone recall? @Elitre maybe?

We need to finish up updating the template data to remove the backwards compatibility.

Mvolz updated the task description. (Show Details)Apr 29 2016, 9:58 PM
Mvolz updated the task description. (Show Details)
Mvolz updated the task description. (Show Details)Apr 29 2016, 10:08 PM
Mvolz updated the task description. (Show Details)

I believe we checked whether the related json page existed (though I'm not sure I recall how) - the list we made earlier this year is at https://www.mediawiki.org/wiki/Citoid#cite_note-2 .

@Whatamidoing-WMF, did we just click through a list to find these wikis?

Thanks @Elitre, that list is a great jumping off point :).

Mvolz updated the task description. (Show Details)Apr 30 2016, 11:43 AM
Mvolz updated the task description. (Show Details)Apr 30 2016, 11:54 AM
Mvolz updated the task description. (Show Details)Apr 30 2016, 12:16 PM
Mvolz updated the task description. (Show Details)May 1 2016, 3:36 PM
Mvolz updated the task description. (Show Details)May 2 2016, 5:25 PM
Mvolz updated the task description. (Show Details)May 4 2016, 9:59 PM
Mvolz updated the task description. (Show Details)May 4 2016, 10:27 PM
Mvolz updated the task description. (Show Details)May 25 2016, 11:48 AM
Mvolz updated the task description. (Show Details)May 25 2016, 12:57 PM
Mvolz updated the task description. (Show Details)Jul 12 2016, 4:58 PM
Mvolz updated the task description. (Show Details)Jul 13 2016, 12:50 PM
Mvolz updated the task description. (Show Details)Jul 13 2016, 1:34 PM
Rical added a subscriber: Rical.Jul 13 2016, 7:31 PM

Issue with pl unfortunately; they use all the same template for the most part, so there's no way now for it to tell it that it's a journal versus a website, since it's all publicationTitle now. Before publicationTitle = journal, now websiteTitle is being set to journal as well.

Maybe we should make the request type configurable for each wiki? Instead of hardcoded into the extension?

What are the pros and cons of each option?

@Elitre It would make the documentation more confusing; because basically it means you have different fields depending on which option that language wiki was using.

In retrospect, maybe we should have stuck to doing it the old way after all; a lot of the big wikis had already discovered workarounds for having multiple fields named different things that worked pretty well. The only one it didn't work well for was bookSection/chapter and this didn't fix that either!

Mvolz added a comment.EditedJul 14 2016, 12:52 PM

So basically I would just undo everything I just did, however, on the plus side a LOT of these were missing citoid tempalte data altogether and weren't working at all, so not a total waste :)>

I'll let James, Ryan and others comment on the specific issue. (I think the priority is accommodating communities' needs if we can: if Citoid won't work for them, and they can't find a workaround, they may use it less (best case scenario), which is not our goal for sure.) Documentation problems may be not impossible to overcome, actually.

Mvolz added a comment.Jul 14 2016, 1:11 PM

Yes, I guess what I've realised is:

*The new way makes template data *simpler* to write for most wikis. However, many of them had figured out how to actually write it well and it was working effectively.

*The old way had much superior results for pl wiki, which has been bad every since we made the changes to the service in October of last year... it has been casting ALL websites as journal articles. With this new way the only way to fix this is to make separate templates.

*If we make this configurable, it will be confusing for people to know which fields to use, whereas in the old way, the only disadvantage was it made the TD more complicated. In my estimation, having it configurable makes it more complicated than how the old way was complicated.

I think I just answered my own question...

Yes, I guess what I've realised is:

*The new way makes template data *simpler* to write for most wikis. However, many of them had figured out how to actually write it well and it was working effectively.

*The old way had much superior results for pl wiki, which has been bad every since we made the changes to the service in October of last year... it has been casting ALL websites as journal articles. With this new way the only way to fix this is to make separate templates.

*If we make this configurable, it will be confusing for people to know which fields to use, whereas in the old way, the only disadvantage was it made the TD more complicated. In my estimation, having it configurable makes it more complicated than how the old way was complicated.

I think I just answered my own question...

Yeah. :-( I think you're right that the old way (plus hand-holding on TD) is better overall for users and wikis.

Which API query VisualEditor uses isn't that important to me, but if you end up deprecating the basefields option in the API please let me know, as this is currently being used by the refToolbar in the English Wikipedia WikiText editor. I'm sure I can hack it to work around whatever is returned by the API, but it would be good to have advanced notice.

Mvolz added a comment.Jul 14 2016, 8:48 PM

@kaldari, there's no reason to change the API, so no worries on your end!
This is just about which query the citoid visual editor extension makes.

Mvolz renamed this task from Add ability to request base types in citoid, and offer use of both in extension for backwards compatibility until all templateData has been updated. to Add ability to request base types in citoid, and offer use of both in extension for backwards compatibility until all templateData has been updated, and undo its use in extension, including use of templateData.Aug 12 2016, 1:35 PM
Mvolz updated the task description. (Show Details)
Rical removed a subscriber: Rical.Aug 13 2016, 6:08 AM
Mvolz updated the task description. (Show Details)Sep 2 2016, 4:17 PM
Mvolz moved this task from Waiting on Deploy to IO Tasks on the Citoid board.Sep 19 2016, 2:53 PM
Mvolz updated the task description. (Show Details)Nov 9 2016, 1:28 PM
Mvolz updated the task description. (Show Details)Nov 9 2016, 2:23 PM
Mvolz updated the task description. (Show Details)Nov 9 2016, 3:14 PM
Mvolz updated the task description. (Show Details)Nov 9 2016, 4:40 PM
Mvolz updated the task description. (Show Details)Nov 9 2016, 5:07 PM
Mvolz updated the task description. (Show Details)Nov 28 2016, 1:57 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 1:22 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 2:48 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 6:58 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 7:06 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 7:14 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 7:18 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 7:21 PM
Mvolz updated the task description. (Show Details)Dec 7 2016, 7:27 PM
Mvolz updated the task description. (Show Details)Dec 8 2016, 11:40 AM
Mvolz updated the task description. (Show Details)Dec 8 2016, 12:49 PM
Mvolz updated the task description. (Show Details)Dec 8 2016, 1:53 PM
Mvolz updated the task description. (Show Details)Dec 11 2016, 3:46 PM
Mvolz updated the task description. (Show Details)Dec 12 2016, 5:33 PM

Weirdly, nl has no oclc field in its Citeer boek template, but it does in Citeer journal.

Mvolz updated the task description. (Show Details)Dec 13 2016, 1:33 PM
Mvolz updated the task description. (Show Details)Dec 13 2016, 2:37 PM
Mvolz updated the task description. (Show Details)
Mvolz updated the task description. (Show Details)Dec 14 2016, 11:02 AM
Mvolz updated the task description. (Show Details)Dec 14 2016, 11:24 AM
Mvolz updated the task description. (Show Details)Dec 14 2016, 11:58 AM
Mvolz updated the task description. (Show Details)Dec 14 2016, 12:31 PM
Mvolz updated the task description. (Show Details)Dec 14 2016, 1:57 PM
Mvolz updated the task description. (Show Details)

Change 327244 had a related patch set uploaded (by Mvolz):
Use new restbase endpoint for service

https://gerrit.wikimedia.org/r/327244

Change 327244 merged by jenkins-bot:
Use new restbase endpoint for service

https://gerrit.wikimedia.org/r/327244

Extension update will roll out with wmf.8, so should be completed by 20th January if nothing goes wrong. Do the removal of the back-compat in the service in February?

Change 346749 merged by Mobrovac:
[mediawiki/services/citoid@master] Remove backwards compatibility from basefields

https://gerrit.wikimedia.org/r/346749

Stashbot added a subscriber: Stashbot.

Mentioned in SAL (#wikimedia-operations) [2017-04-13T15:46:23Z] <mobrovac@tin> Started deploy [citoid/deploy@212800d]: Enable multiple results for T115248 and remove b/c for T114515

Mentioned in SAL (#wikimedia-operations) [2017-04-13T15:49:33Z] <mobrovac@tin> Finished deploy [citoid/deploy@212800d]: Enable multiple results for T115248 and remove b/c for T114515 (duration: 03m 10s)

Mvolz closed this task as Resolved.May 1 2017, 4:23 PM
Mvolz updated the task description. (Show Details)
Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptMay 1 2017, 4:23 PM
Jdforrester-WMF changed the point value for this task from 8 to 0.May 2 2017, 6:58 PM