Change Details

One of the main goals of [[ https://meta.wikimedia.org/wiki/Grants:IEG/Enhance_the_ProveIt_gadget | my grant to enhance the ProveIt gadget ]] is connecting it with Wikidata, so that references are **autocompleted** when the user enters enough data to uniquely identify them. Furthermore, new references added with the gadget should be inserted into Wikidata, and changes done to existing references should be inserted too, so that users only need to enter the information once. This is a very desirable feature, but I've been thinking about it since the grant began and I still don't see very clearly how the UX and UI should be, not to mention the actual implementation details. So I start this task to get some input and help. Some months ago [[ https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/2016/07#Reference_items_for_ProveIt | I contacted the Wikidata community ]] and they are ok with allowing ProveIt to create items at Wikidata, even if this means that items for non-notable sources may be created. They also pointed me to the [[ https://www.wikidata.org/wiki/Wikidata:WikiProject_Source_MetaData | WikiProject Source MetaData ]] that seems involved in making this kind of connections possible. == UX and UI == What would be the best way to present and interact with this functionality? The best I could think of so far is this: every time the user moves from one field to the next, we query Wikidata in search for a match. If we find one, we display a link to the item at the **bottom-left corner of the gadget**. If the link is clicked, the Wikidata item opens on a new browser tab. Next to the link, a button that says "Use" appears. When the button is clicked, the reference form is filled with the data from the Wikidata item. After that, any changes done to the form will be pushed to Wikidata when the user hits the Insert button (well, not all, there are many irrelevant fields, such as the page number, citation text, etc. which rises the question of how to distinguish them, see below). Once the Use button is clicked, it should probably be replaced with an Unlink button that unlinks the form from Wikidata, in case the user doesn't want to push the changes to Wikidata, or in case the Use button was hit by mistake. == Implementation == How to map template parameters with Wikidata properties? My best idea so far is this: in many cases, there is a correlation between parameter names and property names. For example, the "author" parameter in the English citation templates corresponds to the "author" property in Wikidata. Same in Spanish: the "autor" parameter corresponds to the "autor" property, and in other languages it will probably be the same, simply because it's the most obvious name to give to the field. If the property name has spaces, we replace them with dashes and we often get a match, for example, the "publication-date" parameter of the English citation templates is called "publication date" in Wikidata. Lastly, we can use **parameter aliases** to match the properties that are called differently in Wikidata. For example, the "publication-place" parameter of the English citation templates is called "published in" in Wikidata, but we can add a parameter alias called "published-in" to the TemplateData of the templates so that the gadget finds the match. This way of matching parameter names with property names can probably be used not only for filling the reference form with the data retrieved from Wikidata, but also for querying Wikidata effectively when searching for a match. Given that properties in Wikidata are predefined and cannot be arbitrarily created, it should be possible to distinguish between data that should be pushed into Wikidata, and data that should not. If a corresponding property exists in Wikidata, then update it. Else don't. Some cases however may be problematic. For example, I found that articles have a "page(s)" property for indicating the pages at which the article is found within the journal ([[ https://www.wikidata.org/wiki/Q1895685 | example ]]) but in Wikipedia citations, the page(s) parameter is used for specifying the page(s) relevant to the reference, not the whole article. Not sure about the solution here, but maybe these special cases can be hard-coded. These are my ideas and solutions so far, and I think they are pretty good. But because I haven't started with the actual coding yet, and because I don't know Wikidata too well, I may be missing important issues. If so, I hope you can help me detect them, and solve them. Thanks!

One of the main goals of my [[ https://meta.wikimedia.org/wiki/Grants:IEG/Enhance_the_ProveIt_gadget | grant to enhance the ProveIt gadget ]] is connecting it with Wikidata, so that references are **autocompleted** when the user enters enough data to uniquely identify them. Furthermore, new references added with the gadget should be inserted into Wikidata, and changes done to existing references should be inserted too, so that users only need to enter the information once. This is a very desirable feature, but I've been thinking about it since the grant began and I still don't see very clearly how the UX and UI should be, not to mention the actual implementation details. So I start this task to get some input and help. Some months ago [[ https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/2016/07#Reference_items_for_ProveIt | I contacted the Wikidata community ]] and they are ok with allowing ProveIt to create items at Wikidata, even if this means that items for non-notable sources may be created. They also pointed me to the [[ https://www.wikidata.org/wiki/Wikidata:WikiProject_Source_MetaData | WikiProject Source MetaData ]] that seems involved in making this kind of connections possible. == UX and UI == What would be the best way to present and interact with this functionality? The best I could think of so far is this: every time the user moves from one field to the next, we query Wikidata in search for a match. If we find one, we display a link to the item at the **bottom-left corner of the gadget**. If the link is clicked, the Wikidata item opens on a new browser tab. Next to the link, a button that says "Use" appears. When the button is clicked, the reference form is filled with the data from the Wikidata item. After that, any changes done to the form will be pushed to Wikidata when the user hits the Insert button (well, not all, there are many irrelevant fields, such as the page number, citation text, etc. which rises the question of how to distinguish them, see below). Once the Use button is clicked, it should probably be replaced with an Unlink button that unlinks the form from Wikidata, in case the user doesn't want to push the changes to Wikidata, or in case the Use button was hit by mistake. == Implementation == How to map template parameters with Wikidata properties? My best idea so far is this: in many cases, there is a correlation between parameter names and property names. For example, the "author" parameter in the English citation templates corresponds to the "author" property in Wikidata. Same in Spanish: the "autor" parameter corresponds to the "autor" property, and in other languages it will probably be the same, simply because it's the most obvious name to give to the field. If the property name has spaces, we replace them with dashes and we often get a match, for example, the "publication-date" parameter of the English citation templates is called "publication date" in Wikidata. Lastly, we can use **parameter aliases** to match the properties that are called differently in Wikidata. For example, the "publication-place" parameter of the English citation templates is called "published in" in Wikidata, but we can add a parameter alias called "published-in" to the TemplateData of the templates so that the gadget finds the match. This way of matching parameter names with property names can probably be used not only for filling the reference form with the data retrieved from Wikidata, but also for querying Wikidata effectively when searching for a match. Given that properties in Wikidata are predefined and cannot be arbitrarily created, it should be possible to distinguish between data that should be pushed into Wikidata, and data that should not. If a corresponding property exists in Wikidata, then update it. Else don't. Some cases however may be problematic. For example, I found that articles have a "page(s)" property for indicating the pages at which the article is found within the journal ([[ https://www.wikidata.org/wiki/Q1895685 | example ]]) but in Wikipedia citations, the page(s) parameter is used for specifying the page(s) relevant to the reference, not the whole article. Not sure about the solution here, but maybe these special cases can be hard-coded. These are my ideas and solutions so far, and I think they are pretty good. But because I haven't started with the actual coding yet, and because I don't know Wikidata too well, I may be missing important issues. If so, I hope you can help me detect them, and solve them. Thanks!

One of the main goals of my [[ https://meta.wikimedia.org/wiki/Grants:IEG/Enhance_the_ProveIt_gadget | my grant to enhance the ProveIt gadget ]] is connecting it with Wikidata, so that references are **autocompleted** when the user enters enough data to uniquely identify them. Furthermore, new references added with the gadget should be inserted into Wikidata, and changes done to existing references should be inserted too, so that users only need to enter the information once. This is a very desirable feature, but I've been thinking about it since the grant began and I still don't see very clearly how the UX and UI should be, not to mention the actual implementation details. So I start this task to get some input and help. Some months ago [[ https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/2016/07#Reference_items_for_ProveIt | I contacted the Wikidata community ]] and they are ok with allowing ProveIt to create items at Wikidata, even if this means that items for non-notable sources may be created. They also pointed me to the [[ https://www.wikidata.org/wiki/Wikidata:WikiProject_Source_MetaData | WikiProject Source MetaData ]] that seems involved in making this kind of connections possible. == UX and UI == What would be the best way to present and interact with this functionality? The best I could think of so far is this: every time the user moves from one field to the next, we query Wikidata in search for a match. If we find one, we display a link to the item at the **bottom-left corner of the gadget**. If the link is clicked, the Wikidata item opens on a new browser tab. Next to the link, a button that says "Use" appears. When the button is clicked, the reference form is filled with the data from the Wikidata item. After that, any changes done to the form will be pushed to Wikidata when the user hits the Insert button (well, not all, there are many irrelevant fields, such as the page number, citation text, etc. which rises the question of how to distinguish them, see below). Once the Use button is clicked, it should probably be replaced with an Unlink button that unlinks the form from Wikidata, in case the user doesn't want to push the changes to Wikidata, or in case the Use button was hit by mistake. == Implementation == How to map template parameters with Wikidata properties? My best idea so far is this: in many cases, there is a correlation between parameter names and property names. For example, the "author" parameter in the English citation templates corresponds to the "author" property in Wikidata. Same in Spanish: the "autor" parameter corresponds to the "autor" property, and in other languages it will probably be the same, simply because it's the most obvious name to give to the field. If the property name has spaces, we replace them with dashes and we often get a match, for example, the "publication-date" parameter of the English citation templates is called "publication date" in Wikidata. Lastly, we can use **parameter aliases** to match the properties that are called differently in Wikidata. For example, the "publication-place" parameter of the English citation templates is called "published in" in Wikidata, but we can add a parameter alias called "published-in" to the TemplateData of the templates so that the gadget finds the match. This way of matching parameter names with property names can probably be used not only for filling the reference form with the data retrieved from Wikidata, but also for querying Wikidata effectively when searching for a match. Given that properties in Wikidata are predefined and cannot be arbitrarily created, it should be possible to distinguish between data that should be pushed into Wikidata, and data that should not. If a corresponding property exists in Wikidata, then update it. Else don't. Some cases however may be problematic. For example, I found that articles have a "page(s)" property for indicating the pages at which the article is found within the journal ([[ https://www.wikidata.org/wiki/Q1895685 | example ]]) but in Wikipedia citations, the page(s) parameter is used for specifying the page(s) relevant to the reference, not the whole article. Not sure about the solution here, but maybe these special cases can be hard-coded. These are my ideas and solutions so far, and I think they are pretty good. But because I haven't started with the actual coding yet, and because I don't know Wikidata too well, I may be missing important issues. If so, I hope you can help me detect them, and solve them. Thanks!