Page MenuHomePhabricator

Add support for 'external identifier' data type (top-level and qualifiers)
Closed, ResolvedPublic


Add support for the data type 'external identifier' for statements and qualifiers on commons

A external identifier data type is essentially a string with constraints - on wikidata it's entered as a string, but it's validated against the property's constraints (it seems - under constraints)

Users need to be able to add an external id for any property that takes external identifiers as a value, in the File page and in UploadWizard


An external identifier can be an IMDB ID, an ISBN, or a Library of Congress authority ID or others (see full list of Wikidata properties that use the external identifier data type).

User stories

  • As a Commons editor, I want to add an external identifier to a media file’s metadata, so that GLAMs can connect this ID to other databases for archival purposes.
  • As a Commons editor, I want to add an external identifier to a media file’s metadata, so that other systems can find this Q item by looking up this ID.

Acceptance criteria:

  • can select a property with data type 'external identifier' and enter a value for it, on File page and UploadWizard
    • on File page
    • in UploadWizard
  • incorrect values return an error

Event Timeline

Change 535975 had a related patch set uploaded (by Eric Gardner; owner: Eric Gardner):
[mediawiki/extensions/WikibaseMediaInfo@master] Add support for properties with URL datatypes

Change 535975 abandoned by Eric Gardner:
Add support for properties with URL datatypes

Better to handle this systematically than doing each data type piecemeal; abandoned in favor of this patch: /537208

External Identifier property datatypes correspond to string value types, so (basic) support for these types is enabled via: Some UI work will be necessary to improve the experience.

Here's a question about this datatype RE: acceptance criteria.

Wikidata allows Properties to specify constraints (the WikibaseQualityConstraints extension seems to be what powers this). These constraints can control what values are allowed as well as how values get displayed using regular expressions. I'm assuming that we should honor these constraints when they exist, as part of the acceptance criteria for this feature.

Here's a concrete example of this feature on Wikidata. The property Instagram username has the "external identifier" data type. It has some additional rules in place so that when users type in a string username, the displayed value becomes a link to the appropriate instagram page using the "formatter URL" rule:$1/

So a user inputs this:

Screen Shot 2019-11-15 at 1.15.11 PM.png (316×1 px, 44 KB)

But sees this once they publish:

Screen Shot 2019-11-15 at 1.15.25 PM.png (340×1 px, 44 KB)

(this is a working link to Obama's IG page).

I assume that if Instagram disallowed certain characters or patterns, Wikidata could also validate and inform the user.

Fortunately for us, it looks like we can get Wikidata to do most of this work for us by using the wbformatvalue API. The key is that instead of providing a datatype for formatting, you can provide the specific property that a given value will be formatted against. The API params look like this:

	"action": "wbformatvalue",
	"format": "json",
	"datavalue": "{\"value\":\"barackobama\",\"type\":\"string\"}",
	"property": "P2003"

Here's a link to the API sandbox for further exploration.

So, long story short, it looks like we can rely on existing APIs to do most of this. The question is, how do we determine when to format against a specific property rather than against a datatype? Can we use a specific property in all cases (assuming the default rules for datatypes will be used if a property has no constraints of its own)? And it's nice we can rely on Wikidata in production, but for local development and testing it looks like we're going to need to set up some of these rules locally – something I don't really have any idea how to do yet.

@PDrouin-WMF here's what I was thinking for an updated input UI here (this would actually apply to all "string" types: URL, external-identifier, plain string, maybe monolingual-text).

I've introduced one additional element, a button that says "add" (exact language TBD). Users can either hit enter or use this button to insert the text in the field as a new item below. Server-side validation and re-formatting would take place at this time, based on the rules of the property in question. So a user may see error text above the input field if they hit enter/"add" with some invalid data.

Initial state:

Screen Shot 2019-11-15 at 5.28.48 PM.png (444×1 px, 42 KB)

Edit state:

Screen Shot 2019-11-15 at 5.29.13 PM.png (714×1 px, 71 KB)

Let me know what you think.

Change 551299 had a related patch set uploaded (by Eric Gardner; owner: Eric Gardner):
[mediawiki/extensions/WikibaseMediaInfo@master] Improved support for string input types

@egardner I feel you're mixing two things. An external id is an identifier that gets expanded to an URL (and sometimes an URI too). For example on we have the formatter url at . The formatter url takes care of the expansion.

Tested with multiple properties (Twitter Username, WTA Player ID, BBC News Live Democracy, etc). All worked.