@GoranSMilovanovic As per @agray : No -- my numbers were counting distinct items with the external IDs, rather than distinct statements, so should exactly match what you're aiming to compute.

@GoranSMilovanovic Overlap table for P 1367 (Art UK Artist ID) now showing about two-thirds of the overlap hits that it should be. (VIAF 6825, ULAN 6153, RKD 5582, ISNI 4240, Benezit 4643) vs true VIAF 11633, ULAN 10552, RKD 9769, ISNI 8237, Benezit 7529.

@Envlh That's very nice. So for example, here are your comparison tables for
P 1367 (Art UK Artist ID): https://tools.dicare.org/properties/?property=1367&type[]=ExternalId
and for P 650 (RKDArtists ID): https://tools.dicare.org/properties/?property=650&type[]=ExternalId

When you've got the data sorted, a table showing the closest identifiers by Jaccard similarity, rather than total overlap, might be quite interesting.

The numbers in the overlap data table for P 1367 (Art UK Artist ID) are way off as well -- only one tenth of the VIAF and ULAN overlaps correctly reported, only one twentieth of the RKD Artist ID overlaps.

Why are we baking in the assumption that only one Wikibase instance can be associated with a particular entity type?

As a postscript to my comment two posts above, note that in such a scenario a Commons category page might well be associated with both an item on Wikidata (via a sitelink equivalence) and a local item on the Commons wikibase.

See also: T180113 "Support the creation and use of volunteer tools that help to convert information in Commons categories to structured data"

In the context of this thread, it's worth recalling the ongoing wish from Commons users for Commons categories to be able to have their own local items on the Commons wikibase.

Hi @JeanFred . The point is not so much to link multiple versions of the same viewer, but rather to link to some of the different viewers available, for example here is the Alba Madonna in the "Universal Viewer" used as a default by the Wellcome Collection and the British Library and others.

Note that there has been some discussion on the IIIF mailing list as to whether a https://www.wikidata.org/wiki/Property:P1630 URL formatter should be added for the property:

To support what Smalyshev said: occasional termporary update lag may not be such a high-priority issue; but prolonged or repeated update lag rapidly would be.

Very nice!

That's nice. I didn't know that service included Commons.

(Copied from T173346)
I'd like to note strong interest on two different occasions from two different people in the last month in a top-tier GLAM that I'm working with, about the possibility of being able to use Commons as an IIIF hosting service for high-resolution images, eg for serving tiles of old maps for geo-referencing to platforms that need to be able to access images from an IIIF service.

Not sure if this is the right ticket for this information, but I'd like to note strong interest on two different occasions from two different people in the last month in a top-tier GLAM that I'm working with, about the possibility of being able to use Commons as an IIIF hosting service for high-resolution images, eg for serving tiles of old maps for geo-referencing to platforms that need to be able to access images from an IIIF service.

Presumably (as the original ticket suggests), one option would be to set up a Commons Data Query Service (CDQS) as a distinct endpoint on a new server that had both Wikidata and CommonsData installed in a Blazegraph instance locally. This would allow the existing WDQS service to continue without alteration, while allowing the CDQS service set-up to be optimised, as we learn more about the sort of queries that people want to run.

The 400 character limit can also be a problem for the titles of old books, maps, etc -- the original titles of these, as given eg in library catalogues, can get pretty long.

See this query: tinyurl.com/y76l2bft for an example of how to draw lines on a WDQS results map, based on values from variables in a WDQS query (in this case, drawing the bounding boxes of some maps)

Thanks @SandraF_WMF . I've started putting together some links at :c:User:Jheald/IIIF_180815 that I or Andy Mabbett could talk through, to give an idea of what sort of IIIF interaction is possible at the moment.

A heads-up that "Wikipedia and IIIF" is the proposed subject for the IIIF community call this week -- see https://groups.google.com/forum/?hl=en#!topic/iiif-discuss/wy2uRl_ukJ0

Interesting slide-show. But the fundamental problem -- as some of the attached tickets start to appreciate -- is that the key information that determines whether an image fits the specification being looked for is not going to be stored in statements just on the CommonsData item for the image in question, nor just on the Wikidata item for the thing it depicts, but is going to depend on statements distributed throughout the database.

Well of course you're going to copy all the CommonsData statements to Blazegraph.

I have now found T198261 and T199119, which investigate how some of the drawbacks above with the simple string-matching approach might be addressed.

I'd say don't give up too easily. This is probably as good an approach as any. If the issues are structural, bots will fall prey to them in just the same way, just more slowly and more haphazardly.

Don't you think you should maybe talk to the community about this first ?

Another example, where such image searches may depend quite sensitively on query construction or query optimisation: Category:Grade I listed buildings in Bedfordshire.

The attached subtickets are an interesting read. They all seem to be based on taking the Q-number value of "depicts", storing it as a string in the text-search index, and then doing an indexed string-match for it. Of course first baby-steps are important, and this facility will be crucial to be able to confirm correct entry, storage, and direct retrievability of "depicts" values.

One sidelight on this. QS2 is great (invaluable!) for adding new statements, but not so good for modifying existing statements, especially if they are currently heavily qualified or referenced -- at the moment, the entire statement complete with all qualifiers and references has to be re-created, even if one only wants to change one qualifier value, or migrate the property being used for the qualifier.

IIIF image extracts seem to be broken again: see eg the 'Virgin amongst the virgins' test page at Crotos

Hi Magnus,

That looks so great now. Thank you! (And @Shonagon ).

@dschwen Any idea why the IIIF-based detail viewer is working for the images on this page of Shonagon's:

• Vierge entre les vierges,

It would be good to make this work.

+1 on the need for this.

To make this really useful, it would be valuable for a WDQS query to be able to display such shapes in the map view of query results.

So you're saying, in effect, I should think of the strings being stored as a great big hash table rather than a B-tree, so there's nothing there that can help even STRSTARTS. And of course I know very little about the internals of BlazeGraph, whereas you've actually written modules for it. But I did think BlazeGraph uses B+ trees, which do specifically facilitate rapid in-order traversal and retrieval. But perhaps only to retrieve hashes, not strings?

Thanks for your speedy diagnosis. I've gone through and reverted and then un-reverted the ten edits by hand, so they are now fine.

Ouch. That sounds quite nasty. The only thing I can think of from the user side that was perhaps slightly different about this set of edits was they were made with QuickStatements while I already had a different QuickStatements run open and going in another window (a big batch of 10,000 corrections in capitalization of item labels, which took about two and a half hours to run, all of which went through fine).

Hi Smalyshev, thanks for the comment; but if I can come back on your two objections:

Perfect. I would never ever have found that.

Another case where it could be useful to be able to cast calculated co-ordinates back to a geo:wktLiteral would be to allow AVG() co-ordinates to be calculated and plotted, in cases where SAMPLE() is not sufficient.

Trying to get round this, by just plotting the values in a scatterplot, without a map, also failed -- though that's likely to be an issue with the scatterplot routine. (Ticket T160325 opened).

Example case:

As one possible hack that could be a way forward on this, I see that the Blazegraph GeoSpatial extension (description on BlazeGraph wiki) allows custom compound data-types to be defined -- which do not necessarily need to include a geospatial element.

As an aside, what I really wanted to do was to plot the values on a map.

Indeed, I still get the same behaviour even if I dispense with the third column entirely, and SELECT just the first two (query) -- so the issue may not involve the counts at all.

Example query (view in query editor):

Thanks for this Jonas, but given there is already the nice viewer on the page (which I didn't know about when I wrote the bug, because until then I'd only right-clicked on it), perhaps on consideration a better link is just the simple Commons file page, ie 'https://commons.wikimedia.org/wiki/File:{FILENAME}'

+1 on this.

Re: my previous comment, nodes that can store both a string and a URL would appear to be necessary to enable T121274 "Provide an RDF mapping for external identifiers" -- even though there are no such value nodes at the moment.

So something like p:P569/psv:P569 returns a URI, something like wdv:8f6e57348b9035361151ee05475253ef

So is there no way to create any kind of compound type that the GUI can interpret appropriately?
-eg date + precision, here
-or url + linktext, on other tickets

So if instead one wrote a query to return a wikibase:Time,
-eg by using p:P569/psv:P569 instead of wdt:P569
could it be possible for the GUI to pick that up and return an appropriately precisioned time?

It would also be useful for the GUI to show Wikipedia sitelinks as links rather than full URLs.

and T148245 "Explore making WDQS access mediawiki API"

See also T157676 "Provide access to category information from WDQS SPARQL"

You're right. I see that now.

• https://www.w3.org/TR/sparql11-query/#scopeFilters

Presumably what's happening here is that BlazeGraph thinks it can execute a bracketed block { ... } as a self-standing sub-query, independent of what has gone before.

And this query can take 20 seconds or even crash:

SELECT ?number  WHERE {
       VALUES ?number { 1 2 3 4 } .
       { FILTER ( ?number IN (1 , 2 ))  }
}

With regard to external identifiers and to the P:373 Commons category values, it strikes me that what we really need in the triplestore is a special datatype -- in a similar way to the way we have a special datatype for Commons media.

The gallery view is nice!! I don't know how I missed it - I thought I had tried it, and it just went to the same place as the file link.

The other (simplest) approach would just be to link to the Commons file page,

The standard URLs for the MediaViewer appear to be the wiki page it was launched from, then #/media/ and then the name of the file to display, so

• https://en.wikipedia.org/wiki/Cambridge#/media/File:KingsCollegeChapelWest.jpg
• https://commons.wikimedia.org/wiki/Category:Cambridgeshire#/media/File:A_dovecote_near_Leverington_-_geograph.org.uk_-_1538610.jpg

Hi Smalyshev, thanks for taking the time to get back to me.

If we can get the IIIF endpoint up and running again (or/and create an IIIF image-serving functionality more centrally to the main MediaWiki code), something that might be good would be the possibility for a user to download an IIIF manifest pretty much whenever the MediaViewer is used (maybe even specify the carousel of images for MediaViewer together with its metadata as an IIIF manifest).

For comparison, in September 2015 there were 376,524 references using P:854, and a query to search them could complete in 3.7 seconds -- it was used as an example on the query optimisation page. Such a query now times out.

Some further interesting examples in this thread on Project Chat of searches of this form that work and others that don't.

For completeness, the P:854 can also be used as a qualifier, principally on property P1343 "described by source", though the number of uses is much much smaller.

Some queries:

(Got the diff wrong, but the next paragraph shows why Alexmar983 would find this useful).

Trying to copy-paste the image links for some of the broken detail images on your
Virgin amongst the Virgins page

Hi Shonagon

A couple of wrinkles to examine, reported at Commons Village Pump: https://commons.wikimedia.org/wiki/Commons:Village_pump#WP_sidebars_now_link_to_Commons_categories

So queries won't have to be written like this any more ? :-)

It's good to see this going forwards at last.

Daniel, thank you *so* much for fixing this -- so now it should be all systems go!

It seems that the links above (the ones without the diacritics) work if the image has first been viewed in zoomviewer, which presumably is when the tiles get made and cached.

I'm not sure what I'm doing wrong, but I don't seem to be getting success for any of these:

That's fantastic! I am so excited, this is going to open up so much!