@Lucas, congratulations on your command line wizardry! I know "jq" but not
nearly to that extent, and I how did I not know about "units"?

@Tacsipacsi Labels are missing at https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/P2088. Maybe because it has a much bigger number of violations?

ok! posted T290961 as a replacement

Here's the goal: a SPARQL query should return all violations of a certain kind, with a possible data lag of a few hours.
So you need:

• a baseline of having processed all items (TODO)
• processing of changed items (DONE)
• periodic processing of every item because constraint definitions or implementations can change globally (TODO?)

https://www.wikidata.org/wiki/Module_talk:Constraints/SPARQL#tweak_queries_to_return_offending_value_not_offending_statement

@Mohammed_Sadat_WMDE Can you explain why a QS batch that merely deletes (does not create statements with qualifiers) returns errors and has to be restarted 10 or 20 times until all deletes go through? Or is "throttling" the same as "failing"?

@Addshore I think that's a fair description. To add:

• those pages are linked to the Discussion pages of each property, so are perceived as an integral part of WD, eg https://www.wikidata.org/wiki/Property_talk:P245:
  • Database reports/Constraint violations/P245: KrBot
  • Database reports/Complex constraint violations/P245: not sure who generates
  • Database reports/Humans with missing claims/P245: not sure who generates
  • And there's a link for each individual constraint
• It also generates useful Type Statistics (eg https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/P245#Types_statistics) that allow us to evaluate whether the type constraints are right. Eg the first "No" is https://www.wikidata.org/wiki/Q15632617 "fictional human" (ULAN is only supposed to include real persons and organizations) and I can search for Q15632617 to see whether to remove that type from WD items, or to add it to the ULAN property

@Addshore Resolved T204031 on 11 Aug: "Deploy regular running of wikidata constraint checks using the job queue: These now run after every edit"

(Yes, we mean WMDE.)

Can I make two related requests? Not sure how to post them as separate tasks related to this task, can someone from WMD do that?

@Gehel sorry, I don't see any investigation by WMD.
Yes, I care because that's a very valid use case of extracting and reshaping WD data.
Why does the Select succeed but Construct on the same resultset fail? Construct should be a very cheap operation after the results are found.
I will reopen.

https://news.ycombinator.com/item?id=28283350 is a discussion between @Denny and someone else where he says "There is no difference between 7-7-2000 and 07-07-2000 in xsd".

The following equivalent query works fine:

Final location of discussion: https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Ships#Home/Registry_Port_vs_City/Country

Reposted at https://www.wikidata.org/wiki/Wikidata:WikiProject_Ships/Maintenance#Home%2FRegistry_Port_vs_City%2FCountry

Here are some queries to add the counts: directly to the Item node, and using some namespace ontoRecon:. I show the above counts, plus 3 more.
They would need a lot of memory (group over 90M items) and a lot of time (especially statements).
Not tested yet.

PREFIX skos:   <http://www.w3.org/2004/02/skos/core#>
PREFIX schema: <http://schema.org/>
PREFIX wikibase: <http://wikiba.se/ontology#>

@ericP Wikidata doesn't use OWL axioms. It uses blanks only for the special values "unknown" and "no value"

Out of 100 results of this query, about a third are such broken "classes" (dump URLs)

select distinct ?c {
  [] wdt:P31|wdt:P279 ?c 
} limit 100

T10217 describez other zh langs that need to renamed to conform to official IANA lang tags.

Use FILTER(?dateOfDeath >= "2020-01-01"^^xsd:date) which is as convenient, but is also a valid literal (note the datatype).

The reason is overloading of the WD backend, therefore throttling of update rates of any particular user or tool.
I had similar issues with adding 1.5M "Worldcat Identity" identifiers, together with "source: <VIAF ID>".

Examining https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/P7704#%22Type_human_(Q5),_musical_ensemble_(Q2088357),_fictional_character_(Q95074),_fictional_profession_(Q17305127),_duo_(Q10648343),_pseudonym_(Q61002),_collective_(Q13473501),_group_of_humans_(Q16334295)%22_violations, the first thing "plasma" did not sound right to me (what has Plasma have to do with culture).
So I looked at https://www.wikidata.org/wiki/Q10251 and sure enough: Europeana entity: agent/base/33720 : broken link.

More details at https://www.wikidata.org/wiki/Property_talk:P7704#Remove_Disambiguation_entities_from_Europeana_Entities: 253+45 to remove.
Furthermore, https://www.wikidata.org/wiki/Property_talk:P7704#Europeana_Entity_class_constraint_test shows more classes to be analyzed.

I cannot post to https://europeana.atlassian.net/browse/EA.
So I made https://europeana.atlassian.net/browse/PRO-60: allow posting issues to more Europeana projects

Their issue trackers were at https://europeanadev.assembla.com/p/projects, in particular https://europeanadev.assembla.com/spaces/europeana-apis
https://europeanadev.assembla.com/spaces/europeana-infrastructure.
But they also have https://europeana.atlassian.net. https://europeana.atlassian.net/browse/RD-16 says at least one of the spaces (R&D) was migrated to Atlassian on 21/Oct/18.
So I looked at https://europeana.atlassian.net/secure/BrowseProjects.jspa and there are several relevant projects:

• https://europeana.atlassian.net/browse/EA: APIs. The project lead is Hugo Manguinhas who I think is also responsible for Europeana Entities
• https://europeana.atlassian.net/browse/IN
• https://europeana.atlassian.net/browse/RD

As of today, a slightly modified query https://w.wiki/K36 works ok (I added "country is Bulgaria")

I've done a lot of work with GLAM data that often includes "unknown" for creator.
Getty ULAN has a whole slew of "unknowns" http://vocab.getty.edu/doc/#ULAN_Hierarchy_and_Classes (note: the counts are several years old, I imagine there are a few more thousands of those now):

• 500355043 Unidentified Named People includes things like "the master of painting X"
• 500125081 Unknown People by Culture includes things like "unknown Egyptian" (to be used in situations like "unknown creator, but Egyptian culture"). We've modeled those as gvp:UnknownPersonConcept and groups (schema:Organization) but users still think of them as "persons".
• Further, there are things like "unknown but from the circle of Rembrandt" or "unknown but copy after Rembrandt" etc, about 20 varieties of them, see

https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Visual_arts/Item_structure#Attribution_Qualifiers and https://www.wikidata.org/wiki/Wikidata:Property_proposal/Attribution_Qualifier

With this query http://yasgui.org/short/ZI1xPxBvI :

http://yasgui.org/short/fLTUDXqOy are 16k portraits of @smithsoniannpg in #RDF, collected by http://americanartcollaborative.org, ready to integrate to @wikidata thru https://wikidata.org/wiki/Property:P6152 and https://wikidata.org/wiki/Property:P4692. Both NPG ID and AAC ID are 2.4k in WD; AAC ID's are total 234k. So there are 14k more portraits to be harvested from AAC and posted on WD #SumOfAllPaintings.

See https://www.wikidata.org/wiki/Property_talk:P7844#regex_and_formatterUrl_exceptions and https://www.wikidata.org/wiki/Property_talk:P7711#Format trying to get culture.fr people into some discussion regarding the identifiers used in their 30 thesauri, that their GINCO system cannot generate sequential IDs, mixes all thesauri into one namespace, and likely the result will be that we'll merge these thesauri on Wikidata.

I think this is your message: ShEx element already has 168 child elements, adding Wikidata annotations might break them and it points to the reason? I guess you give up generating the comments in this case: but why do you think 168 is too many?
The extra elements probably come from an add on (SingleFile comes to mind) but if you add a couple more to those 168, that would be ok?

Same result after relaunch

@Jelabra I see the tooltips at http://wikishape.weso.es/ Schema>Info:

Color is nice but imho decoding the P and Q numbers is critical. See this comment for details https://phabricator.wikimedia.org/T224962#5789394

Color is nice but imho decoding the P and Q numbers is critical.
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js does that: it linkifies Pnnn and Qnnn and shows a tooltip on hover.

@Salgo60 I could translate to BG but bg - Шаблон:Europeana is missing.

@Peb will this work in a streaming fashion? I know WD has a 60s timeout but still it can produce some 10-100 megabytes in that time. Will the JS approach be reliable enough ?
It's also important to be able to get Turtle, using all available prefixes, which is a lot more readable than ntriples.

Use construct when you want to get an rdf graph out of the repo, especially when you want the data to be shaped differently or to use ontology terms different from the original

@GerardM The IANA language tag list has a subfield Script exactly because languages can be written in different scripts.
Neither you nor I could write a single word in Phoenician script. Yet we could write a Phoenician word in English and be able to read it, and pronounce it approximately correctly.
This is valuable, no matter if you acknowledge it or not.

It's fixed!

https://github.com/eclipse/rdf4j/issues/1291 got this answer:

"Unless GerardM thinks it is not useful" ;-)

It does not make an argument go away.

There is no Phoenician in Latin script.

That is false. Gades is a Phoenician word, even if used in an English or Spanish text. The corresponding English word is Cadiz and the Spanish word is Cádiz.

"Gades" is in phn-Latn, that's for sure.

I've posted https://github.com/eclipse/rdf4j/issues/1291, let's see what the rdf4j developers say.

Agree with @PKM: the name is "Gades"@phn-Latn, "Cadiz"@es and I guess "Cadiz"@en.
@Lydia_Pintscher will you name the language "Phoenician (Transliterated)"? The Getty does that, search for Chinese in that page and you'll see eg

		............................ Chinese (transliterated Hanyu Pinyin) (language)
		............................ Chinese (transliterated Pinyin) (language)
		............................ Chinese (transliterated Pinyin without tones) (language)
		............................ Chinese (transliterated Tongyong Pinyin) (language)
		............................ Chinese (transliterated Wade-Giles) (language)

Hmmm, rdf4j returns the same false. Also if i use = instead of sameTerm

This returns False, which I believe is a bug in Blazegraph

select * {
  bind (
    sameTerm(<https://en.wikipedia.org/wiki/University_of_Tromsø>,
     <https://en.wikipedia.org/wiki/University_of_Troms%C3%B8>)
        as ?x)
  }

Can you point to the encoding rules so we do the same before querying. Sparql's encode-for-uri encodes everything except upper- and lower-case letters A-Z, the digits 0-9, HYPHEN-MINUS ("-"), LOW LINE ("_"), FULL STOP ".", and TILDE "~". See https://www.w3.org/TR/xpath-functions/#func-encode-for-uri.

Ok, this works and returns 18650 rows as expected:

prefix wd:   <http://www.wikidata.org/entity/>
prefix wdt:  <http://www.wikidata.org/prop/direct/>
prefix bd:   <http://www.bigdata.com/rdf#>
prefix wikibase: <http://wikiba.se/ontology#>

So can you perhaps rewrite the query in a way that it works?

I want to use federation from an external repo (as the descr says "another repo").
You can test the bug either using an external repo, or self-federation.

phn means "Phoenician in Phoenician script" but the example given is in Latin script, thus it is phn-Latn.

https://www.wikidata.org/wiki/Wikidata:Project_chat#Phoenician_language: according to the IANA lang tag list, the default script for this language is "Phnx" Phoenician but the example there is given in Latin transcription so shoul de denoted like "gadir"@phn-Latn.

• To capture this, I believe you should register a lang tag phn-Latn with name eg "Phoenician (Latin transcription)".
• The tag phn should be used only if you install Phoenician script (alphabet) in the language tools; and somehow direct the user to select "Phoenician (Latin transcription)" if they intend to enter a transliterated label

Where is the template?

@Nikkimaria excellent work! Left feedback at Discussion