I made a copy of the Wikibase wiki, updated it to 1.33 (yay for 22 hour spent on database schema updates), and made a new rdf dump (another 10 hours) and spend half a day in a debugger trying to understand the Munge Java code. It is very complicated, so I did not learn much, other than that parsing seems to work, but somehow the writer doesn't seem to be called to write out processed entities. I am feeling very frustrated now as this is blocking a project I am hoping to complete this year.

Oh and I also tried version 0.3.40 of the wdsq image, but no difference. Why is that not used by default, by the way?

Many thanks to @dcausse who identified two issues:

Localised namespace in wdata prefix:

@prefix wdata: <https://nimiarkisto.fi/wiki/Toiminnot:EntityData/> .

I worked this around with sed 's/Toiminnot:EntityData/Special:EntityData/g'. This should not be necessary and should be fixed.

Other issue was the use of https, for which passing -- -U https://nimiarkisto.fi to munge.sh fixed it.

How I worked around this in the end:

1. Set $wgWBRepoSettings['conceptBaseUri'] = 'http://nimiarkisto.fi/entity/';
2. Explicitly set the conceptUri to http using https://github.com/wmde/wikibase-docker/pull/140
3. Unlocalise the special page name using this snippet (for Finnish, needs adaptation for other languages):

$wgHooks['GetLocalURL'][] = function ( &$title, &$url, $query ) {
        if ( !$title->isExternal() && $query === '' && $title->getPrefixedText() === 'Toiminnot:EntityData' ) {
                $url = str_replace( '/wiki/Toiminnot:', '/wiki/Special:', $url );
        }
};

Cross-linking https://github.com/wmde/wikibase-docker/pull/140 which actually makes it possible to use http-protocol concept uris in wikibase-docker.

@dcausse is it realistic to expect this bug to be fixed by Wikimedia in the near future? Several institutuions that we are helping with Wikibase adoption are running into it.

I see two ways to fix this:

• wikibase should always use Special:EntityData and not the localized page name for its RDF output (similar to the workaround suggested)
• wdqs accepts new options to the munger/updater/loader to indicate the localized version it has to look for

I'm fine either ways, the second option will require wdqs admins to configure an additional option.

@Addshore do you have an opinion on this?

It would be cool if the munger's validation behaviour would be configurable, which could maybe remedy some weird behavior overall, see T274354

@dcausse: will anything bad happen if the URL comparison that is currently failing is removed? And is that applicable to people running their own Wikibase?

In this case, Special:EntityData in the dump is a technical identifier and should not be translated. We should have a way to create dumps in a language agnostic way.

@Addshore : would it be possible to fix this on the Wikibase side? Having an option to standardize dumps, independent of the language?

wikibase should always use Special:EntityData and not the localized page name for its RDF output (similar to the workaround suggested)

In this case, Special:EntityData in the dump is a technical identifier and should not be translated. We should have a way to create dumps in a language agnostic way.

Seems to make sense to me, and sounds like the easiest solution too!
I'll check this with @Samantha_Alipio_WMDE

Change 1010885 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[mediawiki/extensions/Wikibase@master] RDF: use unlocalized name of Special:EntityData in data URIs

https://gerrit.wikimedia.org/r/1010885

Change #1010885 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] RDF: use unlocalized name of Special:EntityData in data URIs

https://gerrit.wikimedia.org/r/1010885