MWAPI service throws “could not find binding for parameter” if optimizer is not disabled
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Lucas_Werkmeister_WMDE
	Jun 26 2017, 3:31 PM

Description

The following query:

# trying to use mwapi to get (license) categories of commons images
# see https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual/MWAPI
SELECT * WHERE {
  BIND(wd:Q668563 as ?leinestr) # random item with picture
  ?leinestr wdt:P18 ?picture.

  BIND(STRAFTER(str(?picture), "Special:FilePath/") AS ?filename)
  BIND(CONCAT("File:", wikibase:decodeUri(?filename)) AS ?file)

  SERVICE wikibase:mwapi {
    bd:serviceParam wikibase:api "Categories".
    bd:serviceParam wikibase:endpoint "commons.wikipedia.org".
    bd:serviceParam mwapi:titles ?file. # throws IllegalArgumentException: Could not find binding for parameter titles
    ?cat wikibase:apiOutput mwapi:category.
  }
}

results in the following error:

…
Caused by: java.lang.IllegalArgumentException: Could not find binding for parameter titles
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall.getRequestParams(MWApiServiceCall.java:152)
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall.getHttpRequest(MWApiServiceCall.java:174)
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall.access$000(MWApiServiceCall.java:57)
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall$MultiSearchIterator.doSearchFromBinding(MWApiServiceCall.java:317)
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall$MultiSearchIterator.doNextSearch(MWApiServiceCall.java:300)
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall$MultiSearchIterator.<init>(MWApiServiceCall.java:262)
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall.call(MWApiServiceCall.java:121)
at org.wikidata.query.rdf.blazegraph.mwapi.MWApiServiceCall.call(MWApiServiceCall.java:56)
at com.bigdata.bop.controller.ServiceCallJoin$ChunkTask$ServiceCallTask.doBigdataServiceCall(ServiceCallJoin.java:770)

If the optimizer is disabled via query hint, the query works (though it finds no results, but that’s another topic: T168877), so it looks like the optimizer reorders the query in such a way that the service doesn’t have access to the ?file binding.

Details

	Subject	Repo	Branch	Lines +/-
	Fix “could not find binding for parameter” in MWAPI	wikidata/query/rdf	master	+355 -176

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		Igorkim78	T235759 [TRACKING] WDQS / Blazegraph optimization / bug fixes
		Resolved		debt	T168876 MWAPI service throws “could not find binding for parameter” if optimizer is not disabled

Event Timeline

• Lucas_Werkmeister_WMDE created this task.Jun 26 2017, 3:31 PM

Restricted Application added a project: Discovery-ARCHIVED. · View Herald TranscriptJun 26 2017, 3:31 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

robbi5 subscribed.Jun 26 2017, 3:34 PM

• Lucas_Werkmeister_WMDE mentioned this in T168877: Add support for using MWAPI to search for Commons media file statement values.Jun 26 2017, 3:41 PM

• Lucas_Werkmeister_WMDE updated the task description. (Show Details)

This is the result of query being reordered, but I'm not sure why - input parameters should be bound by the time SERVICE is run. I'll check why it doesn't happen and see what can be done.

Smalyshev moved this task from Incoming to Current work on the Wikidata-Query-Service board.Jun 26 2017, 5:44 PM

Looking at the binding set incoming to service, I see:

{ leinestr=Vocab(2)[http://www.wikidata.org/entity/Q]:XSDUnsignedInt(668563), picture=TermId(1U)[http://commons.wikimedia.org/wiki/Special:FilePath/U-Bahn%20Berlin%20Leinestra%C3%9Fe.JPG], filename=TermId(0L)[U-Bahn%20Berlin%20Leinestra%C3%9Fe.JPG] }

So the order is right (since other vars are bound) but somehow ?file is missing from bindings. I wonder is Optimizer somehow drops it and whether I should tell it somehow not to do it.

Looks like Optimizer for some reason puts the second BIND after the service. Not sure why. Workaround for this one would be:

hint:Prior hint:runLast true .

after the service, but this of course is just a workaround, not a fix.

Smalyshev moved this task from Current work to Blazegraph on the Wikidata-Query-Service board.Jun 27 2017, 1:05 AM

Smalyshev removed a project: Discovery-Wikidata-Query-Service-Sprint.Jun 29 2017, 12:02 AM

Smalyshev added a project: WDQS-Optimizer.Sep 24 2017, 11:49 PM

Change 533116 had a related patch set uploaded (by Igor Kim; owner: Igor Kim):
[wikidata/query/rdf@master] Fix “could not find binding for parameter” in MWAPI

https://gerrit.wikimedia.org/r/533116

gerritbot added a project: Patch-For-Review.Aug 28 2019, 11:36 PM

Smalyshev updated the task description. (Show Details)Aug 29 2019, 5:22 AM

Change 533116 merged by jenkins-bot:
[wikidata/query/rdf@master] Fix “could not find binding for parameter” in MWAPI

https://gerrit.wikimedia.org/r/533116

Smalyshev added a project: Discovery-Wikidata-Query-Service-Sprint.Aug 29 2019, 9:41 PM

Smalyshev moved this task from Backlog to Done on the Discovery-Wikidata-Query-Service-Sprint board.

Smalyshev removed a project: Patch-For-Review.

debt closed this task as Resolved.Sep 5 2019, 6:44 PM

debt claimed this task.

• Lucas_Werkmeister_WMDE mentioned this in T235540: StackOverflowError when SPARQL query uses same variable name before and after aggregation.Oct 15 2019, 5:56 PM

Gehel added a parent task: T235759: [TRACKING] WDQS / Blazegraph optimization / bug fixes.Oct 17 2019, 12:42 PM