Page MenuHomePhabricator

BlazeGraph Finalization: Validate AST rewrite
Closed, ResolvedPublic

Description

We'd like to use query AST rewriting for:

  • Synthetic properties (instead of inference)
  • Preventing overwhelming queries (unlimited ORDER BY)
  • Other stuff?

Event Timeline

Manybubbles raised the priority of this task from to Medium.
Manybubbles updated the task description. (Show Details)

@Thompsonbry.systap, is there an existing hook for overriding DefaultOptimizerList? I couldn't find one.

No. But this is very easy to add. I have created a ticket for this [1]. You can register with trac and subscribe to that ticket.

Update: I've implemented the hook in the SF GIT branch TICKET_1113.

[1] http://trac.bigdata.com/ticket/1113 (Add hook for the optimizer list)

OK! I've validated AST rewriting. It works (of course) but it''ll be a huge pain to implement some things with it. I imagine it'll be just fine for aliases. It's _possible_ to rewrite stuff from a regular statement pattern (wd:Q23 wdq:instanceOf wd:Q5) into a property path pattern (wd:Q23 wd:P31/wd:P279* wd:Q5). I haven't quite worked out all the kinks but is certainly good enough to validate the choice.

@Thompsonbry.systap, I was thinking of using this mechanism to prevent writes. Is there a simpler way to do that? It'd probably be something we'd want to turn on and off with url parameter or header or something. I perfectly fine to implement a simpler way if you have any ideas.

Do you have a pointer to your code? I'd like to understand where the pain is.

Do you have a pointer to your code? I'd like to understand where the pain is.

https://github.com/earldouglas/blazegraph-scratchpad/pull/1

The pain is just that you have to cover lots of cases. You have to catch the property being mentioned in a flat statement, a property path, and probably tons more places I haven't thought of. Its not impossible at all but it is an area that will require lots of testing.

Ok. That makes sense.

In terms of turning this feature on and off automatically, there are some options. One is to add a query hint. See the com.bigdata.rdf.sparql.ast.hints.QueryHintRegistry. It looks like this will work out of the box for 3rd party query hints. Or if this is a generally useful query hint (e.g., case hierarchy) then it could go into the core project.

Query hints passed through with the REST API requests are not processed in a generic fashion. See BigdataRDFContext.AbstractQueryTask<init>. For example

this.analytic = getEffectiveBooleanValue(
        req.getParameter(ANALYTIC), QueryHints.DEFAULT_ANALYTIC);

These hints are then propagated to the ASTContainer in setupQuery()

if (analytic) {

    // Turn analytic query on/off as requested.
    astContainer.setQueryHint(QueryHints.ANALYTIC, "true");

}

This could probably be turned into an extensible capability.

Bryan

Manybubbles moved this task from Backlog to Done on the MediaWiki-Core-Team board.

I consider this feature validated :)