Page MenuHomePhabricator

WDQ UI should support download of RDF formats
Open, Needs TriagePublic

Description

If I use a CONSTRUCT or DESCRIBE query, the WDQ UI https://query.wikidata.org/ should offer to save RDF formats (Turtle, JSONLD, NTriples, maybe also RDF/XML).
Instead, it offers the same tabular formats as for SELECT queries.

The WDQ endpoint https://query.wikidata.org/sparql returns RDF formats, I've tried with text/turtle.

Related Objects

Event Timeline

It shouldn't be hard to make at least NTriples export format - basically the same as CSV/TSV but with different separators.

According to blazegraph documentation, the endpoint supports various formats.
I tried using my sparql query app and all listed formats there are returned.

Blazegraph supports it, yes, but that’s not really useful for us, as far as I can tell. The download should always result in the same data that is shown to the user, so you can’t send the query to the server again, asking for a different format, and hope that it will still return the same result – you have to turn the results you already have client-side (originally downloaded as JSON) into RDF.

@Lucas_Werkmeister_WMDE this is true, though I think adding a button/link/some other UI to convert a query to result download wouldn't hurt either, with the understanding that this download makes a separate query (so it's distinct from downloading result of existing query). We already have "SPARQL result" button but it might be possible to have more result formats like this.

made a starting code to download as NTriples by converting from SPARQL-Result-JSON.
The JSON-LD might be quite straightforward using jsonld.js fromRdf function given NT input

diff --git a/wikibase/queryService/api/Sparql.js b/wikibase/queryService/api/Sparql.js
index 9a4e5e1..7003587 100644
--- a/wikibase/queryService/api/Sparql.js
+++ b/wikibase/queryService/api/Sparql.js
@@ -422,6 +422,60 @@ wikibase.queryService.api.Sparql = ( function( $ ) {
 	};
 
 	/**
+	 * Get the result of the submitted query as N-Triples
+	 *
+	 * @return {string}
+	 */
+	SELF.prototype.getResultAsNTriples = function() {
+		var output = '',
+			data = this._rawData;
+
+		output = this._processData( data, function( row, out ) {
+			var rowOut = '';
+			var rowVar = 'subject';
+			if ( ( row[rowVar] || {} ).type === 'uri' ) {
+				rowOut += '<';
+				rowOut += ( row[rowVar] || {} ).value;
+				rowOut += '> ';
+			} else if ( ( row[rowVar] || {} ).type === 'bnode' ) {
+				rowOut += '_:';
+				rowOut += ( row[rowVar] || {} ).value;
+				rowOut += ' ';
+			}
+			//predicate MUST be of type uri
+			rowVar = 'predicate';
+			rowOut += '<';
+			rowOut += ( row[rowVar] || {} ).value;
+			rowOut += '> ';
+			//object
+			rowVar = 'object';
+			if ( ( row[rowVar] || {} ).type === 'uri' ) {
+				rowOut += '<';
+				rowOut += ( row[rowVar] || {} ).value;
+				rowOut += '> ';
+			} else if ( ( row[rowVar] || {} ).type === 'literal' ) {
+				var obj = ( row[rowVar] || {} );
+				var escapedObj = JSON.stringify( String( obj.value ) );
+				escapedObj = escapedObj.substring( 1, escapedObj.length - 1 );
+				rowOut += '"';
+				rowOut += escapedObj;
+				rowOut += '"';
+				if ( 'datatype' in obj ) {
+					rowOut += '^^<';
+					rowOut += obj.datatype;
+					rowOut += '>';
+				} else if ( 'xml:lang' in obj ) {
+					rowOut += '@';
+					rowOut += obj['xml:lang'];
+				}
+			}
+			rowOut += ' .\n';
+			return out + rowOut;
+		}, output );
+		return output;
+	};
+
+	/**
 	 * Render value as per http://www.w3.org/TR/sparql11-results-csv-tsv/#tsv
 	 *
 	 * @param {Object} binding

ignore the previous patch, i should have used existing _renderValueTSV function. so far implemented for NTriples and JSON-LD. for JSON-LD (using jsonld.js), i need to modify the download function in the ResultView.js (the one calling handler function) to accomodate handler function returning Promise object instead of string.

@Peb will this work in a streaming fashion? I know WD has a 60s timeout but still it can produce some 10-100 megabytes in that time. Will the JS approach be reliable enough ?
It's also important to be able to get Turtle, using all available prefixes, which is a lot more readable than ntriples.

I guess Blazegraph has this, and I disagree with Lucas that the query should not be run a second time. If the user acted a minute later, they would get that data with both approaches, so what dors it matter if the results are a bit different between ui and download?