Page MenuHomePhabricator

WDQ UI should support download of RDF formats
Open, Needs TriagePublic

Description

If I use a CONSTRUCT or DESCRIBE query, the WDQ UI https://query.wikidata.org/ should offer to save RDF formats (Turtle, JSONLD, NTriples, maybe also RDF/XML).
Instead, it offers the same tabular formats as for SELECT queries.

The WDQ endpoint https://query.wikidata.org/sparql returns RDF formats, I've tried with text/turtle.

Related Objects

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 5 2018, 7:56 AM
Restricted Application added a project: Wikidata. · View Herald TranscriptDec 5 2018, 7:57 AM

It shouldn't be hard to make at least NTriples export format - basically the same as CSV/TSV but with different separators.

Peb added a subscriber: Peb.May 15 2019, 12:52 PM

According to blazegraph documentation, the endpoint supports various formats.
I tried using my sparql query app and all listed formats there are returned.

Peb claimed this task.May 15 2019, 12:53 PM

Blazegraph supports it, yes, but that’s not really useful for us, as far as I can tell. The download should always result in the same data that is shown to the user, so you can’t send the query to the server again, asking for a different format, and hope that it will still return the same result – you have to turn the results you already have client-side (originally downloaded as JSON) into RDF.

@Lucas_Werkmeister_WMDE this is true, though I think adding a button/link/some other UI to convert a query to result download wouldn't hurt either, with the understanding that this download makes a separate query (so it's distinct from downloading result of existing query). We already have "SPARQL result" button but it might be possible to have more result formats like this.

Peb added a comment.May 15 2019, 3:09 PM

made a starting code to download as NTriples by converting from SPARQL-Result-JSON.
The JSON-LD might be quite straightforward using jsonld.js fromRdf function given NT input

diff --git a/wikibase/queryService/api/Sparql.js b/wikibase/queryService/api/Sparql.js
index 9a4e5e1..7003587 100644
--- a/wikibase/queryService/api/Sparql.js
+++ b/wikibase/queryService/api/Sparql.js
@@ -422,6 +422,60 @@ wikibase.queryService.api.Sparql = ( function( $ ) {
 	};
 
 	/**
+	 * Get the result of the submitted query as N-Triples
+	 *
+	 * @return {string}
+	 */
+	SELF.prototype.getResultAsNTriples = function() {
+		var output = '',
+			data = this._rawData;
+
+		output = this._processData( data, function( row, out ) {
+			var rowOut = '';
+			var rowVar = 'subject';
+			if ( ( row[rowVar] || {} ).type === 'uri' ) {
+				rowOut += '<';
+				rowOut += ( row[rowVar] || {} ).value;
+				rowOut += '> ';
+			} else if ( ( row[rowVar] || {} ).type === 'bnode' ) {
+				rowOut += '_:';
+				rowOut += ( row[rowVar] || {} ).value;
+				rowOut += ' ';
+			}
+			//predicate MUST be of type uri
+			rowVar = 'predicate';
+			rowOut += '<';
+			rowOut += ( row[rowVar] || {} ).value;
+			rowOut += '> ';
+			//object
+			rowVar = 'object';
+			if ( ( row[rowVar] || {} ).type === 'uri' ) {
+				rowOut += '<';
+				rowOut += ( row[rowVar] || {} ).value;
+				rowOut += '> ';
+			} else if ( ( row[rowVar] || {} ).type === 'literal' ) {
+				var obj = ( row[rowVar] || {} );
+				var escapedObj = JSON.stringify( String( obj.value ) );
+				escapedObj = escapedObj.substring( 1, escapedObj.length - 1 );
+				rowOut += '"';
+				rowOut += escapedObj;
+				rowOut += '"';
+				if ( 'datatype' in obj ) {
+					rowOut += '^^<';
+					rowOut += obj.datatype;
+					rowOut += '>';
+				} else if ( 'xml:lang' in obj ) {
+					rowOut += '@';
+					rowOut += obj['xml:lang'];
+				}
+			}
+			rowOut += ' .\n';
+			return out + rowOut;
+		}, output );
+		return output;
+	};
+
+	/**
 	 * Render value as per http://www.w3.org/TR/sparql11-results-csv-tsv/#tsv
 	 *
 	 * @param {Object} binding
Peb added a comment.May 18 2019, 3:48 PM

ignore the previous patch, i should have used existing _renderValueTSV function. so far implemented for NTriples and JSON-LD. for JSON-LD (using jsonld.js), i need to modify the download function in the ResultView.js (the one calling handler function) to accomodate handler function returning Promise object instead of string.

@Peb will this work in a streaming fashion? I know WD has a 60s timeout but still it can produce some 10-100 megabytes in that time. Will the JS approach be reliable enough ?
It's also important to be able to get Turtle, using all available prefixes, which is a lot more readable than ntriples.

I guess Blazegraph has this, and I disagree with Lucas that the query should not be run a second time. If the user acted a minute later, they would get that data with both approaches, so what dors it matter if the results are a bit different between ui and download?