Page MenuHomePhabricator

Using data sets from another wiki for the Graph extension
Closed, ResolvedPublic

Description

See https://hu.wikipedia.org/wiki/Szerkeszt%C5%91:Tgr/graph_teszt for an attempt. This fails to CORS violations. It would be nice if there was a way to reuse datasets without copying them over.

Event Timeline

Tgr created this task.Dec 27 2015, 9:47 PM
Tgr raised the priority of this task from to Needs Triage.
Tgr updated the task description. (Show Details)
Tgr added a project: Graphs.
Tgr added a subscriber: Tgr.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 27 2015, 9:47 PM
Yurik added a subscriber: Yurik.Dec 27 2015, 10:03 PM

Agreed 144% - apparently there is a CORS bug somewhere, I will check with Security.

Tgr added a comment.Dec 27 2015, 10:11 PM

Some options:

  • T91162: RFC: Shadow namespaces would be the nice solution, but it's probably not happening anytime soon.
  • just add Allow-Origin headers to action=raw requests coming from a $wgCrossSiteAJAXdomains domain. As long as authenticated CORS is not allowed, that does not seem scary.
  • CORS is already enabled for the API, meaning that the page content can be fetched by a revisions API request. That works if the data format is JSON as Vega allows specifying a path in the returned JSON object. There doesn't seem to be any way to tell Vega that something is a CSV etc. wrapped in JSON, though.
Tgr added a comment.Dec 27 2015, 10:14 PM

I don't think it's a bug per se, CORS is simply not enabled for anything other than the API (and images). For authenticated CORS, this is probably for the better (at the very least, access to Special:UserLogin should be forbidden) but basic CORS would be harmless as far as I can see.

Yurik added a comment.EditedDec 27 2015, 10:19 PM

@Tgr, it should work, but it doesn't. The "raw" can be replaced with wikiraw://www.mediawiki.org/Extension:Graph/data/us-10m-json -- the wikiraw: protocol uses API, but it doesn't yet work on server side graphoid, but it works for graph preview and interactive. Yet it still fails CORs. On the other hand, page views graph works fine, even though it makes a request to another domain: https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/user/Main_Page/daily/2015121722/2015122722

Change 261169 had a related patch set uploaded (by Yurik):
Fixed CORS issue with MW api calls

https://gerrit.wikimedia.org/r/261169

Change 261200 had a related patch set uploaded (by Yurik):
Fixed CORS issue with MW api calls

https://gerrit.wikimedia.org/r/261200

Change 261169 merged by jenkins-bot:
Fixed CORS issue with MW api calls

https://gerrit.wikimedia.org/r/261169

Change 261200 merged by jenkins-bot:
Fixed CORS issue with MW api calls

https://gerrit.wikimedia.org/r/261200

Yurik added a comment.Dec 28 2015, 8:45 PM

@Tgr, i deployed a fix and updated your example. Now it does not show the graph until you click the "play' button, but than it works. I won't close this ticket until i update Graphoid to support wikiraw:// and other protocols.

Tgr added a comment.Dec 28 2015, 9:47 PM

Awesome, thanks!

I'm unable to use dataset (JSON) from another wikipedia. (it works, but only using interactive mode)

Simple example - which is a copy of copy https://www.mediawiki.org/wiki/Extension:Graph/Demo/Map to hewiki (or other wiki, not dewiki):

  1. "url": "wikiraw://de.wikipedia.org/Modul:Graph//WorldMap-iso2.json" - FAIL
  2. "url": "wikiraw://www.mediawiki.org/Extension:Graph/Demo/RawData:WorldMap-iso2-json" - WORKS