Page MenuHomePhabricator

Support for text/syntax/markup driven or WYSIWYG editable charts, diagrams, graphs, flowcharts etc. (Identify, develop, review and deploy extension on Wikimedia wikis to add)
Open, MediumPublicFeature

Description

Moving here the following proposals:
https://strategy.wikimedia.org/wiki/Proposal:Text_or_Syntax_driven_Charts,_Diagrams,_Graphs_and_more
https://strategy.wikimedia.org/wiki/Proposal:Editable_Graphics
https://strategy.wikimedia.org/wiki/Proposal:Markup_for_charts_and_graphs
(with some of the participants in cc).

  • «By now Wikipedia has markup/code for everything—math (TeX), parser functions, location maps, and even SVG is supported. However, what is notably missing from many articles (other than high-quality portraits) are graphs and charts.»
  • «There are many types of diagrams. The need for collaborative graphics is wide ranging, from map making to biochemical pathways to flow charts to genealogy to general charting. Each type of graphic has or could have a little language, XML or other representation to define graphics of that type succinctly. Whilst all can be presented as .pngs or .SVGs, the more succinct description is necessary for editability.»

This is distinct from bug 43616 (Provide a way to create interactive 2D/3D timelines and infographics e.g. Java applets, AJAX, Flash) in that it's not about *interactive* stuff and that there are some existing extensions which someone should assess and develop for Wikimedia purposes. I'd also leave maps out of it because that's being addressed (at last) elsewhere.
Part of it will be solved by bug 38271, but only to a degree.

Mentioned extensions: [[mw:Extension:Graph]], [[mw:Extension:GnuplotBasic]], [[mw:Extension:Plotters]], [[mw:Extension:WikiPlot]].

Keywords from the proposals: http://bloodgate.com/perl/graph/, Java .jar, Flash .swf, Silverlight .scr, ActiveX .ocx, Vector (.svg) .svg (XML), AnyWikiDraw, AnyWikiDraw, WikiPathways, WikiTex, Graph Plugin, DPL Bundle, TeX, gluplot, pgfplots.


Version: unspecified
Severity: enhancement

Details

Reference
bz54221

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:54 AM
bzimport set Reference to bz54221.
bzimport added a subscriber: Unknown Object (MLST).

What about reviving [[Extension:WikiTeX]] (not to be confused with [[Extension:WikiTex]])? This would have done graphs, plots, and more. It would be handy for it to also support simple circuit diagrams.

Correction, the links are: [[mw:Extension:WikiTeX]] (and [[mw:Extension:WikiTex]]).

Is there an interest in proposing this project for Outreach Program for Women?
If so, and if there at least one mentor for it. please move it to the "Featured projects" section. This way it will be automatically transcluded in
https://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_7

Thank you!

vladjohn2013 wrote:

Hi, this project is still listed at https://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects#Support_for_text.2Fsyntax.2Fmarkup_driven_or_WYSIWYG_editable_charts.2C_diagrams.2C_graphs.2C_flowcharts_etc.

Should this project be still listed in that page? If not, please remove it. If it still makes sense, then it could be moved to the "Featured projects" section if it has community support and mentors.

(In reply to comment #4)

Should this project be still listed in that page?

Yes.

If not, please remove it.
If
it still makes sense, then it could be moved to the "Featured projects"
section
if it has community support and mentors.

It lacks a mentor, plus (probably) someone to set a direction/the requirements (who may or not be the same person).

tl;dr; I would be interested in being a mentor here, and probably collaborating until a solution is reached.

We've done some thinking about this while we developed Limn [1] last year. I believe Limn was headed in the wrong direction, but we (team analytics) are interested in solving this problem and it would be nice to combine efforts with anyone that is also interested. Currently, I believe that Vega JS [2] is a very promising library. It has a great mixture of simplicity and power.

My personal opinion is that creating and editing infographics would work much like Visual Editor does. There would be a simple JSON editor on the front end, which could evolve over time. When saved, this would use a server to generate static infographics for consumption by the cache layer. The dynamic infographic could be accessed through a link on the static image. The cache could be repopulated with up-to-date infographics periodically if they depend on external data. For this vision, Vega JS is quite suitable as it can generate PNGs server side for example.

[1] https://github.com/wikimedia/limn
[2] http://trifacta.github.io/vega/

I'd suggest using a scripting language as markup for the editable charts, diagrams, graphs, flowcharts etc.

For end users this may be like the example:

Extension:Graph

G { "Bonn" --> "Berlin" [ style: bold; ] }
As
Graph( [ "Bonn" => [ "Berlin", "style"=>"bold" ] ] );
or
$bonn = node("Bonn");
$bonn->linkTo( node("Berlin", "bold") );
Graph( $bonn );

Extension:WikiPlot

<wikiplot height="200" width="200" caption="Simple plot" xspan="-100;100" yspan="-100;100">

<graph label="Graph 1." color="255,0,0">x+4</graph>
<graph label="Graph 2.">3*x-3</graph>

</wikiplot>

as

$plot = new Plot( ['height'=>200, 'width'=>200, 'caption'=>'Simple plot', 'xspan'=>[-100, 100], 'yspan'=>[-100, 100] ] );

or

$plot = new Plot();
$plot->height = 200;
$plot->width = 200;
$plot->caption = "Simple plot";
$plot->xspan = [-100, 100];
$plot->yspan = [-100, 100];

and

$plot.addGraph( ['x+4', 'label'=> 'Graph 1.', 'color'=>[255,0,0] ]);

$graph = new Graph( "3*x-3" );
$graph->label = 'Graph 2.'
$plot.add( $graph );

echo $plot;

It looks like directly code execution, but it is not.
I did mw:Extension:Foxway for testing possibility of this. I ascertained that this may work and may work quickly.
Now I do new mw:Extension:PhpTags. There any extension can register a hook as a constant, function or object.
So far I have only worked on the functions and constants, and did not start work on objects.
I am sure that the PhpTags will cover all the needs and will be convenient and powerful tool.
I would like to find someone interested in discussing this.

Thanks Pavel. I like the syntax you mention. The advantage to using Vega JS syntax would be that it's a healthy active project and we get the parser, etc. for free. Check it out if you haven't had a chance, I linked to it above.

Dan, thank you for pushing this proposal. If you want to jump to the GSoC 2014 train you need to:

  • Find a second mentor.

The sooner the better, since Wikimedia will be evaluated by the GSoC organizers in the next 1-2 weeks. You will know you are ready when your project looks great at https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014

More: http://lists.wikimedia.org/pipermail/wikitech-l/2014-February/074277.html

Thanks, Dan. Vaga is excellent.

But, for example, I need a template to draw a bar of population.
The template takes a comma-separated list of cities.
The population is taken from the wikidata.

{{MyTemplate

cities=London,Berlin,Paris

}}

Source of wikipage Template:MyTemplate:

... Some wiki markup ...
<phptag>
if ( isset($args[cities]) ) {

$cities = explode( $delimiter, $args[cities] );
$population = AskWikidata( 'Give me population of cities', $cities );
$data = array_combine($cities, $population);
echo new Vega( 
    $data,
    [ 'size'=>[400,200], 'scales'=>['x'=>'cities', y=>'population'] ] 
  );

}
</phptag>
... Some wiki markup ...

There are:

  • functions 'explode', 'array_combine' is hooks of the extension PphTags Functions. (already done)
  • function 'AskWikidata' is fictional, but can be implemented as hook of some extension
  • objects 'Vega' is fictional, but can be implemented as hook of some extension
  • 'if', 'isset', 'echo' and '[...] (array definition)' is functions of extension PhpTags (already done)

Suppose I'm developer of the extension Vega.
I should just take the javascript source code from Vega and register hook in the extension PhpTags.

source of Vega.php:

$wgHooks['PhpTagsRuntimeFirstInit'][] = 'VegaExtension::initializeRuntime';

class VegaExtension {

public static function initializeRuntime() {
  \PhpTags\Runtime::setObjectsHook( 'VegaObject', array('Vega') );
}

}

class VegaObject extends PhpTags\BaseHooks {

public static function onObjectHook( ... ) {
  // some code
}

}

The Class VegaObject will receive the arrays of data and parameters from the PhpTags,
and when the function 'echo' is called returns all the necessary data for drowing the bar on the wikipage.
a large part of the code (such as processing of various parameters) have to be implemented in the class PhpTags\BaseHooks.
The class VegaObject will just have to inform PhpTags that it needs 'vega.min.js' and convey to one ready parameters.
Since it is very simple, similar extensions can be done in one day.

Maybe I'm writing all this is off topic, forgive me please.
But I can not find anyone interested in this.
I guess I'm looking for not out there, or can not speak understandable.
Maybe I just doing nonsense and distract people from her work, I do not know.

I think what you're saying makes sense Pavel. And no worries, to participate in this community you have to be really good at dealing with distractions. I'm still waiting for someone else to help with mentoring on this important project. I think someone with mediawiki extension development experience would be great. Part of the reason I can't comment more, Pavel, is that I don't know anything about mediawiki extensions :)

Graph extension might be applicable, even though it is not dynamic (yet), or even in production wikis, it does support various timelines, e.g. "Napoleon" at http://trifacta.github.io/vega/editor/ :)

See https://www.mediawiki.org/wiki/Extension:Graph and samples at https://www.mediawiki.org/wiki/Extension:Graph/Demo

Discussion at https://meta.wikimedia.org/wiki/Wikimedia_Forum#Graphs.2C_Maps.2C_..._.28experimental.29

Sorry, meant to post last comment for bug 43616. Yet, for this request, graph ext is still applicable, except that it is not wysiwyg unless some brave soul starts hacking a proper vega editor.

I'd like to take this up as a project. Is this still open for taking up and developing for this year?

I'd like to take this up as a project. Is this still open for taking up and developing for this year?

It's open, but not ready. Please choose one of the extensions mentioned by this report and start filing patches against them. When you find one you're comfortable developing, you'll know the area you can work on.

@Milimetric, @Yurik, is there a possible project related to this task that students like @Sumit could plan for in the context of Google Summer of Code 2015 (and that you would like to mentor)? Something about MediaWiki-extensions-Graph ?

Wikimedia will apply to Google Summer of Code and Outreachy on Tuesday, February 17. If you want this task to become a featured project idea, please follow these instructions.

@Qgil, I can totally see this becoming a GSoC project, but most of the mentorship would have to come from the UI/Visual Editor team because they have the expertise in building custom editors. The student would build an editor with a dynamically updated viewer for the VE. I already have a simplified version of that here -- click preview and try changing the graph definition.

This task is about wikitext. VisualEditor support for the markup is a separate issue.

Actually, this was one of the rawest ideas in the project list and is absolutely not usable for GSoC, unless a student comes who has an extremely clear idea of a feature to add and a plan for it. I'll move the possible project tag to T56222. The other blocker, T40271, would also be suitable, but I doubt it would benefit from GSoC (iterative GSoC cycles over the same problem rarely helped anything).

Yurik, how does this address "biochemical pathways to flow charts to genealogy" from the description?

@JamesCrook, we have to narrow down this task, otherwise it sounds like "create a platform that allows everything", which is not really actionable. From my reading of the description, Vega 2.0 should be able to handle most of it. If not, please create separate task that we can discuss.

Just found this expandable tree sample that seems to be based on Vega with some additional hacks. Might be interesting to build something similar. And of course there are existing samples like heatmap and many other that might be useful.
CC: @JamesCrook

@Yurik Vega doesn't currently seem to support those, at least not from the examples I can find. They are motivating examples, so should not be swept under the carpet. Biochemical pathways, flow charts and genealogy are all similar in being based on graphs (i.e. networks). The Vega examples I am seeing are plots and maps.

I am wondering why I am seeing no force directed layouts with Vega, although the visual encoding transform is described in the documentation. If we can get even a basic demo of some biochemical pathway done using a force directed layout then I am cool with this task being closed. Elaboration and prettiness can come later.

@Yurik , looking at the source for that tree it looks to be d3.js rather than Vega.

@JamesCrook, you mean this example? Also here is an interactive version.

As for d3 - vega is based on d3, so, in theory, most of the capabilities of d3 are exposed. But yes, we will have to dig through it.

@Yurik OK. I'm happy now. Yep, I did know d3.js was under Vega, just previously had doubts that enough of that flexibility was exposed. We can't have end users writing arbitrary javascript, so how the functionality is exposed really matters.

I think I am already now OK with this task being closed. There is obviously tons more to do and tons more that could be done, but the basics of the template driven transform of json to svg or canvas are there, which is what this task was asking for.

Yurik claimed this task.

Closing because we now have <graph>, and because this task is too broad. Please add more specific tasks for anything that's missing :)

This bug's purpose is precisely to collect requirements and map alternatives. Two of them are identified by the two blockers. :) If you think the usecase behind that extension is another, please add to blockers.

@Nemo_bis, this is not what the bug description states. I think the better use of phabricator would be to create small, actionable, and well scoped tasks for specific feature, and mark those with some project tag for tracking. Otherwise this task grows infinitely large and not very useful. Having small tasks allows each feature to be discussed separately, and different devs can pick them up for implementation. The only reason why we had the [tracking] tasks in Bugzilla was because there was no good support for tagging (projects).

Actionable tasks are not the only use case for Phabricator; cf. for example the Epic project. Additionally, I think tasks with subtasks are very preferable to projects in some cases as they themselves have a status, a dependency tree, etc.

This task has two blocking subtasks that have not been resolved, so this task itself does not seem to be resolved. AFAIUI, for example you don't want to work on T56222 (which is fine), but that also means that someone else needs to do the deed to convert those cladograms to Extension:Graph (or something else).

Yurik removed Yurik as the assignee of this task.May 20 2020, 7:03 PM

nowadays there is an extenstion https://www.mediawiki.org/wiki/Extension:Diagrams , would enabling it be a step into the right direction?

Aklapper changed the subtype of this task from "Task" to "Feature Request".Feb 4 2022, 12:23 PM
Aklapper removed a subscriber: wikibugs-l-list.