I'm not done writing this, but feel free to tell me if you guys think I'm not on the right path ;)

Should we call it GraphEditor or GraphVeEditor instead? Data seems incorrect.
My biggest concern with wrapping data definition into edit forms is that the definition tends to change very rapidly. If you really plan to do it, you should contact Vega JS people to oversee the design and give some feedback. As their schema changes, these forms should stay relevant. They are currently adding animation aspect. Also, your forms should always be ready for an element that it does not know about. For example, if JSON contains an element "foo" at any level, it should stay there after editing, and should be shown to the user as part of the form.

We should actually just keep the files in the Graph Extension, there's no need for a separate extension. See Extension:Math for an example:

https://github.com/wikimedia/mediawiki-extensions-Math/tree/master/modules/VisualEditor

@Yurik, which reminds me, probably @Mooeypoo will need +2 on the Graph extension in order to make the review process run smoothly.

@Mvolz, no problems with +2, but gerrit permissions are scary, any experts in that? :)

@Yurik: As far as the name goes, I've simply based it on the existing TemplateData, which is already used in VE to edit preformatted data. I have no problems with changing the name if you guys can come up with something better.

@Mvolz: Ahh yes, I agree it would make a ton of sense to develop a VisualEditor module for the Graph extension instead. I'll try to edit it ASAP.

@ferdbold, @Mvolz, please comment on my other concern re structured data. The naming of the project is less important :)))

@Yurik: Would it be sufficient to add a "raw data" tab of some sort to the UI who would be simply displaying the JSON hierarchy in a generic manner? This would allow users to edit all graph data with the UI without having to wait for support as Vega pushes out new changes.

Sure, as long as all edits on the structured tab immediately reflect in the raw tab, and vice-verse. And obviously the structured tab should support things like missing data (not everything is required). Btw, re picture -- each data source might be external/internal, so it should be a property of each source, not a drop down at the top. Thanks!

Okay, eventually when this project is done, there should not be JSON blob exposed to the user from within VisualEditor, as that is going against the point of having a visual editor rather than a source one.

However, in the beginning stages, we could start with a small inspector with the json blob to give the user the ability to change the data while we build a better user experience. We should take into account that this is temporary, though, and ideally should not really be part of the user experience in VE. We need to also make sure we are validating the json blob and only apply it if it's valid. Considering VE's inspector behavior, that might be a bit of a messy user experience in the beginning -- but will become a much better experience at the end of the project.

After talking about this more with the rest of the VE team, we might want to change the order of the stages to make this better on the user (@Yurik, I think you will like this better too) -- we might be better off doing the live rendering *first* in the first stage, including Vega in VE and making the graph appear as a focusable node that has a very basic JSON-blob inspector. The next step would then slowly build a better UI for controlling the data; it can start with still having a json-blob editor but allowing for easier switching of basic configuration options like graph type or name, etc. Then we can proceed with having a full blown UI for the entire option set.

@Mooeypoo, I don't think it would ever be a good idea to get rid of the "raw" view - graph will always change before the editor does, plus copy/pasting definition to a separate editor is always a very useful design approach, while still using the visualizer of VE. Look at every other attempt to "structurize" development languages or HTML editors - it has always failed, keeping raw text entry a the best approach, while at the same time doing structure coloring/autocomplete/other text-based aids.

You can never have a proper validation as an add-on - validation can only be done by the Vega itself. If you modify Vega to expose validator - good, otherwise, lets not to even try.

And yes, I do think doing a live-updated graph would provide the highest value at first.

The 'raw' view has no place in VisualEditor, though, just like wikitext has no place in VisualEditor. It will exist there while we have no other way to let the user edit things, but it shouldn't exist there indefinitely, and if it does, it should be a very very back-end and hidden option.

In any case, this will probably be done as a final stage at the very end of this project (likely beyond the scope of the GSoC project anyways) so we can revisit this in the future.

As for validation, yes, we'll need to validate the json blob itself *and* the json blob's data (Vega). It can't be an optional add-on, however, at least not the basic json validation itself, because it can corrupt the wikitext and prevent MW from saving it. The user may lose information otherwise. We can have that validation on two fronts, though -- a basic raw-json validation to make sure nothing breaks, and a proper Vega-related validation to make sure the data is what we want it to be.

We should have this conversation in the main bug, by the way, especially since there are other students interested in the same project.

It sounds good in theory, but might not work in practice. Take a look at these graphs - Each graph is a template that uses a template that uses a graph page. Graph expands template parameters to provide it with the source data. Thus, that graph is actually not a valid JSON. Since you should never have two graph-data validators (one inside the vega, and one outside), because they will always get out of sync, you will need to validate against the Vega validator. But it won't work until you expand the template parameters. And by the way, I think this will be the most common way graphs will be used - as templates with parameters, not as all-included blobs on wiki pages.

A template is a whole new layer of complication in VisualEditor. If this is a template of a template of a graph, then VE will probably read it as a template first, which has completely different UI mechanisms than a plain ol' graph. If this is common behavior, we'll have to figure out a specific solution. Templates are completely different things than straight forward extensions like graph data, and VE treats them as such. It will probably mean that VE will display the parameters of those graphs as plain JSON just because it treats it as a template rather than an extension. That happens to practically any template that encapsulates an extension.

Whether we want to add this as a future feature or figure out how to untemplate-ify VE's handling of these templates (eh) -- this will most probably (and by that I mean almost absolutely) will not be part of the GSoC project scope.

We'll deal with templates of templates of graph data when we have the non-edge-case-basic-implementation up and running. We will start with a JSON blob available for editing, but we should in general design this (a) with the idea of shying away from raw json editing and (b) with the idea that these data blobs should probably, at some point, be part of wikidata. (Though not in the near future, probably)

The graph extension has been launched on all production wikis.

Here is a new version of the timetable following the conversations on this task and with my mentors. Feel free to ask questions and to amend :)

M56 Revised timetable for "Allow VisualEditor support in Graph extension"

10