Page MenuHomePhabricator

Enable VisualEditor support in Graph extension (GSoC 2015 Proposal)
Closed, DuplicatePublic

Description

THIS IS A DRAFT OF THE FINAL PLAN FOR THIS PROJECT. THE FINAL PLAN IS AVAILABLE HERE T89287: Graph editing in VisualEditor

Profile

Name: Frédéric Bolduc
Email: bolducfrederic@gmail.com
IRC or IM networks/handle(s): ferdbold on freenode
Web Page / Blog / Microblog / Portfolio: Personal portfolio being developed, you can look at some of my work in the meantime on https://linkedin.com/in/bolducfrederic (French alert!) and https://www.github.com/ferdbold
Resume (optional): http://bit.ly/1IkmK2K (French right now, English one coming soon™)
Location: Quebec
Typical working hours: 9 to 17, GMT-5

Synopsis

This project aims to develop an extension for Extension:Graph which will allow users to edit Vega specifications (in the form of JSON strings) from a tailored UI inside VisualEditor, without having to open up the page source, thus reducing user-error risk.

Mentors: @Mooeypoo and @Mvolz

Deliverables

Right now, Vega specifiations are considered by VE as a alien extension and are not rendered at all in the editor. My proposition is to divide this project into 5 core steps, which will need to be completed sequentially, but are all viable on their own (so the project could be shipped as-is at the end of any step):

  1. Enable Vega support into VisualEditor, in order for graphs to be properly rendered inside the edition page
  2. Enable real-time Vega previews in VE during edition, for all supported chart types. (At this point, this will work through raw JSON edition, but will be phased out as the UI develops)
  3. Develop a Vega parser allowing the module to read and edit bar chart specifications
  4. Implement an extensible UI dialog inside VE which can support read and edit operations on Vega specifications
  5. Extend progressively the module to other chart types (namely area charts, point charts, pie charts, maps and force-directed graphs)

Proposed timetable

This is being actively refined in the following mock. Once a final timetable is agreed there, it will be posted here.
  • KEY DATETASKDeliverable
    --
    March 27Application deadline
    April 27Student applications approved
    Until May 25Community bonding, research on MV extension development, basic prototyping (as much as finals allow!)
    May 25 - 31Design GraphData's UI, define the JSON parser's architectureMockup bundle, class diagrams
    June 1st - 7Enable Vega support into VE, Integrate basic UI framework for debug purposesStatic UI instanced on VE
    June 8 - 14Implement GraphData's JSON parser reading ability *(bar charts only)*Simple UI filled with data fetched from JSON
    June 15 - 25Implement GraphData's JSON parser editing ability *(bar charts only)*Simple UI able to save JSON data for a bar chart
    June 26 - July 3Midterm evaluation
    July 3 - 9Implement GraphData final UI (see below for module list), ability to create graphsGraph creation process implemented, support for multiple chart types
    July 10 - August 2Extend JSON parser functionality to other chart typesExtended UI able to support as many chart types as time allows
    August 3Feature freeze
    August 4 - 16Bug-fixing, documentation polish
    August 17Pencils down
    August 18 - 22Final evaluations

    Early UI mockups

  • Participation

    I've been actively using Git (and other VCSes) for a few years now, so I've got the basic code review workflow and collaboration aspect pretty well established.

    While I'll be working on this project, I'll submit my code for review daily to Gerrit. I like to keep things backed up, and doing this daily ensures that other developers will be able to keep track of my work much better.

    Also, these are the additionnal means I plan to use to communicate progress:

    • IRC: I'll stay online on freenode in #mediawiki-visualeditor as ferdbold during my working hours
    • I'll setup up a personal blog on ferdbold.com, on which I will post a weekly summary of my work.
    • I will post all weekly summaries and reports on this MediaWiki page.

    About you

    Education
    I'm a video game development undergrad in my second year at Université du Québec à Chicoutimi.
    In 2013, I graduated in Multimedia Integration (which is pretty much fancy-talk for front-end web development) at Cégep de Sainte-Foy.

    How did you hear about Google Summer of Code?
    I actually don't remember how I first heard about Google Summer of Code, I just know it was a while ago when I didn't know how to code and it looked like a great experience. Last year though, I went to a local dev meetup where it was mentioned, but it was already past the student application deadline. So here I am this year ;)

    Will you have any other time commitments, such as school work, another job, planned vacation, etc., during the duration of the program?
    I am heavily involved in a personal project of mine named Terra Arcana, a live-action role-playing organization who creates events every summer and supports the local LARP community. This year, our events will be held at the end of May and mid-August, which means I will have to devote some more time to them during these periods. Overall though, this should not hinder significantly the time I intend to devote to GSoC.
    (Psst... I actually made that website!)

    We don't just care about your project -- you are a person, and that matters to us! What drives you? What makes you want to make this the most awesomest wiki enhancement ever?
    What initially sparked my interest for this particular project was that I've previously developed a node-based graph editor with MooTools and KineticJS on Terra Arcana's website, fetching . This editor is used as a skill tree that players of a role-playing game can use to customize their characters (inspired by Path of Exile's own web-based passive skill tree). I've really enjoyed doing this, so this task looked like it'd be right up my sleeve!

    (This node-based editor is available at http://www.gnterraarcana.com. Sadly, it's kind of hard to find right now, since you have to actually make an account on the website and then make a new character and access their Skills ("Compétences" in French) page.)

    As a video game developer, I like to work daily with visual assets and UI is one of the aspects of game dev that I enjoy the most. As such, I really like working with graphs and statistics, as well as design and deploy UI. I have the confidence that I have the dedication and patience needed to make this extension reliable and fun to use.

    Past experience
    Since my Cégep graduation in 2013, I have been working on web design and development with multiple agencies, as well as by myself. I've mainly dabbled in WordPress development, as well as a lot of responsive web design, as I've often been tasked to take the lead in front-end integration. While I haven't worked on any open source projects yet, I'm confident that my prior work experience will serve me well. Having already worked as a freelance designer and developer makes me even more confidence that I am able to give it my all during the entire summer.

    I'm also a big fan of game jams. I've done almost ten of them now, including Global Game Jam, and I'll be adding the Ludum Dare to that list in three weeks. I always have a blast doing jams, and most of my games' source code is available on my GitHub account (since these projects are small, I don't really count them as open source experience).

    I've also participated in various competitions and hackathons, such as Hack Québec, Ubisoft's Academia and the CS Games. These kinds of events helped round out my skills, as I've picked up useful knowledge on domains like VCSes, teamwork (especially with strangers!) and code rigor.

    Other info

    I also have a Bitbucket account, though I'm mostly working with Github now so stuff on there tend to be old. For the curious only ;)

    I am not applying for Outreachy Round 10, as I'm not eligible.

    Microtasks submitted

    https://gerrit.wikimedia.org/r/#/c/200177/
    https://gerrit.wikimedia.org/r/#/c/200319/

    Event Timeline

    ferdbold claimed this task.
    ferdbold raised the priority of this task from to Medium.
    ferdbold updated the task description. (Show Details)
    ferdbold updated the task description. (Show Details)
    ferdbold set Security to None.

    I'm not done writing this, but feel free to tell me if you guys think I'm not on the right path ;)

    Should we call it GraphEditor or GraphVeEditor instead? Data seems incorrect.
    My biggest concern with wrapping data definition into edit forms is that the definition tends to change very rapidly. If you really plan to do it, you should contact Vega JS people to oversee the design and give some feedback. As their schema changes, these forms should stay relevant. They are currently adding animation aspect. Also, your forms should always be ready for an element that it does not know about. For example, if JSON contains an element "foo" at any level, it should stay there after editing, and should be shown to the user as part of the form.

    We should actually just keep the files in the Graph Extension, there's no need for a separate extension. See Extension:Math for an example:

    https://github.com/wikimedia/mediawiki-extensions-Math/tree/master/modules/VisualEditor

    @Yurik, which reminds me, probably @Mooeypoo will need +2 on the Graph extension in order to make the review process run smoothly.

    @Mvolz, no problems with +2, but gerrit permissions are scary, any experts in that? :)

    @Yurik: As far as the name goes, I've simply based it on the existing TemplateData, which is already used in VE to edit preformatted data. I have no problems with changing the name if you guys can come up with something better.

    @Mvolz: Ahh yes, I agree it would make a ton of sense to develop a VisualEditor module for the Graph extension instead. I'll try to edit it ASAP.

    @ferdbold, @Mvolz, please comment on my other concern re structured data. The naming of the project is less important :)))

    @Yurik: Would it be sufficient to add a "raw data" tab of some sort to the UI who would be simply displaying the JSON hierarchy in a generic manner? This would allow users to edit all graph data with the UI without having to wait for support as Vega pushes out new changes.

    Sure, as long as all edits on the structured tab immediately reflect in the raw tab, and vice-verse. And obviously the structured tab should support things like missing data (not everything is required). Btw, re picture -- each data source might be external/internal, so it should be a property of each source, not a drop down at the top. Thanks!

    Okay, eventually when this project is done, there should not be JSON blob exposed to the user from within VisualEditor, as that is going against the point of having a visual editor rather than a source one.

    However, in the beginning stages, we could start with a small inspector with the json blob to give the user the ability to change the data while we build a better user experience. We should take into account that this is temporary, though, and ideally should not really be part of the user experience in VE. We need to also make sure we are validating the json blob and only apply it if it's valid. Considering VE's inspector behavior, that might be a bit of a messy user experience in the beginning -- but will become a much better experience at the end of the project.

    After talking about this more with the rest of the VE team, we might want to change the order of the stages to make this better on the user (@Yurik, I think you will like this better too) -- we might be better off doing the live rendering *first* in the first stage, including Vega in VE and making the graph appear as a focusable node that has a very basic JSON-blob inspector. The next step would then slowly build a better UI for controlling the data; it can start with still having a json-blob editor but allowing for easier switching of basic configuration options like graph type or name, etc. Then we can proceed with having a full blown UI for the entire option set.

    @Mooeypoo, I don't think it would ever be a good idea to get rid of the "raw" view - graph will always change before the editor does, plus copy/pasting definition to a separate editor is always a very useful design approach, while still using the visualizer of VE. Look at every other attempt to "structurize" development languages or HTML editors - it has always failed, keeping raw text entry a the best approach, while at the same time doing structure coloring/autocomplete/other text-based aids.

    You can never have a proper validation as an add-on - validation can only be done by the Vega itself. If you modify Vega to expose validator - good, otherwise, lets not to even try.

    And yes, I do think doing a live-updated graph would provide the highest value at first.

    The 'raw' view has no place in VisualEditor, though, just like wikitext has no place in VisualEditor. It will exist there while we have no other way to let the user edit things, but it shouldn't exist there indefinitely, and if it does, it should be a very very back-end and hidden option.

    In any case, this will probably be done as a final stage at the very end of this project (likely beyond the scope of the GSoC project anyways) so we can revisit this in the future.

    As for validation, yes, we'll need to validate the json blob itself *and* the json blob's data (Vega). It can't be an optional add-on, however, at least not the basic json validation itself, because it can corrupt the wikitext and prevent MW from saving it. The user may lose information otherwise. We can have that validation on two fronts, though -- a basic raw-json validation to make sure nothing breaks, and a proper Vega-related validation to make sure the data is what we want it to be.

    We should have this conversation in the main bug, by the way, especially since there are other students interested in the same project.

    It sounds good in theory, but might not work in practice. Take a look at these graphs - Each graph is a template that uses a template that uses a graph page. Graph expands template parameters to provide it with the source data. Thus, that graph is actually not a valid JSON. Since you should never have two graph-data validators (one inside the vega, and one outside), because they will always get out of sync, you will need to validate against the Vega validator. But it won't work until you expand the template parameters. And by the way, I think this will be the most common way graphs will be used - as templates with parameters, not as all-included blobs on wiki pages.

    A template is a whole new layer of complication in VisualEditor. If this is a template of a template of a graph, then VE will probably read it as a template first, which has completely different UI mechanisms than a plain ol' graph. If this is common behavior, we'll have to figure out a specific solution. Templates are completely different things than straight forward extensions like graph data, and VE treats them as such. It will probably mean that VE will display the parameters of those graphs as plain JSON just because it treats it as a template rather than an extension. That happens to practically any template that encapsulates an extension.

    Whether we want to add this as a future feature or figure out how to untemplate-ify VE's handling of these templates (eh) -- this will most probably (and by that I mean almost absolutely) will not be part of the GSoC project scope.

    We'll deal with templates of templates of graph data when we have the non-edge-case-basic-implementation up and running. We will start with a JSON blob available for editing, but we should in general design this (a) with the idea of shying away from raw json editing and (b) with the idea that these data blobs should probably, at some point, be part of wikidata. (Though not in the near future, probably)

    ferdbold renamed this task from GraphData extension for VisualEditor (GSoC 2015 Proposal) to Enable VisualEditor support in Graph extension (GSoC 2015 Proposal).Apr 29 2015, 2:30 AM

    The graph extension has been launched on all production wikis.

    Here is a new version of the timetable following the conversations on this task and with my mentors. Feel free to ask questions and to amend :)