Very important issue to consider is (1) we need to be able to use template and template parameters inside chart syntax (e.g. {{#chart:foo|param1={{foo}}|param2={{{1}}}}}; (2) Lua module should be able to generate chart tag with variable number of parameters.

Another thing is easy integrate with WDQS result - embedding it server side needs T67626 which is too far to consider; rendering it client side is much easier.

I've expanded the task description to include another option for having chart definitions as their own pages. This would limit the need (but also the ability) to use templates in chart syntax.

Going to get some links from Chris on past Graphs usage that'll help me in this research :D

links obtained

A suggestion for supporting both server side rendering, and future interactivity at client side:

Define graphs as webcomponent like

<graph src="path to wiki page with json"></graph>

MW can consider this as parser function and add a server side rendered image in it. So it becomes the following html.

<graph src="path to wiki page with json">... server side rendered image goes here.. </graph>

This is valid html. Browsers just treat these unknown tags as div tags. The image will be rendered.

At client side, define a custom element - webcomponent - that enhances this feature to add interactivity. Replace the SVG with say, <canvas> or whaver a chart library does.

This means, our graphs can be reused anywhere in web, by including the webcomponent definition and adding this html snippet <graph src="path to wiki page with json"></graph> without MW dependency.

I would like to see charts from wikipedia easily reusable in webpages.

An example implementation: https://codepen.io/santhoshtr/pen/wvLBoLr
Screenshot based on COVID-19 cases in Santa Clara County, California:
{F56326792 size=full}

Some quick notes catching up:

We still want to decide whether we want to send all params and data in with the JSON data pages (fixed) or as wiki markup parameters on a parser function (meaning people can use templates and lua modules to expand or process data).

If we push everything to the json blob we might have an invocation like this:

{{#chart:Foobar.json}}

-> Foobar.json:

{
    graph: "line",
    width: 600,
    height: 400,
    x: {
        label: {
            // Should we use the MW namespace for localization instead?
            "en": "Day",
            "fr": "Jour",
        },
        // Should data be queriable from some source?
        data: [1, 2, 3, 4]
    },
    y: {
        label: {
            "en": "Degrees C",
            "fr: "..."
        },
        data: [0, 10, 20, 30],
    }
}

The advantage of the separate page is the same definition and data set can be reused very easily. The downside is you have to set up more stuff outside your page to set up an invocation. Potential advantage is we can have a nice visual editor for the chart. Disadvantage is that's out of scope and we won't get to it on this project.

Questions:

• should labels be localizable in a shared definition? or let them be customized as copies
• should data sets exist separately from definitions?
• should definitions exist as standalone json data pages or be invocation parameters?

In the latter case you might see:

{{#chart:graph=line
|width=600
|height=400
|xAxisLabel=Day
|x=1, 2, 3, 4
|yAxisLabel=Degrees C
|y=0, 10, 20, 30
}}

In the inline case, localizations would not be dealt with directly -- you'd localize it when you copy the template or inline invocation. And you could potentially use wikidata queryies for the x & y data series and labels.

However note that there's not likely a good cache invalidation story on such things, I don't know if any of that's ideal right now.

Secondarily, if we need to be able to query, do we need to be able to subset/filter? The filter options in vega are javascript and thus dangerous, so we want to be very careful and explicit about any filter language we define. It's simplest to avoid this and leave it to the existing wikidata query modules or whatever?

In T368350#9981853, @bvibber wrote:

Secondarily, if we need to be able to query, do we need to be able to subset/filter? The filter options in vega are javascript and thus dangerous, so we want to be very careful and explicit about any filter language we define. It's simplest to avoid this and leave it to the existing wikidata query modules or whatever?

I think the minimum we would need is the ability to select only some columns from a larger data set. For example, for a table with ballot measure election results, you'd want to be able to pull out just "County" and "For %" to make a simple bar chart.

I think the minimum we would need is the ability to select only some columns from a larger data set. For example, for a table with ballot measure election results, you'd want to be able to pull out just "County" and "For %" to make a simple bar chart.

*nod* selecting columns by index or name should be pretty straightforward from .tab data pages, which provide a field name for each column. Tabular pages also have a facility for localizing column labels, which sounds useful and avoids having to duplicate it in the invocation.

A range limit might be useful too, but I don't want to add too much.

That also leaves us open to the possibility of generating dynamically updatable query data: pages in the future, that act like the JSON tabular pages but are filled with data from a sparql query to wikidata...

So I think it'd be very useful to support .tab Data pages now, and future stuff later. Do we _also_ need to accept raw data in a data definition page (Data:Foobar.chart ?) / raw parserfunction invocation? That's one we can't walk back, even if we provide better tools alongside. ;)

Note the chart definitions could live in the Data: namespace

One of common types of graphs is annual passagers of an airport. Data of such graphs is filled from Wikidata.

Another type is list of historical page views of a page. Data comes from API

There are a number of graphs whose data come from WDQS.

Only allowing pages in Data namespace in Commons may not be enough. This is some use cases it can not solve:

• Some local wikis may want to store their project-internal data, such as number of open block appeals per day. They may hope they can use a page in local wiki (not Commons), convert it to Tabular Data data model, then it can be edited and used like Commons Data page (cf T252711).
• A third party wiki installed Chart. They may want to store data in their wiki without using a dedicated namespace.
• (maybe out of scope) Again a non-WMF wiki installed Chart, but they want to use data from Wikimedia Commons. So Chart should support fetching data via API instead of database access.

Some local wikis may want to store their project-internal data, such as number of open block appeals per day. They may hope they can use a page in local wiki (not Commons), convert it to Tabular Data data model, then it can be edited and used like Commons Data page (cf T252711).

Note that there's nothing preventing using Commons to store this data.

A third party wiki installed Chart. They may want to store data in their wiki without using a dedicated namespace.

I don't think there's any particular reason to need to support it without a dedicated Data: namespace. If they just dislike the aesthetics, then it's on them to figure out how to make the necessary changes to support per-page content types and maintain that code.

(maybe out of scope) Again a non-WMF wiki installed Chart, but they want to use data from Wikimedia Commons. So Chart should support fetching data via API instead of database access.

This isn't at all inconsistent with us using the Data: namespace on commons, as shown by the existence of InstantCommons for media files. Out of scope for now, but 100% consistent with our plans already.

https://www.mediawiki.org/wiki/Extension_talk:Chart/Project#Data_source