First question: Is there a migration document to follow?

Good point Andre, I added a subpoint which was implicit in the former first point.

Replacing single-purpose tool with multipurpose one is always good idea

I don't think that's necessarily a true statement...

For the actual bug, anecdotally I think a lot of users find the timeline syntax easier to understand than the graph syntax (Both are pretty hard though).

@Bawolff I would even say that general purpose language tends to be more complicated that the domain-specific language. That said, I think we should utilize Lua and templates to solve this - in other words, create a domain language, specifically for timelines, and have Lua transform it into a generic Vega syntax. This way the language can be developed on-wiki, by the entire community, without the need of the lengthy js/php modification.

How closely can the EasyTimeline syntax be matched by Graph syntax? Is it possible to automatically migrate them? If yes (and I hope it's "yes"; or maybe "yes in 90% of cases, except where the user did something very weird"), can we replace EasyTimeline with a small extension that depends on Graph, has the same syntax as EasyTimeline (or a wide subset of it), and converts on the fly to Graph syntax, rather go through the grueling process of updating all the wikitext pages?

@matmarex, i am afraid it's not. EasyTimeline's syntax is fairly complex and non-standard (a cross-breed of config files, yaml, and Basic as far as i can tell). Even the simple parsing is a challenge for Lua code, but I'm sure it can be done (or we could reuse EasyTimeline's own parser if it exists outside of the rendering). Vega grammar is not context-aware - its basically a set of D3-like operations, like take this data, transform it with these expressions, scale it this way, and draw it with these visual primitives. It is much more generic, powerful, and hard to work with.
Also, EasyTimeline inserts clickable links - something that Vega can only do in the "interactive" mode (after you click the image to activate it, and it loads all the javascript and required data, and redraws it on the client). I had a discussion about generating clickable SVGs, but that might be some time away. Alternatively the EasyTimeline wrapper could generate some image map as part of the parsing.
Yet,I'm not sure the server-side wrapper is a good idea - what's the point of replacing custom engine with Vega engine if the result is identical, and all existing pages will continue to have existing syntax? We might as well leave both in place. Instead, I really think we should have a lua script that can generate the required vega graph, and a bot that will go throw the existing graphs and replace them with that script call. This way the community can continue to improve the Lua module after the conversion script is done.

what's the point of replacing custom engine with Vega engine if the result is identical, and all existing pages will continue to have existing syntax?

Huge reduction in tech debt and huge improvement in maintainability. Most of EasyTimeline is written in densely packed Perl with Perl dependencies. Even if the new code would be just as tough, it would at least be in PHP or JavaScript, which are familiar to us.

The generated images would probably be a huge improvement too. EasyTimeline can't produce SVG, and it can't produce 2x PNG, and it can't do anything else to make them prettier on high DPI screens. And the normal DPI output also looks quite ugly (it seems to have some super broken font antialiasing, especially on links, and the kerning is worse than if there were none at all).

@matmarex, I agree about the output, but I'm more concerned with the wiki markup. Implementing and maintaining a very complex parser for the original syntax could be about as much work as implementing a conversion script into something more "wiki-like", e.g. a template call with a number of predefined parameters. And once done, we won't ever have to deal with it again. Instead we will have a set of well known templates (one for each type of the timeline), with a well defined template parameters (allowing much better VE integration with parameter entry). Continuing support for the <timeline> tag with all of its custom markup might be much more resource demanding, especially in the longer run.

Can we get an example of using Graph to generate at least a basic timeline? That might help starting the manual conversion process :]

@hashar, sure, could you give me an example of a "basic" timeline? :)

EasyTimeline inserts clickable links - something that Vega can only do in the "interactive" mode

This is a feature used in some highly visible timelines, BTW.

In T137291#2794524, @Yurik wrote:

@hashar, sure, could you give me an example of a "basic" timeline? :)

https://www.mediawiki.org/wiki/Extension:EasyTimeline#Code_example is rather simple: it has a single line, several events and a date range. My idea is to eventually point EasyTimeline to an example that looks familiar to them so they can migrate from it :]

In T137291#2794703, @Nemo_bis wrote:

EasyTimeline inserts clickable links - something that Vega can only do in the "interactive" mode

This is a feature used in some highly visible timelines, BTW.

Upstream feature request: Generate SVGs with links

In T137291#2794783, @hashar wrote:

https://www.mediawiki.org/wiki/Extension:EasyTimeline#Code_example is rather simple: it has a single line, several events and a date range. My idea is to eventually point EasyTimeline to an example that looks familiar to them so they can migrate from it :]

https://www.mediawiki.org/wiki/Extension:Graph/Demo#Timeline_.2F_lifeline comes probably closest here?

(Broader picture: https://en.wikipedia.org/wiki/Wikipedia:Graphs_and_charts#Timeline and https://en.wikipedia.org/wiki/Wikipedia:Timeline )

In T115188#4196921, @Amire80 wrote:

EasyTimeline relies on Ploticus, an upstream graphics library. It was possbly good for the task in mid-2000s when EasyTimeline was developed, but by current standards it is generally buggy, has poor graphics quality and poor Unicode support, and most importantly—it hasn't been maintained since 2012.

The most realistic thing to do about EasyTimeline is to stop using it for creating new timelines, and to migrate to the Graph extension. Yes, it will require a lot of work, but I cannot think of anything better. If anybody can develop an automatic converter from EasyTimeline's unique syntax to the Graph extension's syntax, it will be pretty awesome, but it's definitely not my expertise. (An opportunity for a grant-funded project, I'd say.)

Provide a tool to find all current content to migrate

For the records, T137310 has been resolved. Pages such as
https://en.wikipedia.org/wiki/Category:Pages_using_Timeline
https://de.wikipedia.org/wiki/Kategorie:Seiten,_die_Timeline_verwenden
https://es.wikipedia.org/wiki/Categor%C3%ADa:P%C3%A1ginas_que_usan_Timeline
now exist.
Local page names can be found via editing the language code xy of the URL https://translatewiki.net/w/i.php?title=MediaWiki:Timeline-tracking-category/xy

Are the communities aware of this ticket? On frwiki, the extension is massively used and I believe that no one has planned to replace them with the Graph extension. Is there a documentation to organize this transition?

In T137291#5260785, @Lofhi wrote:

Are the communities aware of this ticket?

Probably not because nobody has worked on this. See the unchecked checkboxes in the task description above.

Is there a documentation to organize this transition?

No. Please see the unchecked checkboxes in the task description above.

I saw the unchecked checkboxes above, but the ticket was opened in 2016 and I was not a contributor at that time. It may not be up to date. All the details are good to start making the necessary changes on frwiki. Thank you for confirming.

Here are search pages from each of the big ten Wikipedias on usage:

There are just two uses on Commons, and only fifty on Wikidata.

Obviously our Cyrillic-speaking friends have used this a lot more than everyone else, and will need special efforts.

There is also Category:Pages using Timeline on many wikis if someone wants to look at existing use cases.

In T137291#5263830, @Jdforrester-WMF wrote:

Obviously our Cyrillic-speaking friends have used this a lot more than everyone else, and will need special efforts.

FYI: many usages in Russian Wikipedia are bot/batch-created articles with pretty much the same code, so it’s easy to fix once you get community consensus. I can try to do some cleanup in this department if it would be helpful.

For ‘known usages’: {{Graph:Chart}} really needs ability to do horizontal bar charts and maybe some ability to change the angle for values.

Note that while EasyTimeline is limited and feels outdated, it has a light footprint and maintenance burden and has (almost?) never been the subject of production errors or issues with security, performance, caching, etc. Its code is a single Perl file that has needed little to no changes in over a decade.

The Graph extension on the other hand, has been subject of many such issues, and involves a substantial amount of PHP code, front-end JS, and the Graphoid Node.js service.

This by itself isn't an problem per se, but given that currently neither has a steward (listed as Editing, but it's been clarified recently that this is intended to apply to the VE plugin only). See also T211881.

It hurts to even say this, but unless and until either of these is resourced by Product, the lesser of these two evils is clearly EasyTimeline (as measured by shared burden for Tech and Product teams, development velocity, and infra cost).

I think I would be ideal if Graph and Graphoid become stewarded, and then we can go all-in on this migration. But lacking that, should we promote this migration, knowing it will demand more unallocated time to maintain, and make sunsetting even harder?

@Krinkle, I agree, we should make Graph/Graphoid maintainable before migrating anything to it. At the very least we would need to support Vega 4 and Vega Lite syntax, and address the problems of storing graph definitions and graph data. But maybe Tech Com can help push this effort in support of recent strategic decisions: https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Medium-term_plan_2019/Platform_evolution#Metrics. There are some fundamental pieces missing from our platform that cause Graph/Graphoid to have an overly complicated architecture, I'll try to remember to bring this up at our next go-around.

The extension requires ploticus which the package has been removed from debian 10/buster.

@Paladox: Might be worth a separate heads-up task under EasyTimeline and Packaging with low priority, explicitly mentioning Debian 10 in the summary.

In T137291#5471571, @Paladox wrote:

The extension requires ploticus which the package has been removed from debian 10/buster.

Split into T253377: WMF deployed EasyTimeline extension depends on Ploticus package which is not available in Debian Buster (but available again in Debian Bullseye)

Recently Jules* was interested to convert EasyTimeline timelines to Graph and was asking for some examples: see this discussion on French Wikipedia’s Village Pump. I provided the example on MW.org (which is also the only official example of a timeline), but he judges that the syntax is “too low-level for his competences”, and I aggree with him that the syntax is quite complicated for a timeline. Probably Vega’s syntax is more powerful, but EasyTimeline’s syntax is largely used and documented for the use case of a timeline.

If the timelines are to be converted to Graph/Vega, I find the current EasyTimeline’s syntax should be kept and automatically translated to Vega – I’m aware it’s not a small project. Also, there would be bonus points if the old syntax would be still recognised after a transition to be able to still view historical versions of the timelines.

@Seb35: apologies for the drive-by comment but it should actually be quite easy to write either a static conversion script or a dynamic Lua script:

EasyTimeline syntax is a subset of Vega's, except for a few visual details. So converting one to another is just a matter of reading EasyTimeline, parsing out the important data, and sticking it into a Vega template ala the example you provided. It could even be done as a plugin of the graph extension, but I'm not sure that's a great idea.

Somewhat big blocker for this is that after some work Wikimedia Foundation engineers have done on Graph extension, it does not have a non-JS fallback now, making EasyTimeline superior in that regard. MediaWiki page says that 5% of edits are done without JS enabled.

In T137291#9708587, @Bawolff wrote:

Clearly this is no longer going to happen.

Though EasyTimeline may be migrated to the future new graph extension.

In T137291#9708917, @Bugreporter wrote:

In T137291#9708587, @Bawolff wrote:

Clearly this is no longer going to happen.

Though EasyTimeline may be migrated to the future new graph extension.

Indeed. Resurrecting.

Maybe, but i would argue a new task would be better, when and if that happens.

Like we are talking about transitioning to something that doesn't exist yet. We dont know if the solution will be intended to replace easytimeline (posts on wiki have implied it wont be but its still up in the air), we dont know the issues. We dont know anything at this point, it doesnt make sense to have a task to do something that may or may not mske sense, just like it doesnt make sense to have a task to transition to php 9.

In T137291#9731083, @Bawolff wrote:

Maybe, but i would argue a new task would be better, when and if that happens.

Like we are talking about transitioning to something that doesn't exist yet.

The focus of the task is presumably more to migrate away from EasyTimeline than specifically to Graph. But it's a good point: given the Graph extension is effectively dead in its current form, how does the calculus for EasyTimeline look now? And if the conclusion is still that there's a need to migrate away from it, what is the target of that migration?

If the Graph-replacement is to be the target of that migration this needs to go into the requirements for that effort ASAP.

@MMiller_WMF and @CCiufo-WMF is functionality sufficient to replace EasyTimeline usage currently within scope for T334940?

Whether it makes sense to shoehorn timelines into whatever replaces Graph depends entirely on the approach taken. Graph was a good target because it was so incredibly broad and versatile. As I understand the current plan the replacement is likely to be a lot more targeted and narrow, which maybe means it will not be.

But if we don't find a replacement my assessment is that we will eventually end up in the same situation with timelines as we currently are with graphs.