Page MenuHomePhabricator

Transition all use of EasyTimeline to the Graph extension and decommission it from Wikimedia's servers
Open, LowestPublic

Description

Per @Yurik, the features of EasyTimeline can be replicated with Graphs, possibly via templates to make it easier.

A possible plan:

  • Write a help page with all the known usages and instructions on how to convert them
  • By consulting the help page, determine whether we can replace all the features we use/want, and decide to make the change
  • Announce the need to migrate and overall plan
  • Provide a tool to find all current content to migrate (T137310: Provide tracking category or page property for <timeline> uses?)
  • Provide a tool/guide to migrate current content over
  • Warn people and wait (six months? a year?)
  • Disable EasyTimeline

Event Timeline

Restricted Application added subscribers: Zppix, Matanya, Aklapper. · View Herald TranscriptJun 8 2016, 11:48 AM
fbstj awarded a token.Jun 8 2016, 12:07 PM

I don't think it would be responsible to destroy the timelines. Either it's feasible, for whoever desires disabling EasyTimeline, to convert all its usages on Wikimedia wikis, or the disabling shouldn't happen.

Danny_B updated the task description. (Show Details)Jun 8 2016, 2:17 PM
Danny_B added a subscriber: Danny_B.

Assuming enwiki has the largest number of usages, then we can say, that ~ 10,500 (rough search on enwiki) is the maximum per wiki. The average and median will be much lower though...

Replacing single-purpose tool with multipurpose one is always good idea, especially when the former has its own syntax which does not match anything commonly used.

Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Jul 20 2016, 9:11 PM

First question: Is there a migration document to follow?

Nemo_bis updated the task description. (Show Details)Jul 27 2016, 8:18 PM

Good point Andre, I added a subpoint which was implicit in the former first point.

Replacing single-purpose tool with multipurpose one is always good idea

I don't think that's necessarily a true statement...

For the actual bug, anecdotally I think a lot of users find the timeline syntax easier to understand than the graph syntax (Both are pretty hard though).

@Bawolff I would even say that general purpose language tends to be more complicated that the domain-specific language. That said, I think we should utilize Lua and templates to solve this - in other words, create a domain language, specifically for timelines, and have Lua transform it into a generic Vega syntax. This way the language can be developed on-wiki, by the entire community, without the need of the lengthy js/php modification.

How closely can the EasyTimeline syntax be matched by Graph syntax? Is it possible to automatically migrate them? If yes (and I hope it's "yes"; or maybe "yes in 90% of cases, except where the user did something very weird"), can we replace EasyTimeline with a small extension that depends on Graph, has the same syntax as EasyTimeline (or a wide subset of it), and converts on the fly to Graph syntax, rather go through the grueling process of updating all the wikitext pages?

@matmarex, i am afraid it's not. EasyTimeline's syntax is fairly complex and non-standard (a cross-breed of config files, yaml, and Basic as far as i can tell). Even the simple parsing is a challenge for Lua code, but I'm sure it can be done (or we could reuse EasyTimeline's own parser if it exists outside of the rendering). Vega grammar is not context-aware - its basically a set of D3-like operations, like take this data, transform it with these expressions, scale it this way, and draw it with these visual primitives. It is much more generic, powerful, and hard to work with.
Also, EasyTimeline inserts clickable links - something that Vega can only do in the "interactive" mode (after you click the image to activate it, and it loads all the javascript and required data, and redraws it on the client). I had a discussion about generating clickable SVGs, but that might be some time away. Alternatively the EasyTimeline wrapper could generate some image map as part of the parsing.
Yet,I'm not sure the server-side wrapper is a good idea - what's the point of replacing custom engine with Vega engine if the result is identical, and all existing pages will continue to have existing syntax? We might as well leave both in place. Instead, I really think we should have a lua script that can generate the required vega graph, and a bot that will go throw the existing graphs and replace them with that script call. This way the community can continue to improve the Lua module after the conversion script is done.

what's the point of replacing custom engine with Vega engine if the result is identical, and all existing pages will continue to have existing syntax?

Huge reduction in tech debt and huge improvement in maintainability. Most of EasyTimeline is written in densely packed Perl with Perl dependencies. Even if the new code would be just as tough, it would at least be in PHP or JavaScript, which are familiar to us.

The generated images would probably be a huge improvement too. EasyTimeline can't produce SVG, and it can't produce 2x PNG, and it can't do anything else to make them prettier on high DPI screens. And the normal DPI output also looks quite ugly (it seems to have some super broken font antialiasing, especially on links, and the kerning is worse than if there were none at all).

Yurik added a comment.Jul 28 2016, 2:33 AM

@matmarex, I agree about the output, but I'm more concerned with the wiki markup. Implementing and maintaining a very complex parser for the original syntax could be about as much work as implementing a conversion script into something more "wiki-like", e.g. a template call with a number of predefined parameters. And once done, we won't ever have to deal with it again. Instead we will have a set of well known templates (one for each type of the timeline), with a well defined template parameters (allowing much better VE integration with parameter entry). Continuing support for the <timeline> tag with all of its custom markup might be much more resource demanding, especially in the longer run.

hashar added a subscriber: hashar.Nov 14 2016, 11:21 PM

Can we get an example of using Graph to generate at least a basic timeline? That might help starting the manual conversion process :]

Yurik added a comment.Nov 15 2016, 2:40 AM

@hashar, sure, could you give me an example of a "basic" timeline? :)

EasyTimeline inserts clickable links - something that Vega can only do in the "interactive" mode

This is a feature used in some highly visible timelines, BTW.

@hashar, sure, could you give me an example of a "basic" timeline? :)

https://www.mediawiki.org/wiki/Extension:EasyTimeline#Code_example is rather simple: it has a single line, several events and a date range. My idea is to eventually point EasyTimeline to an example that looks familiar to them so they can migrate from it :]

Yurik added a comment.Nov 15 2016, 4:14 PM

EasyTimeline inserts clickable links - something that Vega can only do in the "interactive" mode

This is a feature used in some highly visible timelines, BTW.

Upstream feature request: Generate SVGs with links

https://www.mediawiki.org/wiki/Extension:EasyTimeline#Code_example is rather simple: it has a single line, several events and a date range. My idea is to eventually point EasyTimeline to an example that looks familiar to them so they can migrate from it :]

https://www.mediawiki.org/wiki/Extension:Graph/Demo#Timeline_.2F_lifeline comes probably closest here?

(Broader picture: https://en.wikipedia.org/wiki/Wikipedia:Graphs_and_charts#Timeline and https://en.wikipedia.org/wiki/Wikipedia:Timeline )

In T115188#4196921, @Amire80 wrote:

EasyTimeline relies on Ploticus, an upstream graphics library. It was possbly good for the task in mid-2000s when EasyTimeline was developed, but by current standards it is generally buggy, has poor graphics quality and poor Unicode support, and most importantly—it hasn't been maintained since 2012.
The most realistic thing to do about EasyTimeline is to stop using it for creating new timelines, and to migrate to the Graph extension. Yes, it will require a lot of work, but I cannot think of anything better. If anybody can develop an automatic converter from EasyTimeline's unique syntax to the Graph extension's syntax, it will be pretty awesome, but it's definitely not my expertise. (An opportunity for a grant-funded project, I'd say.)

Xover added a subscriber: Xover.Sep 12 2018, 9:16 AM
Lofhi added a subscriber: Lofhi.Jun 15 2019, 5:42 PM

Are the communities aware of this ticket? On frwiki, the extension is massively used and I believe that no one has planned to replace them with the Graph extension. Is there a documentation to organize this transition?

Are the communities aware of this ticket?

Probably not because nobody has worked on this. See the unchecked checkboxes in the task description above.

Is there a documentation to organize this transition?

No. Please see the unchecked checkboxes in the task description above.

Lofhi added a comment.Jun 15 2019, 6:01 PM

I saw the unchecked checkboxes above, but the ticket was opened in 2016 and I was not a contributor at that time. It may not be up to date. All the details are good to start making the necessary changes on frwiki. Thank you for confirming.

Here are search pages from each of the big ten Wikipedias on usage:

wikiUsage
enwiki~8,500
frwiki~7,000
zhwiki~1,500
dewiki~6,500
jawiki~1,000
ruwiki~12,500
eswiki~2,500
itwiki~38,000
ukwiki~45,000
ptwiki~1,000

There are just two uses on Commons, and only fifty on Wikidata.

Obviously our Cyrillic-speaking friends have used this a lot more than everyone else, and will need special efforts.

There is also Category:Pages using Timeline on many wikis if someone wants to look at existing use cases.

stjn added a subscriber: stjn.Tue, Jun 18, 4:39 PM

Obviously our Cyrillic-speaking friends have used this a lot more than everyone else, and will need special efforts.

FYI: many usages in Russian Wikipedia are bot/batch-created articles with pretty much the same code, so it’s easy to fix once you get community consensus. I can try to do some cleanup in this department if it would be helpful.

For ‘known usages’: {{Graph:Chart}} really needs ability to do horizontal bar charts and maybe some ability to change the angle for values.

Note that while EasyTimeline is limited and feels outdated, it has a light footprint and maintenance burden and has (almost?) never been the subject of production errors or issues with security, performance, caching, etc. Its code is a single Perl file that has needed little to no changes in over a decade.

The Graph extension on the other hand, has been subject of many such issues, and involves a substantial amount of PHP code, front-end JS, and the Graphoid Node.js service.

This by itself isn't an problem per se, but given that currently neither has a steward (listed as Editing, but it's been clarified recently that this is intended to apply to the VE plugin only). See also T211881.

It hurts to even say this, but unless and until either of these is resourced by Product, the lesser of these two evils is clearly EasyTimeline (as measured by shared burden for Tech and Product teams, development velocity, and infra cost).

I think I would be ideal if Graph and Graphoid become stewarded, and then we can go all-in on this migration. But lacking that, should we promote this migration, knowing it will demand more unallocated time to maintain, and make sunsetting even harder?

@Krinkle, I agree, we should make Graph/Graphoid maintainable before migrating anything to it. At the very least we would need to support Vega 4 and Vega Lite syntax, and address the problems of storing graph definitions and graph data. But maybe Tech Com can help push this effort in support of recent strategic decisions: https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Medium-term_plan_2019/Platform_evolution#Metrics. There are some fundamental pieces missing from our platform that cause Graph/Graphoid to have an overly complicated architecture, I'll try to remember to bring this up at our next go-around.