Page MenuHomePhabricator

Create Kartographer Lua extension library
Open, LowestPublic

Description

Maps are called usually by the <maplink> and <mapframe> extension tags. In practice these tags are not used directly but are called in Lua scripts as it is done for instance by the module https://en.wikivoyage.org/wiki/Module:Map. Unfortunately the expensive frame:extensionTag(tag, geojson, tagArgs) has to be used to convert the data into an extension tag.

As noted in T189409 calling <maplink> via Lua script can take several hundreds of ms. For example, in https://de.wikivoyage.org/wiki/Halle_%28Saale%29 frame:extensionTag is called about 240 times which takes about 1500 ms.

Therefore, I propose a Scribunto extension library at least with the following functions:

  • mw.kartographer.maplink( args, geoJson )
  • mw.kartographer.mapframe( args, geoJson )

args is an array containing the tag parameters -- same as for the extension tags --, and geoJson is either a JSON array or a JSON-encoded string noted between start and end tag.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 15 2018, 11:34 AM

Example profile data given by @Anomie:

Scribunto_LuaSandboxCallback::callParserFunction		1560 ms	45.1%
Scribunto_LuaSandboxCallback::getEntity		580 ms	16.8%
Scribunto_LuaSandboxCallback::gsub		320 ms	9.2%
Scribunto_LuaSandboxCallback::addStatementUsage		300 ms	8.7%
?		100 ms	2.9%
Scribunto_LuaSandboxCallback::preprocess		100 ms	2.9%
recursiveClone	<mwInit.lua:41>	60 ms	1.7%
Scribunto_LuaSandboxCallback::getExpandedArgument		60 ms	1.7%
Scribunto_LuaSandboxCallback::getFileInfo		40 ms	1.2%
Scribunto_LuaSandboxCallback::fullUrl		40 ms	1.2%
[others]		300 ms	8.7%
Mholloway triaged this task as Lowest priority.Aug 21 2018, 3:21 PM
Mholloway edited projects, added Maps (Kartographer), patch-welcome; removed Maps.
MSantos moved this task from Unsorted to General on the Maps (Kartographer) board.Sep 11 2018, 2:17 PM
TheDJ added a subscriber: TheDJ.Apr 9 2020, 12:34 PM

@Anomie question, why is frame:: extensionTag expensive and why would a direct call into kartographer be less expensive ?

Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptApr 9 2020, 12:34 PM
Anomie added a comment.Apr 9 2020, 1:03 PM

@Anomie question, why is frame:: extensionTag expensive

It's not expensive in the sense of counting towards the "expensive parser function count" limit.

It may be "expensive" in that it goes through Parser::callParserFunction() instead of doing something more direct. But whether doing something more direct would actually be appreciably faster I don't know. The time attributed to "Scribunto_LuaSandboxCallback::callParserFunction" in the trace in T189769#4052824 doesn't (and can't) break down what part of that time is spent in callParserFunction() versus in Kartographer itself.

TheDJ added a comment.Apr 9 2020, 1:58 PM

Yeah as I suspected. I doubt going directly in the tag logic of Kartographer is actually that much faster. Both are bridges into a tagfunction, one just specifies a specific tag function.

So I suspect the most expensive parts are:
1: json parsing of the kartographer features
2: storing json of the kartographer features
3: wikitext parsing of the properties of features
4: wikitext parsing of the maplink contents/mapframe 'text'

I guess we can specify that the content passed needs to be already preprocessed if you want to use wikitext, thereby bypassing the internal parser run most of the time...?

Base added a subscriber: Base.Apr 9 2020, 2:33 PM