Page MenuHomePhabricator

Add "wikitext" as language in Pygments
Closed, ResolvedPublic

Assigned To
Authored By
bzimport
Mar 2 2011, 8:32 PM
Referenced Files
F7693: mediawiki.php
Nov 21 2014, 11:31 PM
Tokens
"Cookie" token, awarded by Xaosflux."Love" token, awarded by Quiddity."Love" token, awarded by Iniquity."Like" token, awarded by Naseweis520."Like" token, awarded by Ailbeve."Like" token, awarded by SerDIDG."Like" token, awarded by Kghbln."Like" token, awarded by IKhitron."Like" token, awarded by Miguu."Like" token, awarded by He7d3r."Like" token, awarded by Fae.

Description

Add "wikitext" as a language lexer in Pygments, as used in Extension:SyntaxHighlight

See attachment below for (incomplete) lang='mediawiki" patch.


Upstream Pygments feature request: https://github.com/pygments/pygments/issues/827

Older notes at the GeSHi feature requests: http://sourceforge.net/p/geshi/feature-requests/112/ ("wikitext") and https://sourceforge.net/p/geshi/feature-requests/84/ (MediaWiki). (but nothing in GeSHi's newer GitHub issues).

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
This comment was removed by Spage.
Fae added a subscriber: Fae.
Quiddity renamed this task from Add "wikitext" as language to Add "wikitext" as language in Pygments.Oct 19 2018, 4:35 PM
Quiddity updated the task description. (Show Details)
Quiddity removed a subscriber: Spage.
Quiddity added subscribers: Seb35, Quiddity.

@Seb35 thank you for your note on the upstream issue. Is there anything specific you need help on, to move this forward?

What was blocking this? Was there an upstream issue?

@ShakespeareFan00 Please read the task description first which does include an upstream issue link. Thanks.

Has this really been stalled with an upstream provider for 10 years? Can we just fork this?

Has this really been stalled with an upstream provider for 10 years? Can we just fork this?

I don't think a fork is necessary or helps us. Also this task was previously against GeSHi.

Someone needs to write the file and submit it upstream (pygments) for review, inclusion etc. AFAIK no one has attempted that just yet.

The upstream task at https://github.com/pygments/pygments/issues/827 has only existed since August 2019.

Thanks, must have misread the notes above, thought the patch was already submitted to them.

Diskdance added a subscriber: Diskdance.

I have been working on this for quite a while. Now I am going though a bunch of tests, hopefully a PR would be filed soon.

Upstream PR is merged, now waiting for a Pygments release so we can bundle it in MediaWiki.

Do we know what lang="" this will be yet? lang="mediawiki"? lang="wikitext"? etc.

Do we know what lang="" this will be yet? lang="mediawiki"? lang="wikitext"? etc.

'WikitextLexer': ('pygments.lexers.markup', 'Wikitext', ('wikitext', 'mediawiki'), (), ('text/x-wiki',)),

Looks like both should work

Great, thanks. One more question. Will this ride the train, or do we need to take some kind of manual action to deploy it, such as merging a gerrit patch to bump the pygments verison in some file somewhere? https://codesearch-beta.wmcloud.org/search/?q=pygments&files=&excludeFiles=&repos=

Do we have to wait for an upstream "release", or are we using the upstream master branch?

Great, thanks. One more question. Will this ride the train, or do we need to take some kind of manual action to deploy it, such as merging a gerrit patch to bump the pygments verison in some file somewhere? https://codesearch-beta.wmcloud.org/search/?q=pygments&files=&excludeFiles=&repos=

Do we have to wait for an upstream "release", or are we using the upstream master branch?

It will need work tracked by T298399: New upstream release for Pygments as noted in T29828#8759289.

The current release of Pygments is 2.14.0 which was released on 2023-01-01. The new lexer was merged on 2023-04-05 so it obviously is not in the current release. It should be in the planned 2.15.0 release.