In T119687#1912243, @liangent wrote:

In T119687#1911453, @zhaofengli wrote:

FYI, the hardcoded conversion table is includes/ZhConversion.php which may help determine the language variant of arbitrary text.

There is a function LanguageConverter::guessVariant().

So, looking at https://gerrit.wikimedia.org/r/328887 it seems that, if that patch is merged, the primary difference between <poem> and <pre> handling in core is:

• <poem> generates an empty span with margin-left:1em for wikitext like : foo. But I think we could probably tweak the CSS so that the rendering of the standard parser output for :foo (aka, <dl><dd>foo</dd></dl> is the same.
• <poem> respects leading whitespace, but compresses intraline whitespace. This is currently being done by a combination of CSS and manually substituting leading whitespace with &nbsp;. In order to clean this up, the easiest thing would be to stop compressing intraline whitespace. We'd either do that for a new <lines> tag (and merge the <lines> semantics into core) or else do a wikilint type project to simply removing inadvertent intraline whitespace. Depending on how prevalent it is, the latter might not be too hard.

cf https://gerrit.wikimedia.org/r/#/c/411293/11/lib/wt2html/TokenTransformManager.js@939 for example, which has more codepaths than it ought to because of this ambiguity in return types.

<gallery caption="Foo&#10;&#10;bar"> ... </gallery> might be an interesting test case.

I've got no problem with skipping passes, that seems like a fairly coarse-grained tool, but it does potentially open up a new way for preview bugs to creep in. It would be interesting to explore the idea enough to determine how much time we are talking about here, though.

This zombie bug has lurched to life again because of T188478: Parsoid needs a native implementation of <poem>.

@Tiven2240 is there an answer to @Aklapper's question? That seems to me to be what's going on here, as well, but maybe I'm misunderstanding.

...or just "attribute string" (output is html-escaped to be safe to include in attributes, no bare quotes allowed) or "attribute list" (validated to be valid attribute name=value pairs). I'm not convinced we should necessarily be validating number formats, css rules, etc. (Inspired by the set of template types available in spacebars, which impressed me as a good minimal-but-safe-and-structured template engine).

The problem seems to arise when (at least) one of the characters is written as an HTML entity, eg:

&#nnnn;o

Parsoid will wrap a <span> tag around the entity in order to support roundtripping, and that span tag will "separate" the combining character from the character it should apply to, in (broken versions of) webkit.

The outer HTML for the "Girl Drop" section in https://zh.wikipedia.org/api/rest_v1/page/html/POP_TEAM_EPIC is

<span style="font-weight: normal;" about="#mwt31">（<span lang="ja"><span typeof="mw:LanguageVariant" data-mw-variant="{&quot;disabled&quot;:{&quot;t&quot;:&quot;☆色ガールドロップ&quot;}}"></span></span>）</span>

The blank section in the parens is the language variant markup. Your expandtemplates request is executing on en.wikipedia.org, instead of zh.wikipedia.org, so you're expanding the wrong template. This one contains the language converter markup:
https://zh.wikipedia.org/w/api.php?action=expandtemplates&text={{nihongo|%E3%80%8A%E6%98%9F%E8%89%B2Girl%20Drop%E3%80%8B%7C%E2%98%86%E8%89%B2%E3%82%AC%E3%83%BC%E3%83%AB%E3%83%89%E3%83%AD%E3%83%83%E3%83%97}}

Quick cross-reference: the variant parameter is only documented here in phab: T44356#439479 and T34906#381101
The patch in gerrit seems to be stuck based on resolution of caching issues. Note that caching is apparently currently broken in some cases if you use this API parameter as a logged-in user, so beware.

It probably is best to strip these as a post-processing pass on the DOM (as in my comment above), but we should think through some issues of tokenization, etc, and verify consistency with PHP etc etc. But we stripped NULLs from wikitext (T159174), we should be able to strip other characters as well.

We're going to make this a child task of {{#balance}} for now, because when we add a warning people are going to want to know what to do to suppress the warning -- and the thing we're going to want people to do is opt in to balancing (to get their desired start-of-line context).

We resolved this as some point.

All the skeleton code for this has been completed in Parsoid: we honor the Accept-Language header and dispatch to an appropriate language variant converter.

Also possible that someone is actually trying to do the sort of XSS attack that bawolff was protecting against? Deliberately creating a very long strip marker?

A stack trace would also be helpful. Or at least the value of $type or $text in the unstripType caller.

cc'ing brian wolff, since he wrote 939faea318d9c2107fab3a584bc1c023f3c592e9

First impression: somebody is adding very large strip markers, which are tripping the memory limits when unstripped to add to the output.

jsduck is dead, although we haven't fully reproduced the old jsduck organization of the documentation yet.

To the "requirements" list I'd add "ability to show/hide method & member summaries" as well, since it's hard to get a good idea of what a given class does if you have to scroll through pages and pages of docs on every single method.

My theme broke today because I was hotlinking to the WikimediaUI-Style-Guide github repo. But it should be fixed as soon as https://gerrit.wikimedia.org/r/410208 is merged.

I've started on this! https://github.com/cscott/jsdoc-wmf-theme

T175421: Parsoid doesn't insert same spacers when article text starts with two or more newlines. may be related (at least in the sense that it could be fixed at the same time, if not actually due to the same cause).

My jsdoc-wmf-theme does manage to resolve a number of these @link shortcuts.

That stack trace looks much too short for a "maximum call stack size exceeded" exception. What's shown much be just the random tip of the tree. where's the recursive call stack underneath it?

https://github.com/jsdoc3/jsdoc/issues/483 is a problem with jsdoc -- it currently doesn't resolve {@link Foo} or {@link #bar} -- you have to spell out the full name of the class, including module name and full class name, every time if you want it to be linked. It may be possible to resolve this in the jsdoc 'theme'.

In T138401#3896189, @Prtksxna wrote:

You can see the OOUI documentation in the default theme on https://prtksxna.github.io/oojs-ui/ . For OOUI we'll definitely need our own theme, I was thinking we could do something similar to the style guide (something @Volker_E and Design has been polishing). That way all our documentation could have a similar design. Creating custom layouts for JSDoc seems pretty straightforward too: https://rebel-sisters.surge.sh/ (WIP). Thanks for introducing me to surge.sh @Jhernandez

ParsoidConfig.js, WikiConfig.js: https://gerrit.wikimedia.org/r/408375

lib/utils/Util.js lib/utils/DOMUtils.js lib/wt2html/parser.defines.js : https://gerrit.wikimedia.org/r/408826

Argument name space-padding is (apparently) considered a discouraged practice on dewiki, if I understand correctly. When a new longer argument name is added, all of the existing arguments need to get spaces added, and that (human-created) dirty diff is frowned upon.

Patch at https://gerrit.wikimedia.org/r/408593 -- assuming that the underlying failure in setPageSrc would be properly handled by the caller.

Yeah, probably -- the code used to be Promise#tap, so it would still have thrown an exception, but that exception would have been ignored.

Erk, is this my fault w/ 1f89e97a25b5413d90ad36591d0e5766883c5b25 ?

One more: TokenTransformManager and friends: https://gerrit.wikimedia.org/r/408342

In T138401#3913720, @Jdforrester-WMF wrote:

In T138401#3911653, @Jdlrobson wrote:

but this is almost good enough to migrate to right now (and re-skin in time).

Could we articulate what needs fixing? @Prtksxna would be willing to submit a patch of what you've done so far? (No matter if it's a little raw :))

The @event naming that I mentioned and the @property documentation that Timo mentioned are I think it.

Next: Logger-related classes: https://gerrit.wikimedia.org/r/407868

@Alsee in this case, if you had entries which were whitespace aligned, whichever editor put 'format:block' in the templatedata was at fault. Looking at the edit you cited, it appears as if the format should have been:

{{_\n| _____________ = _\n}}\n

In T179259#3942459, @Alsee wrote:

The parameter reordering issue was addressed (yeay), but the original task here was to stop stripping whitespace. Here's a test edit showing that the problem still exists.

Apparently not fixed yet, according to T186281: titles with trailing question marks aren't properly URL-encoded.

Yes, I think so! Darn it, should have fixed this bug the first time I found it.

@JMinor: +1 on dogfooding our own software, instead of using google docs and etherpads!

More followup: T186272: Convert all parser tests to html5 ids

Probably the first step is to flip the DefaultSettings.php $wgFragmentMode from ['legacy','html5'] to ['html5','legacy'].

If this is implemented in Parsoid, the Flow code added for T68289: A16. Flow: external links should have rel="nofollow" property can/should be removed.

Repeating a comment from T58756:

I'd *strongly* prefer to see this added as a post-processing pass, instead of directly to the wt2html pass. We should maintain a good discipline about separating out redundant information which is only used for read views or styling, so that (in the future) this can be generated on-demand and we don't waste storage or transmission bandwidth on unnecessary bits.

Repeating a comment from the patch:

I'd *strongly* prefer to see this added as a post-processing pass, instead of directly to the wt2html pass. We should maintain a good discipline about separating out redundant information which is only used for read views or styling, so that (in the future) this can be generated on-demand and we don't waste storage or transmission bandwidth on unnecessary bits.

Mumble mumble OCG mumble mumble.

This would be a good use of a general "fork storage" mechanism: the edit-in-progress would get stored as an unmerged fork of the main revision.

@tstarling and I are interested in the implementation of this. It would be a good use of the "derived" slots in multicontent revisions. You'd checkpoint the authorship map every N revisions, and store the most recent authorship non-peristently in memcached.

Don't forget that our use of the "lang" feature in SVGs differ from the w3c standard, as implemented by browsers. In addition to blacklisting large SVGs we need to blacklist SVGs using the language selector feature.

The parsing team has a proposal for "heredoc arguments" (T114432) which would allow the use of the {{cite}} template to easily enclose a region of text.

I was a little disappointed that the position papers weren't used more. I understand that the session leaders used the position papers to group sessions and seed the session with topics, but there was a still a lot of oral discussion that just restated points folks have previously made in their position papers.

The posters at the event called it a "development summit". FWIW. ;)

See also the UX mockup at https://phabricator.wikimedia.org/T146397#3241037

See also the UX mockup at https://phabricator.wikimedia.org/T146397#3241037

I'd like to see next steps from the session leaders include some means of publicizing the sessions and result of the sessions to all the folks who couldn't be present, including a means to collect feedback from nonparticipants who may support/object/amend conclusions of the session.

Can we get the slides from this keynote posted?

Some more notes from a twitter discussion started by @Sj, whose developer summit position statement is strongly related to this task.
@cscott says:

I think that's my cue to pitch T113004 which seeks to incorporate a more GitHub-like fork/merge model in Wikipedia. It would allow contributions to be visible, even when unmerged. (Which has its own issues, of course...)

@Phoebe says:

What is an unmerged contrib? Someone makes an edit, someone takes it out again? Or are you imagining that ppl don't contribute directly to article, they propose a change a la GitHub and others can see that change? Wouldn't that take a huge amount of personhours to approve pulls?
I'm sure ideas about this are in the proposal somewhere but as you know the vast amount of edits are added w/o controversy, so keeping workloads the same or lower would be A+
Ps I'm reading all the stuff about enhancing diffs & edit review screens - that's fantastic. Though I'll keep my "diff to penultimate revision" t-shirt forever, diffing could really use some love <3

@cscott says:

Once the basic infrastructure for forked revisions was in place, you could experiment with lots of merge strategies. Automatically merge after N hrs, say, or merge edits scored as not-vandalism...
I think the least disruptive thing would be to automatically merge, but keep the fork when there is a roll back or merge conflict. You could choose to start a revision as a fork as well, like w/ the draft namespace.
The key psychological breakthrough would be to change first-time editors' experience from "I tried to help and it was reverted and thrown away" to "I tried to help and it wasn't merged yet but there were some helpful suggestions for improvements".

I'll add: the happiness kick you see when your constructive edit is made and immediately goes live on the site is even better! And should definitely be preserved whenever possible. But fork/merge can provide gentler off-ramps other than revert in case of issues with an edit.

Hm, not only does it have a hard dependency on bluebird, it uses bluebird's non-standard methods, so it can't be easily ported to actual standard ES6 promises. :(

Does preq let you use native Promises? We don't use bluebird.

First up: lib/api/ParsoidService.js: https://gerrit.wikimedia.org/r/402435

Recapping some discussion in IRC:

Context: https://meta.wikimedia.org/wiki/Help:Interwiki_linking#Interlanguage_links

For projects like Meta, a missing leading colon has no effect, because Meta doesn't support interlanguage links. For Wikipedia and similar projects, it is a major difference.

In T102556#3841701, @Whatamidoing-WMF wrote:

IMO the examples you give would probably be okay (especially the RFC one), but if editors see [[google:this]] or [[flickr:that]], then they will be surprised and angry.

See https://meta.wikimedia.org/wiki/Talk:Interwiki_map/Archives/2009-04#Google and https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)/Archive_46#Jumbled_up_mess_of_link_types and https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)/Archive_AF#[[MemoryAlpha:articlename]]_links:_how,_when,_why? for some previous discussions on this point.

In T152540#3833321, @Jack_who_built_the_house wrote:

See https://bugs.chromium.org/p/chromium/issues/detail?id=789163. The change in Google Chrome behaviour is possible.

I wonder if the technology here (invisible <span> tags) could also help with T160952: Generate Language-converted section anchors/ids? Strawman: $wgFragmentMode could include one or more language variant codes along with the special 'html5' and 'legacy' values; similarly something similar to $wgExternalInterwikiFragmentMode (except for internal wikilinks) could select whether the "canonical" ID is used for a wikilink (matching source text) or a language-converted variant is preferred?