Page MenuHomePhabricator

cscott (C. Scott Ananian)
Parser whisperer

Projects (15)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Oct 21 2014, 6:47 PM (439 w, 12 h)
Availability
Available
IRC Nick
cscott
LDAP User
Unknown
MediaWiki User
Cscott [ Global Accounts ]

Editor since 2005; WMF developer since 2013. I work on Parsoid and OCG, and dabble with VE, real-time collaboration, and OOjs.

On github: https://github.com/cscott

See https://en.wikipedia.org/wiki/User:cscott for more.

Recent Activity

Yesterday

cscott reopened T300325: RESTBase (?) and core REST API (??) produce different Parsoid HTML as "Open".

Thanks. So… should that use 'deduplicateStyles' => false, instead? Because Parsoid already deduplicates styles using a slightly different implementation?

Tue, Mar 21, 9:28 PM · MW-1.38-notes (1.38.0-wmf.20; 2022-01-31), MediaWiki-REST-API
cscott added a comment to T332243: NOTOC causing API action=parse prop=sections to return nothing.

I think this is a won't fix, although I'm open to hearing otherwise. The TOC can have no sections for a number of reasons, including that the author has explicitly requested it (as in this case), or any seen sections are bogus (ie, content model is javascript), etc. Assuming that a page will always have sections seems to be a misunderstanding, and from the above it seems there's already been a fix committed to the affected project.

Tue, Mar 21, 6:09 PM · Patch-For-Review, MediaWiki-Action-API, Ultraviolet, MediaWiki-Parser
cscott added a comment to T328034: ParsoidCachePrewarmJob should warm the useParsoid ParserOption.

Yes, that's what all the patches do: split the cache. 800769 splits the cache via the key (parser option) and 883501 forces a split via the parser cache name. The question is how to describe how/why the parser cache is being split, and to do it in a way that doesn't depend on "magical" properties of the code.

Tue, Mar 21, 5:25 PM · Patch-For-Review, Parsoid-Read-Views (Phase 2 - testwiki Main namespace support), Content-Transform-Team-WIP, MW-1.40-notes (1.40.0-wmf.21; 2023-01-30), Performance-Team (Radar), MediaWiki-Parser, RESTbase Sunsetting, Parsoid
cscott added a comment to T328034: ParsoidCachePrewarmJob should warm the useParsoid ParserOption.

I'm having a bit of trouble figuring out a coherent state for the useParsoid option. https://gerrit.wikimedia.org/r/c/mediawiki/core/+/800769 "uses" the useParsoid option on all paths through the parser, which is /safe/ (in terms of avoiding corruption of a single non-forked parser cache) but (as we discussed previously) means that after that patch were to land in production, any new parse of page [[X]] would invalidate all other parses for [[X]] in the parser cache -- ie, if you look at the parsoid version of X then the legacy versions of X get invalidated; if you do a new parse for a spanish-speaking user of [[en:X]] it has the side effect of invalidating the existing english-language parse of [[X]]. (Briefly, this is because the new parse will have 'useParsoid' in the 'used options' where 'useParsoid' is true or false, and this will cause existing entries with 'useParsoid' not in the used-options set to be invalidated.)

Tue, Mar 21, 4:26 PM · Patch-For-Review, Parsoid-Read-Views (Phase 2 - testwiki Main namespace support), Content-Transform-Team-WIP, MW-1.40-notes (1.40.0-wmf.21; 2023-01-30), Performance-Team (Radar), MediaWiki-Parser, RESTbase Sunsetting, Parsoid

Mon, Mar 20

cscott added a comment to T332457: Adopt a stable interface policy refactor.

Just as a quick add-on -- I think the real blocking child task here is to configure CI (phan?) to enforce some of these guidelines. If we want to have fine-grained stability markers ("some methods in Util are @stable and some are @internal") instead of the big-hammer "everything in this namespace is @stable") then we need some tooling support.

Mon, Mar 20, 3:22 PM · Parsoid
cscott added a comment to T332457: Adopt a stable interface policy refactor.

My interpretation would be that @internal is acceptable to use in core, but maybe we should be a little more precise, because it's not clear that we want to use *all* of this stuff indiscriminately in core -- at the very least we ought to mark the things used in core so that we don't 'accidentally' break core by changing them in Parsoid.

Mon, Mar 20, 3:20 PM · Parsoid

Thu, Mar 16

cscott added a comment to T19587: alias __TOC__ to {{#TOC}} and add TOC style options to such parser function.

I have to make a decision about default case for T204370, but we might want to make this {{#toc}} (aka lowercase) instead of {{#TOC}}. The recommendation from Unicode is to make the word case-sensitive, not case-insensitive, since the rules for case folding are hard to localize well.

Thu, Mar 16, 8:22 PM · Patch-For-Review, MediaWiki-Parser
cscott added a comment to T332242: Wikimedia\Parsoid\Utils\TitleException: Invalid characters in title "Special:ParserFunction/REDIRECT [[]]".

This parse contains the literal wikitext:

{{#REDIRECT [[]]}}

for some reason, in the place where a signature would generally appear. The edit was done by an IP user in March 2014:
https://en.wikipedia.org/w/index.php?title=Talk%3AMehath&diff=598632387&oldid=579155148&diffmode=source
using the wikitext source editor, apparently (ie, it is not tagged with Visual Editor or any other editor tag).

Thu, Mar 16, 2:34 PM · Parsoid, Wikimedia-production-error
cscott updated subscribers of T329302: MediaWiki\Revision\RevisionAccessException: Not an available content version..

@daniel's team should probably be tagged here (I don't know the exact tag to use) as well as DiscussionTools -- seems related to the 'direct parsoid access' patches, but fundamentally seems to be about trying to invoke discussion tools on something which is not actually a talk page?

Thu, Mar 16, 2:27 PM · Parsoid, Wikimedia-production-error
cscott closed T331084: TypeError: Argument 2 passed to Wikimedia\Parsoid\Config\StubMetadataCollector::collect() must be of the type string, int given, as Resolved.

Should be fixed.

Thu, Mar 16, 2:24 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Parsoid, Wikimedia-production-error

Mon, Mar 13

cscott created T331887: Add ::isSameCodeAs() method to wikimedia/bcp47code.
Mon, Mar 13, 3:44 PM · Patch-For-Review, Bcp47Code
cscott added a comment to T331871: CI Parser tests fail on integration/quibble.

Ah, the 891358 patch should have been listed as depending on https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/888116 -- I'd never tested them in isolation and didn't expect one to be merged without the other. We probably need to tag a new parsoid with the miscellaneous test runner fixes in 888116 as well.

Mon, Mar 13, 1:16 PM · ci-test-error, Quibble, Parsoid
cscott added a comment to T331871: CI Parser tests fail on integration/quibble.

Probably caused by the merge of https://gerrit.wikimedia.org/r/c/mediawiki/core/+/891358 at 03:40 Mar 13.

Mon, Mar 13, 1:04 PM · ci-test-error, Quibble, Parsoid

Fri, Mar 10

cscott added a comment to T287419: `mediawiki-core-php72-phan-docker` job runs `composer install` instead of using packages from mediawiki/vendor.

I'm just going to mention composer update here so it show up in my search (since I always forget) and mention https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/612431 as a patch which is blocked on the resolution of this bug.

Fri, Mar 10, 3:41 PM · Parsoid (Tracking), Release-Engineering-Team (Radar), Continuous-Integration-Config, ci-test-error (WMF-deployed Build Failure), MediaWiki-Vendor
cscott created T331727: Deprecate ParserOutput::getCategories() for external use.
Fri, Mar 10, 2:53 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Technical-Debt, MediaWiki-Parser

Thu, Mar 9

cscott added a comment to T104770: Add HTML5 <aside> to the parser whitelist.

This bug should be retitled if the actual problem to be solved is "hide template content from search engine snippets".

Thu, Mar 9, 6:20 PM · MediaWiki-Parser
cscott created T331655: Support `outputHasCoreMwDomSpecMarkup='mixed'` in Parsoid extension registration.
Thu, Mar 9, 6:02 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Parsoid
cscott added a comment to T318433: Templates (and extensions) that mimic parser media output need migration to new structure.

Tangentially (or maybe not), editors should have access to <figure> and <figcaption> in wikitext per my comment in T25932#7070297.

That's a separate debate, one on which I think there is a wide gap in consensus.

Without figure/figcaption literals though, templates won't actually be able to mimic the parser output for the block form. We're back to div soup,

<div class="enwiki-figure">
  <a href=""><img src="" /></a>
  <div class="enwiki-figcaption"></div>
</div>
Thu, Mar 9, 5:44 PM · MW-1.40-notes, MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), Content-Transform-Team-WIP, Patch-For-Review, Parsoid-Read-Views (Phase 0 - Parsoid-Media-Structure), Parsoid
cscott added a comment to T25932: Allow use of semantic HTML5 elements in wikitext.

Briefly: "takes a load off /you/" -- I am a big fan of enabling the community to solve problems the WMF can't resource. But I think we should also acknowledge that a large part of /our/ burden (and why it takes so long to do things on wiki in general) is that we have a huge amount of legacy content to port forward with every change, and by Hyrum's Law, every feature and corner case and bug in our implementation *will* get used by the community and *will* cause us additional effort/porting time next time we need to change or fix that.

Thu, Mar 9, 5:01 PM · HTML5, Epic, Accessibility, MediaWiki-Parser
cscott claimed T331084: TypeError: Argument 2 passed to Wikimedia\Parsoid\Config\StubMetadataCollector::collect() must be of the type string, int given,.

I'm looking at this.

Thu, Mar 9, 3:38 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Parsoid, Wikimedia-production-error
cscott added a comment to T330949: Simple CSS for ebooks on Wikisource (ePub, pdf).

Not sure why Parsoid is tagged here? @Arlolra might be able to offer some insight w/r/t the media styles; it's possible you can use a simplified version of the stylesheet if you are using *only* the "new style media" output; I think we currently have rules for both legacy media output and new media output enabled because there are transclusion scenarios which might cause 'legacy' markup to appear on 'new' pages.

Thu, Mar 9, 3:37 PM · Community-Tech, WS Export, All-and-every-Wikisource, HTML5, Parsoid
cscott moved T331629: HTTP 412 Errors when editing Officewiki from Needs Triage to Bugs & Crashers on the Parsoid board.
Thu, Mar 9, 3:37 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), MediaWiki-REST-API, serviceops, Parsoid
cscott moved T331148: Unify the read-views and REST API caches for Parsoid content from Needs Triage to Missing Functionality on the Parsoid board.
Thu, Mar 9, 3:37 PM · Parsoid
cscott moved T331084: TypeError: Argument 2 passed to Wikimedia\Parsoid\Config\StubMetadataCollector::collect() must be of the type string, int given, from Needs Triage to Bugs & Crashers on the Parsoid board.
Thu, Mar 9, 3:29 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Parsoid, Wikimedia-production-error
cscott moved T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content from Needs Triage to Missing Functionality on the Parsoid board.
Thu, Mar 9, 3:28 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Parsoid
cscott added a comment to T326882: Update Product Infrastructure-owned products that may be affected by IP Masking.

The Content-Transform-Team never actually worked on ReadingLists, as far as I know. We could use some help here, since we're not familiar with these code bases at all.

Thu, Mar 9, 3:18 PM · Content-Transform-Team, IP Masking
cscott edited projects for T331493: Tweets do not display properly across themes in app , added: Page Content Service; removed Content-Transform-Team.
Thu, Mar 9, 3:15 PM · iOS Release FY2022-23, Wikipedia-Android-App-Backlog (Android Release FY2022-23), Page Content Service, Wikipedia-iOS-App-Backlog

Wed, Mar 8

cscott added a parent task for T215628: Make Parsoid and PHP edit-section numbering consistent when <noinclude> and <includeonly> are in use: T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content.
Wed, Mar 8, 6:24 PM · Patch-For-Review, Parsoid
cscott added a parent task for T213468: Parsoid section IDs don't correspond to PHP section IDs when headings are transcluded: T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content.
Wed, Mar 8, 6:24 PM · Parsoid
cscott added a parent task for T214538: MediaWiki shouldn't assign section ids during tokenization, but instead only when headings are generated: T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content.
Wed, Mar 8, 6:24 PM · Parsoid (Tracking), MediaWiki-Parser
cscott added subtasks for T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content: T213468: Parsoid section IDs don't correspond to PHP section IDs when headings are transcluded, T215628: Make Parsoid and PHP edit-section numbering consistent when <noinclude> and <includeonly> are in use, T214538: MediaWiki shouldn't assign section ids during tokenization, but instead only when headings are generated.
Wed, Mar 8, 6:24 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Parsoid
cscott added a parent task for T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content: T269630: Parsoid should support section editing links.
Wed, Mar 8, 6:23 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Parsoid
cscott added a subtask for T269630: Parsoid should support section editing links: T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content.
Wed, Mar 8, 6:23 PM · Parsoid-Read-Views (Phase 2 - testwiki Main namespace support), Parsoid
cscott added a comment to T331483: Resolve diferences between Parsoid & legacy parser TOC metadata output for template, extension, and parser-function generated content.

See also T213468: Parsoid section IDs don't correspond to PHP section IDs when headings are transcluded, T215628: Make Parsoid and PHP edit-section numbering consistent when <noinclude> and <includeonly> are in use, and the longer discussion in T269630: Parsoid should support section editing links.

Wed, Mar 8, 6:20 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Parsoid
cscott added a comment to T326465: Implement TranslateMagicWords / PreferEnglishMagicWords config option.

But the way to do that is just to reorder the aliases. The whole point of the message in MessagesEn.php is to give the wiki community control over which of the alternatives is used by Visual Editor by ordering the alises in their localization. If you want VE to use english rather than localized messages, just list the english ones first. It's a SHOULD not a MUST.

Wed, Mar 8, 6:11 PM · Patch-For-Review, Chinese-Sites, I18n, MediaWiki-Internationalization
cscott added a comment to T326465: Implement TranslateMagicWords / PreferEnglishMagicWords config option.

I'm not sure why this is needed/wanted? The guidance in MessagesEn.php states:

* Note to localisers:
*   - Include the English magic words as synonyms. This allows people from
*     other wikis that do not speak the language to contribute more easily.
*   - The first alias listed MUST be the preferred alias in that language.
*     Tools (like Visual Editor) are expected to use the first listed alias
*     when editing or creating new content.
*   - Order the other aliases so that common aliases occur before more rarely
*     used aliases. The aliases SHOULD be sorted by the following convention:
*     1. Local first, English last, then
*     2. Most common first, least common last.
* @phpcs-require-sorted-array
  • Note to localisers:
  • - Include the English magic words as synonyms. This allows people from
  • other wikis that do not speak the language to contribute more easily.
  • - The first alias listed MUST be the preferred alias in that language.
  • Tools (like Visual Editor) are expected to use the first listed alias
  • when editing or creating new content.
  • - Order the other aliases so that common aliases occur before more rarely
  • used aliases. The aliases SHOULD be sorted by the following convention:
  • 1. Local first, English last, then
  • 2. Most common first, least common last.
  • @phpcs-require-sorted-array
Wed, Mar 8, 5:10 PM · Patch-For-Review, Chinese-Sites, I18n, MediaWiki-Internationalization

Tue, Mar 7

cscott added a comment to T331316: Vector (or Vector 2022?) displays {{DISPLAYTITLE}} only when no ToC is present.

It turns out that this issue *is* related to T306862, via a side-effect of calling LanguageConverter::convertTo(). The linked patch above describes the situation in its commit message.

Tue, Mar 7, 7:47 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Chinese-Sites, MediaWiki-Parser, Desktop Improvements (Vector 2022), Vector (legacy skin)
cscott added a comment to T306862: Chinese Language Converter is not working in the sidebar table of the contents in Vector-2022.

[mediawiki/core@master] WIP: Don't clear LanguageConverter display title when converting ToC

@cscott perhaps we could capture the remaining work in a new ticket?

Tue, Mar 7, 7:46 PM · Unplanned-Sprint-Work, Readers-Web-Backlog (Web Team FY2022-23 Q3 Sprint 4), User-notice, MW-1.40-notes (1.40.0-wmf.25; 2023-02-27), Content-Transform-Team-WIP, Patch-For-Review, MediaWiki-Parser, Vector (legacy skin) (Tracking), MediaWiki-Language-converter, Chinese-Sites
cscott added a comment to T330802: Stray blockquote tag removed.

I don't *think* that parsoid would ever split an existing list in two during selser, so my intuition is that this is probably more likely a DT-side bug w/ the DOM normalization that you are doing.

Tue, Mar 7, 4:29 PM · Parsoid, DiscussionTools
cscott added a comment to T277059: [Bug] Blue links are broken on the Serbian (Latin) Burek article on iOS and Android.

Ideally we should be using MediaWiki::getTitle() exactly to do title lookup in the REST APIs, instead of trying to (partially) reimplement this.

Tue, Mar 7, 3:52 PM · Patch-For-Review, MW-1.40-notes (1.40.0-wmf.23; 2023-02-13), Content-Transform-Team-WIP, Serbian-Sites, Wikipedia-Android-App-Backlog, Product-Infrastructure-Team-Backlog-Deprecated (Kanban), Page Content Service, Wikipedia-iOS-App-Backlog

Mon, Mar 6

cscott added a comment to T331316: Vector (or Vector 2022?) displays {{DISPLAYTITLE}} only when no ToC is present.

(T26072 is a much older but perhaps related bug, although it was apparently fixed back in 2010.)

Mon, Mar 6, 6:04 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Chinese-Sites, MediaWiki-Parser, Desktop Improvements (Vector 2022), Vector (legacy skin)
cscott added a comment to T331316: Vector (or Vector 2022?) displays {{DISPLAYTITLE}} only when no ToC is present.

A somewhat cleaner diff:

Which is presently the latest version, aka the same as:

But note that

(aka the "no conversion" output) seem to appear correctly.

Mon, Mar 6, 5:40 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Chinese-Sites, MediaWiki-Parser, Desktop Improvements (Vector 2022), Vector (legacy skin)
cscott created T331316: Vector (or Vector 2022?) displays {{DISPLAYTITLE}} only when no ToC is present.
Mon, Mar 6, 5:15 PM · MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Chinese-Sites, MediaWiki-Parser, Desktop Improvements (Vector 2022), Vector (legacy skin)
cscott added a comment to T306862: Chinese Language Converter is not working in the sidebar table of the contents in Vector-2022.

@Diskdance That does seem like expected behavior. We'd like to suggest that "best style" is to ensure all conversion rules are at the top of the document (ideally in glossaries, as is typical practice on zhwiki I believe), which avoids this particular discrepancy.

Mon, Mar 6, 5:12 PM · Unplanned-Sprint-Work, Readers-Web-Backlog (Web Team FY2022-23 Q3 Sprint 4), User-notice, MW-1.40-notes (1.40.0-wmf.25; 2023-02-27), Content-Transform-Team-WIP, Patch-For-Review, MediaWiki-Parser, Vector (legacy skin) (Tracking), MediaWiki-Language-converter, Chinese-Sites

Fri, Mar 3

cscott created T331148: Unify the read-views and REST API caches for Parsoid content.
Fri, Mar 3, 4:25 PM · Parsoid

Thu, Mar 2

cscott added a comment to T258118: page/summary does not extract specific template text correctly.

Probably https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/594557 would help, although the summary endpoint needs to actually ask for the content to be converted to zh (the base language` for that patch to help out.

Thu, Mar 2, 3:27 PM · Parsoid-Read-Views, Parsoid, Product-Infrastructure-Team-Backlog-Deprecated

Tue, Feb 28

cscott updated subscribers of T326898: Don't show "add article description" at the top of pages if short description = none.

From chat with WMDE:

Tue, Feb 28, 10:39 PM · Wikidata, Wikipedia-Android-App-Backlog (Android Release FY2022-23), Patch-For-Review, Page Content Service, Content-Transform-Team-WIP
cscott updated the task description for T330677: Ensure ParserCacheMetadata isn't corrupted by 'useParsoid' option.
Tue, Feb 28, 9:57 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Patch-For-Review, Parsoid, MediaWiki-Parser
cscott updated the task description for T330677: Ensure ParserCacheMetadata isn't corrupted by 'useParsoid' option.
Tue, Feb 28, 9:48 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Patch-For-Review, Parsoid, MediaWiki-Parser
cscott updated the task description for T330677: Ensure ParserCacheMetadata isn't corrupted by 'useParsoid' option.
Tue, Feb 28, 9:11 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Patch-For-Review, Parsoid, MediaWiki-Parser
cscott updated the task description for T330677: Ensure ParserCacheMetadata isn't corrupted by 'useParsoid' option.
Tue, Feb 28, 9:09 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Patch-For-Review, Parsoid, MediaWiki-Parser
cscott updated the task description for T330677: Ensure ParserCacheMetadata isn't corrupted by 'useParsoid' option.
Tue, Feb 28, 8:59 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Patch-For-Review, Parsoid, MediaWiki-Parser
cscott updated the task description for T330677: Ensure ParserCacheMetadata isn't corrupted by 'useParsoid' option.
Tue, Feb 28, 8:55 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Patch-For-Review, Parsoid, MediaWiki-Parser
cscott added a comment to T306862: Chinese Language Converter is not working in the sidebar table of the contents in Vector-2022.

Great, thanks!

Tue, Feb 28, 7:06 PM · Unplanned-Sprint-Work, Readers-Web-Backlog (Web Team FY2022-23 Q3 Sprint 4), User-notice, MW-1.40-notes (1.40.0-wmf.25; 2023-02-27), Content-Transform-Team-WIP, Patch-For-Review, MediaWiki-Parser, Vector (legacy skin) (Tracking), MediaWiki-Language-converter, Chinese-Sites
cscott added a comment to T326898: Don't show "add article description" at the top of pages if short description = none.

We probably need to set up a conversation with Wikidata/WMDE about whether they are comfortable with representing "no short description wanted" as the empty string in wikidata. If so, then there's a trivial change to https://en.wikipedia.org/wiki/Template:Short_description which will set the short description to '' if none is passed, and @vadim-kovalenko can simply look for '' in the short description API output to indicate that the "add short description" button should be suppressed (or whatever other representation of this situation the wikidata team would prefer). If wikidata /doesn't/ want to represent this case, making it "just" an enwiki feature, then I guess there's no better solution other than to have @vadim-kovalenko look through the parsoid HTML output for the template arguments and match against the english string 'none'. As documented at https://www.mediawiki.org/wiki/Specs/HTML/2.7.0#Template_markup you'd be looking for nodes with document.querySelectorAll('[typeof="mw:Transclusion"]') and then deserializing the data-mw attribute and looking for mw.parts[0].template.target.href === './Template:Short_description' and mw.parts[0].template.params["1"].wt === 'none' -- which is pretty hacky and ugly. A regexp on the raw wikitext is also possible and also ugly. Probably "better" would be to update [[Template:Short description]] to add pages with 'none' to a special category, and then we can check for that category instead of trying to pull out the template arguments.

Tue, Feb 28, 4:03 PM · Wikidata, Wikipedia-Android-App-Backlog (Android Release FY2022-23), Patch-For-Review, Page Content Service, Content-Transform-Team-WIP
cscott added a comment to T313265: App gives no indication that user script pages are protected when reading them.

@vadim-kovalenko points out that https://en.wikipedia.org/api/rest_v1/page/mobile-html/User%3Acscott%2FTogetherJS.js "works" in production -- which apparently is just because production is still pointed to RESTBase while the development instance is pointed to the core APIs which are expected to replace RESTBase. So once that switchover is complete, apps will start displaying the "dummy content" message instead of any output, which is a bug in itself likely. Generally apps should be using the same mechanism for "non wikitext content" pages as it does for (eg) Special pages. The question is how apps should know that a given Title corresponds to a non-wikitext content type.

Tue, Feb 28, 3:48 PM · Wikipedia-Android-App-Backlog (Android Release FY2022-23), Content-Transform-Team-WIP, Page Content Service
cscott added a comment to T313265: App gives no indication that user script pages are protected when reading them.

The content on the app doesn't seem to be generated by the mobile html service, as the mobile html service shows the output that @vadim-kovalenko shows above and returns 400 for these pages (T324711). So this is probably not a mobile html service bug but instead a bug in the app itself, specifically whatever fallback path the app is using to generate content for these pages after the mobile html service returns 400. (I note that it is rendering the javascript "as if it were wikitext" as well, so there are bigger problems here than "just" the presence of the edit button.)

Tue, Feb 28, 3:27 PM · Wikipedia-Android-App-Backlog (Android Release FY2022-23), Content-Transform-Team-WIP, Page Content Service
cscott added a comment to T330712: Replace external and free classes with more descriptive and prefixed alternatives..

A free external link is an unbracketed url in wikitext; it is grouped with the other unbracketed link types (RFC/PMID/ISBN) in Parser::handleMagicLinks().

Please visit https://cscott.net <!-- free external link -->
Tue, Feb 28, 2:22 PM · MediaWiki-Parser, Content-Transform-Team

Mon, Feb 27

cscott committed rMLID40ab1fc0ee17: Bump version after release (authored by cscott).
Bump version after release
Mon, Feb 27, 5:54 PM
cscott committed rMLID4089b5cc22f0: Release 2.0.0 (and unrelease 1.0.1) (authored by cscott).
Release 2.0.0 (and unrelease 1.0.1)
Mon, Feb 27, 5:54 PM
cscott created T330677: Ensure ParserCacheMetadata isn't corrupted by 'useParsoid' option.
Mon, Feb 27, 4:54 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Patch-For-Review, Parsoid, MediaWiki-Parser
cscott added a comment to T314777: Investigate cache purging needs.

There's a related conversation about tools for managing deploys and rollbacks -- it would be useful to have robust tools for purging the cache of content generated by the 'rolled back revision' when a rollback is needed during a deploy.

Mon, Feb 27, 4:26 PM · Content-Transform-Team-WIP, Product-Infrastructure-Team-Backlog-Deprecated, Page Content Service
cscott added a comment to T313265: App gives no indication that user script pages are protected when reading them.

https://en.wikipedia.org/wiki/User:Cscott/TogetherJS.js is a user script on my page, it should be protected? And I think that https://en.wikipedia.org/wiki/User:GhostInTheMachine/SDsummary.js is the link to the protected page from @vadim-kovalenko's example.

Mon, Feb 27, 4:24 PM · Wikipedia-Android-App-Backlog (Android Release FY2022-23), Content-Transform-Team-WIP, Page Content Service
cscott added a comment to T306862: Chinese Language Converter is not working in the sidebar table of the contents in Vector-2022.

This should ride the train this week; added User-notice to get a mention on tech news. If someone could post in the zhwiki village pump that would be helpful!

Mon, Feb 27, 4:14 PM · Unplanned-Sprint-Work, Readers-Web-Backlog (Web Team FY2022-23 Q3 Sprint 4), User-notice, MW-1.40-notes (1.40.0-wmf.25; 2023-02-27), Content-Transform-Team-WIP, Patch-For-Review, MediaWiki-Parser, Vector (legacy skin) (Tracking), MediaWiki-Language-converter, Chinese-Sites
cscott added a project to T306862: Chinese Language Converter is not working in the sidebar table of the contents in Vector-2022: User-notice.
Mon, Feb 27, 4:13 PM · Unplanned-Sprint-Work, Readers-Web-Backlog (Web Team FY2022-23 Q3 Sprint 4), User-notice, MW-1.40-notes (1.40.0-wmf.25; 2023-02-27), Content-Transform-Team-WIP, Patch-For-Review, MediaWiki-Parser, Vector (legacy skin) (Tracking), MediaWiki-Language-converter, Chinese-Sites

Fri, Feb 24

cscott committed rWLWId057c1235f32: Release 1.1.0 (authored by cscott).
Release 1.1.0
Fri, Feb 24, 8:44 PM
cscott committed rWLWI952228c61c81: Update HISTORY.md (authored by cscott).
Update HISTORY.md
Fri, Feb 24, 8:44 PM
cscott committed rWLWI63a73d5fcf08: Bump HISTORY.md after release (authored by cscott).
Bump HISTORY.md after release
Fri, Feb 24, 8:44 PM
Stang awarded T306862: Chinese Language Converter is not working in the sidebar table of the contents in Vector-2022 a Party Time token.
Fri, Feb 24, 4:57 PM · Unplanned-Sprint-Work, Readers-Web-Backlog (Web Team FY2022-23 Q3 Sprint 4), User-notice, MW-1.40-notes (1.40.0-wmf.25; 2023-02-27), Content-Transform-Team-WIP, Patch-For-Review, MediaWiki-Parser, Vector (legacy skin) (Tracking), MediaWiki-Language-converter, Chinese-Sites
Daimona awarded T330228: PHPUnit infra should check for out-of-date composer.json like update.php does a Like token.
Fri, Feb 24, 1:02 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), MW-1.40-notes, MediaWiki-Core-Tests
kostajh awarded T330228: PHPUnit infra should check for out-of-date composer.json like update.php does a Like token.
Fri, Feb 24, 9:44 AM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), MW-1.40-notes, MediaWiki-Core-Tests

Thu, Feb 23

cscott added a comment to T323052: Parsoid returns mediawiki-internal codes in the Content-Language header.

This was fixed and then reverted, so there's some work to do here still.

Thu, Feb 23, 3:28 PM · Parsoid
cscott added a subtask for T327379: Language variant handling in mobileapps is inconsistent with mw core variants: T323052: Parsoid returns mediawiki-internal codes in the Content-Language header.
Thu, Feb 23, 3:27 PM · Patch-For-Review, Content-Transform-Team-WIP, Page Content Service
cscott added a parent task for T323052: Parsoid returns mediawiki-internal codes in the Content-Language header: T327379: Language variant handling in mobileapps is inconsistent with mw core variants.
Thu, Feb 23, 3:27 PM · Parsoid
cscott added a comment to T327173: Parsoid should expose cache expiry.

I think the subtask T329067 covers Parsoid collecting and propagating this information via ParserOutput (or parsoid's own name for that, ContentMetadataCollector). /This/ task probably should cover if or whether to expose some of this metadata in the PageBundle API. Generally post of the metadata collected in ParserOutput probably doesn't belong in the page bundle (?) but I think there's a reasonable argument that cache lifetime in particular is likely to be of interest to anyone consuming the content from the REST APIs.

Thu, Feb 23, 3:24 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Performance-Team (Radar), Parsoid, RESTbase Sunsetting
cscott added a comment to T327173: Parsoid should expose cache expiry.

I think I fixed this via the subtask, but it should be tested -- I don't know of any explicit checks in core for this (and there are some related patches pending in core, esp T320668 / https://gerrit.wikimedia.org/r/c/mediawiki/core/+/569628

Thu, Feb 23, 3:19 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Performance-Team (Radar), Parsoid, RESTbase Sunsetting
cscott added a parent task for T329067: Parsoid doesn't support the MagicWordFactory->getCacheTTL(): T327173: Parsoid should expose cache expiry.
Thu, Feb 23, 3:17 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), MW-1.40-notes (1.40.0-wmf.23; 2023-02-13), Patch-For-Review, Parsoid
cscott added a subtask for T327173: Parsoid should expose cache expiry: T329067: Parsoid doesn't support the MagicWordFactory->getCacheTTL().
Thu, Feb 23, 3:17 PM · Parsoid-Read-Views (Phase 1 - DiscussionTools support), Performance-Team (Radar), Parsoid, RESTbase Sunsetting
cscott added a comment to T326997: Introduce a real "page content language" variable in mw.config.

This is incorrect -- the page language is *never* a variant. Only the user interface language can be a variant.

Thu, Feb 23, 3:09 PM · MediaWiki-Interface (Tables), Content-Transform-Team, Performance-Team (Radar)
cscott added a comment to T330256: Investigate access checks for Proton: ensure it follows redirects to be compatible with RESTbase.

Might be worth double-checking language-variant redirects as well.

Thu, Feb 23, 3:07 PM · MediaWiki-Language-converter, Patch-For-Review, API Platform, Content-Transform-Team, Proton, WMF-Architecture-Team, RESTbase Sunsetting, Epic, Foundational Technology Requests, Code-Health, Platform Engineering Roadmap, Platform Engineering Roadmap Decision Making

Tue, Feb 21

cscott committed rMLBC0ec9d0d392be: Fix issues with phan and phpunit configuration; fix test case (authored by cscott).
Fix issues with phan and phpunit configuration; fix test case
Tue, Feb 21, 8:30 PM
cscott created T330232: Export TOCData via action API.
Tue, Feb 21, 8:09 PM · MediaWiki-Action-API
cscott created T330228: PHPUnit infra should check for out-of-date composer.json like update.php does.
Tue, Feb 21, 7:28 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), MW-1.40-notes, MediaWiki-Core-Tests
cscott added a comment to T327439: Use JsonCodec serialization for TOCData.

This is blocked because JsonCodec is core-only, and can't be used from the Parsoid library without a cyclic dependency. So this task is effectively blocked until JsonCodec is brought out of core. There's a proposal for doing this at https://github.com/cscott/json-codec but I'm having trouble getting consensus on what the library should look like.

Tue, Feb 21, 6:56 PM · MediaWiki-Parser
cscott added a comment to T277059: [Bug] Blue links are broken on the Serbian (Latin) Burek article on iOS and Android.

In core this is handled via LanguageConverter::findVariantLink(), which is invoked early in the web request dispatch, in MediaWiki::parseTitle(), and in the action API endpoint, in ApiPageSet::processTitlesArray(). IMO it should be done in a top-of-stack manner in the REST apis as well.

Tue, Feb 21, 5:04 PM · Patch-For-Review, MW-1.40-notes (1.40.0-wmf.23; 2023-02-13), Content-Transform-Team-WIP, Serbian-Sites, Wikipedia-Android-App-Backlog, Product-Infrastructure-Team-Backlog-Deprecated (Kanban), Page Content Service, Wikipedia-iOS-App-Backlog
cscott added a comment to T326898: Don't show "add article description" at the top of pages if short description = none.

We should be using the Wikidata api to get short descriptions, instead of trying to look for a specific template (or magic word).

Tue, Feb 21, 4:53 PM · Wikidata, Wikipedia-Android-App-Backlog (Android Release FY2022-23), Patch-For-Review, Page Content Service, Content-Transform-Team-WIP
cscott added a comment to T329662: Edited gallery captions are ignored unless gallery's data-mw is dropped.

So the state of play is that we need Ed's patches deployed, and then we can make the config change to turn on native galleries. My understanding is that we've shipped nativeGalleriesEnabled=>true as the default configuration for "quite a while" even though it's not enabled in production everywhere quite yet.

Tue, Feb 21, 4:09 PM · Parsoid, Content-Transform-Team-WIP, Patch-For-Review
cscott closed T329541: Remove unnecessary targets definitions - Kartographer as Resolved.

Merged and deployed in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Kartographer/+/888754

Tue, Feb 21, 3:32 PM · Maps (Kartographer), Content-Transform-Team-WIP
cscott closed T329541: Remove unnecessary targets definitions - Kartographer, a subtask of T328497: Remove unnecessary targets definitions, as Resolved.
Tue, Feb 21, 3:32 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Discovery-Search (Current work), Wikidata, Performance-Team (Radar), Structured-Data-Backlog, Machine-Learning-Team, Data-Engineering, Timeless, Wikistories, All-and-every-Wikisource, MediaWiki-extensions-WikimediaEvents, MediaWiki-extensions-WikimediaBadges, WikiHiero, Wikidata.org, Wikibase-Quality-Constraints, WikibaseMediaInfo, VueTest, VisualEditor, MediaWiki-extensions-UrlShortener, UploadWizard, MediaWiki-extensions-Translate, TitleBlacklist, EasyTimeline, TimedMediaHandler, SpamBlacklist, SDAW-SearchVue, MediaWiki-extensions-Score, MediaWiki-extensions-Quiz, ProofreadPage, MediaWiki-extensions-Phonos, ORES, MediaWiki-extensions-OAuth, NavigationTiming, SDAW-MediaSearch, MachineVision, MediaWiki-extensions-InputBox, MediaWiki-extensions-Graph, MediaWiki-extensions-FlaggedRevs, MediaWiki-extensions-EventLogging, DismissableSiteNotice, DiscussionTools, MediaWiki-extensions-Disambiguator, patch-welcome, Citoid, Cite, CirrusSearch, CharInsert, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-CentralAuth, MediaWiki-extensions-CategoryTree, BetaFeatures, MediaWiki-extensions-Babel, ArticlePlaceholder, Advanced-Search, ConfirmEdit (CAPTCHA extension), Social-Tools, CampaignEvents, ChessBrowser, CodeEditor, ContentTranslation, ExternalGuidance, MediaWiki-extensions-GlobalWatchlist, QuizGame, MediaWiki-extensions-Screenplay, SyntaxHighlight, Two-Column-Edit-Conflict-Merge, UniversalLanguageSelector, WikiEditor, BlueSky, Metrolook, User-Jdlrobson, Technical-Debt (RW-Tech-Debt), Front-end-Standards-Group
cscott closed T329460: Remove unnecessary targets definitions - ReadingLists, a subtask of T328497: Remove unnecessary targets definitions, as Resolved.
Tue, Feb 21, 3:30 PM · MW-1.41-notes (1.41.0-wmf.1; 2023-03-20), MW-1.40-notes (1.40.0-wmf.27; 2023-03-13), Discovery-Search (Current work), Wikidata, Performance-Team (Radar), Structured-Data-Backlog, Machine-Learning-Team, Data-Engineering, Timeless, Wikistories, All-and-every-Wikisource, MediaWiki-extensions-WikimediaEvents, MediaWiki-extensions-WikimediaBadges, WikiHiero, Wikidata.org, Wikibase-Quality-Constraints, WikibaseMediaInfo, VueTest, VisualEditor, MediaWiki-extensions-UrlShortener, UploadWizard, MediaWiki-extensions-Translate, TitleBlacklist, EasyTimeline, TimedMediaHandler, SpamBlacklist, SDAW-SearchVue, MediaWiki-extensions-Score, MediaWiki-extensions-Quiz, ProofreadPage, MediaWiki-extensions-Phonos, ORES, MediaWiki-extensions-OAuth, NavigationTiming, SDAW-MediaSearch, MachineVision, MediaWiki-extensions-InputBox, MediaWiki-extensions-Graph, MediaWiki-extensions-FlaggedRevs, MediaWiki-extensions-EventLogging, DismissableSiteNotice, DiscussionTools, MediaWiki-extensions-Disambiguator, patch-welcome, Citoid, Cite, CirrusSearch, CharInsert, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-CentralAuth, MediaWiki-extensions-CategoryTree, BetaFeatures, MediaWiki-extensions-Babel, ArticlePlaceholder, Advanced-Search, ConfirmEdit (CAPTCHA extension), Social-Tools, CampaignEvents, ChessBrowser, CodeEditor, ContentTranslation, ExternalGuidance, MediaWiki-extensions-GlobalWatchlist, QuizGame, MediaWiki-extensions-Screenplay, SyntaxHighlight, Two-Column-Edit-Conflict-Merge, UniversalLanguageSelector, WikiEditor, BlueSky, Metrolook, User-Jdlrobson, Technical-Debt (RW-Tech-Debt), Front-end-Standards-Group
cscott closed T329460: Remove unnecessary targets definitions - ReadingLists as Resolved.

Merged & deployed in:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReadingLists/+/888038

Tue, Feb 21, 3:30 PM · Content-Transform-Team-WIP, Reading List Service
cscott added a comment to T277059: [Bug] Blue links are broken on the Serbian (Latin) Burek article on iOS and Android.

The patch was applied to page bundles, but it appears the bug is with the summary endpoint? I'm a little confused. AIUI the title redirection should happen in core, and in common for all endpoints, and it's not clear a patch to do that has landed yet.

Tue, Feb 21, 3:16 PM · Patch-For-Review, MW-1.40-notes (1.40.0-wmf.23; 2023-02-13), Content-Transform-Team-WIP, Serbian-Sites, Wikipedia-Android-App-Backlog, Product-Infrastructure-Team-Backlog-Deprecated (Kanban), Page Content Service, Wikipedia-iOS-App-Backlog

Feb 17 2023

cscott added a comment to T329170: Investigation: is it feasible to drop the mapdata group ID hash magic?.

Possible plan:

  • Add backwards compatibility code that keeps the old hashes working for a while.
  • Note we plan to change the hashes anyway because of parse time expansion. So we need this either way.
  • Make the hashes stable, only based on the raw wikitext, but not on language variant and whatnot.
  • We are now in a situation where users will see maps with the wrong language variant, wrong thumbsize and whatnot. This is already a massive improvement compared to the broken maps from before.
  • Make sure the mapdata API is called with all necessary information to be able to deliver the map in the correct variant.
Feb 17 2023, 8:10 PM · MW-1.40-notes (1.40.0-wmf.23; 2023-02-13), Patch-For-Review, Maps (Kartographer), WMDE-GeoInfo-FocusArea, WMDE-TechWish-Sprint-2023-01-18

Feb 15 2023

cscott added a comment to T325586: 1.40.0-wmf.23 deployment blockers.

Ok, https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/889641 is merged to mediawiki-vendor and https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/889607 is the cherry-pick to the wmf.23 branch. We'll leave it to ops' discretion whether they want to merge that the wmf.23 and deploy the backport, or leave that cherry-pick unmerged and just suppress the errors.

Feb 15 2023, 10:37 PM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
cscott added a comment to T325586: 1.40.0-wmf.23 deployment blockers.

I'm starting the patch-and-tag-and-release-to-vendor process with https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/889637 and we'll get that merged to mainline mediawiki-vendor, then I'll post here and leave it up to you all on ops whether you want to backport that mediawiki-vendor patch to wmf.23.

Feb 15 2023, 9:09 PM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
cscott added a comment to T325586: 1.40.0-wmf.23 deployment blockers.

The spike seems related to the transition between wmf.22 and wmf.23, perhaps? There are logs like https://logstash.wikimedia.org/goto/a21be3de9c29c8905f2621f63dbb0c92 which are against wmf.22 and refer to properties which were removed from wmf.23, so it's possible the front end machines saw an inconsistent vendor directory briefly during the deploy?

Feb 15 2023, 8:44 PM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
cscott added a comment to T329740: PHP Notice: Undefined property: stdClass::$href.

As a quick-and-dirty backport fix we could also just comment out the call to ::computeSectionMetadata() too, couldn't we? That information isn't used by anything yet. That would give us more time to do a proper fix incl test case for the next train.

Feb 15 2023, 3:40 PM · MW-1.40-notes (1.40.0-wmf.24; 2023-02-20), Parsoid, Wikimedia-production-error

Feb 14 2023

cscott added a comment to T325586: 1.40.0-wmf.23 deployment blockers.

The Content-Transform-Team has continued to work on the Table of Contents. Potentially risky changes:

Feb 14 2023, 10:21 PM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
cscott added a comment to T329483: ParserOutput: Undefined array key "extensionData" warning.

I think you just need to composer update to get the latest Parsoid version, that should stop the warning for you.

Feb 14 2023, 3:58 PM · MediaWiki-Parser

Feb 13 2023

cscott updated the task description for T329532: ChangeProp may have race conditions.
Feb 13 2023, 4:35 PM · Platform Team Workboards (Platform Engineering Reliability), ChangeProp