The underlying problem is that the REST framework uses MessageValue is was intended as a nicer replacement of Message. MessageParam works with MessageValue, but not with Message. The two should be made to work together more nicely...

In T229475#9260518, @CCicalese_WMF wrote:

@daniel is there any benefit to leaving this task open at this point? If so, maybe you have more context?

In T338429#9350437, @Jdlrobson wrote:

@MSantos Web team is no familiar with the Mathoid extension so would you be able to put together a test plan?

See T351411#9338177 for some scenarios

To clarify a few points:

In T338429#9349445, @MSantos wrote:

A few notes from our discussion on mathoid future:

• New functionality relies on MathML rendered natively in browsers

In T350219#9342913, @Legoktm wrote:

These endpoints don't expose the same functionality that RESTBase did (based on the documentation). For example, there's no way to get the HTML of an old revision. How are redirects handled? Happy to file tickets for missing functionality or comment elsewhere, but right now it's incomplete for switching.

Looks like this is an actual bug. https://github.com/wikimedia/restbase/pull/1337 should fix it.

@Legoktm This works now:

curl -X POST -F "html=test" 'https://en.wikipedia.org/w/rest.php/v1/transform/html/to/wikitext/Mwbot-rs%2FDISPLAYTITLE'

In T350661#9317708, @apaskulin wrote:

All new endpoints should be documented on the wiki page. Thanks!

The transform endpoint is now live, e.g. at https://en.wikipedia.org/w/rest.php/v1/transform/ (POST only).

But the key question is: would that speed up overall test times (for a subset of patches)? And, if so, would it result in broken builds when we run the full test suite?

In T351357#9337449, @Jdforrester-WMF wrote:

Given the monolithic nature of MediaWiki, I don't think it's reasonable to think we can determine as an expert human (much less through a heuristic) what will and won't affect Wikibase (or any other sub-system).

In T351357#9337139, @Jdforrester-WMF wrote:

This has been discussed extensively many times, with lots of discussion for the benefits and costs; this task should probably be merged into one of the existing (generally Declined) tasks and that re-opened. T42008 is the oldest one I can find quickly – should we merge to that?

In T351357#9336802, @kostajh wrote:

Sorry to nitpick, but I think it's closer to 17 minutes nowadays (e.g. https://gerrit.wikimedia.org/r/c/mediawiki/core/+/972379/32#message-4c2aa7a596758a591510e1efbf28b88dcb46a57f). (See also T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min.)

In T346811#9333764, @Ladsgroup wrote:

Does that sound good to people?

@Jdlrobson @Gehel The fist step is to figure out where this should live, and who should own it. Ideally, APIs should be owned by the team that maintains the underlying logic (Cirrus), or team that maintains a similar component (RelatedPages), or a the team that consumes the API (native apps). The API Platform team can assist, but should not end up owning specific APIs.

In T350219#9326501, @daniel wrote:

In T350219#9323481, @Legoktm wrote:

The actual Rust test case that was failing was posting the page's HTML back at it to see how it roundtrips, but I couldn't figure out how to escape the HTML for the command line so I just tried with a plain string and it reproduced the 404 error so..

Ok. I'll try to figure out what's going on with that.

In T350219#9323481, @Legoktm wrote:

It can, it depends on what you post. And it only works if you had originally requested the HTML with the "stash" flag set.

That's not my experience, I've never set the stash parameter/flag but selser still applies.

In T347034#9324422, @dcausse wrote:

The RelatedArticle MW extension could perhaps be a good fit? It provides the UI for the mobile web frontend for this similar feature and is making a similar request to the search API via the action API.

According to Turnilo, the endpoint is used mostly by the Wikipedia app, and I suspect that it was built for that purpose. Maybe that means the Apps team owns it?

In T346811#9318953, @Ladsgroup wrote:

That being said, this is not about creating new wikis, this is explicitly about one-post creation work. Adding the URL to the list in https://gerrit.wikimedia.org/g/mediawiki/services/restbase/deploy/+/master/scap/vars.yaml. Who owns restbase deploy? We add that team to the post-creation ticket.

In T350219#9321406, @Legoktm wrote:

My understanding is that transformation relies on the current content for selser to work (@ssastry can correct me if I'm wrong!)

I disabled the tool.

In T320021#9319170, @taavi wrote:

You're looking at the wrong button. The correct one is this one at the bottom of the left sidebar:

I made a pull request: https://github.com/wikimedia/restbase/pull/1336

In T350219#9316219, @Legoktm wrote:

I'm seeing less 404s now (yay), but I'm still getting one when POSTing to https://test.wikipedia.org/api/rest_v1/transform/html/to/wikitext/Mwbot-rs%2FDISPLAYTITLE/554795.

I think the fix for now is to return a 404 instead of failing hard.
Note that with this fix, page/MediaWiki:Editcheck-config.json will return a 404, but page/MediaWiki:Editcheck-config.json/html will return the rendered message (modulo a fix for T349868). That seems ok to me, but it means that page/MediaWiki:Editcheck-config.json/with_html will also return a 404. That's a bit awkward.

A while ago, I wrote a doc about why creating wikis is hard: https://docs.google.com/document/d/1EyRfxM39srUGB1bFt_8W36Lthqw8BXrKqnL8se_mgng/edit

The immediate cause of the error is this:

1. HtmlOutputRendererHelper::getETag() calls getParserOutput(), and then calls ParsoidOutputAccess::getParsoidRenderID() on the ParserOutput object returned from getParserOutput().
2. ParsoidOutputAccess::getParsoidRenderID() calls getParsoidRenderId() on that ParserOutput, but gets null and throws an InvalidArgumentException.

The ruprecht tool is still running on the grid engine. If it's no longer used, please stop the grid web services and/or properly delete the tool.

In T350661#9314973, @matmarex wrote:

Duplicate of T310398? Subtask of T334238?

In T210206#9312960, @Ladsgroup wrote:

@daniel @matmarex @Krinkle the LIKE expression implementation gave me an idea for cases of ipb_range_end = ipb_range_start in WHERE conditions and possibly join conditions. We could introduce a RawValue class and turn ipb_range_end = ipb_range_start into $dbr->expr('ipb_range_end', '=', RawValue( 'ipb_range_start' ) ) instead. It's not pretty but it's clearly not a common usecase. Thoughts?

Basically flipping the default of "raw SQL unless specified" to "quoted unless explicitly specified"

@santhosh ok, I filed T350661: Expose parsoid transformation API from MediaWiki core. for enabling it.

@MSantos @ssastry Can the Conten Transform team commit to maintaining the transform endpoint as a public API? CT has been looking after this functionality when it lived in RESTbase, and has ported it to PHP. Derick and I moved it into core last year, and refactored it a bit.

Some thoughts on this:

Required changes in MediaWiki Core: sandbox/urbanecm/community-configuration in Gerrit, see Gerrit

If there is anything we didn’t consider and we should know about, please tell us in the comments in this task.

Upon invocation of a maintenance script, Community configuration 2.0 converts the content of the wiki page.

In that direction three libraries have been evaluated and the current proposal is to use Opis/json-schema

I agree that #1 seems like the most future proof approach.

In T344982#9307497, @santhosh wrote:

cxserver(nodejs)->VisualEditor Extension -> parsoid internal endpoint(production) seems a hacky round about way. If VE already exposes the production parsoid endpoint via an API, why not make the parsoid endpoint public?

I imagine that for most, if not all, of these a redirect follow would generally not retain a Host: header from the first request.

In T344982#9301593, @santhosh wrote:

http://parsoid-external-ci-access.beta.wmflabs.org - Does this use actual production wiki? Or beta.wmflabs.org? If it is beta.wmflabs.org, then we will be limited by content and supported languages right?

In T320021#9298181, @taavi wrote:

The ruprecht tool is still running on the grid engine. If it's no longer used, please stop the grid web services and/or properly delete the tool.

In T344982#9300861, @santhosh wrote:

If you need access to pagebundles or the transform endpoints, then we have to figure something out.

We need tranform endpoints from production wikis in our nodejs based cxserver. We also need the same tranform endpoints while we test/debug/develop publishing as CX works with html and do wikitext tranformation at end. We use production restbase endpoints to avoid having templates and pages available in localwiki.

In T344982#9300282, @santhosh wrote:

CX specifically need to connect to production parsoid instances for fetching pages. It does not make any sense to use local wiki pages for translation during development workflows. It will also limit our ability to debug issues related to production wiki pages.

In T350202#9299627, @Umherirrender wrote:

Maybe on diff view with diffonly=0 it is possible to directly edit the page as it is shown there. Maybe the error does not happen on diffonly=1 view as there is no content shown.
Not sure if the content after the diff is editable by the visual editor.

In T350117#9296451, @Umherirrender wrote:

Related change is https://gerrit.wikimedia.org/r/c/mediawiki/core/+/960663

The call to Authority::authorizeWrite now also checks permission of the authority against $wgNamespaceProtection for the given page, but reports any failure as ratelimited, so it needs something like https://gerrit.wikimedia.org/r/c/mediawiki/core/+/964563 to improve error handling.

In tools that perform edits (VisualEditor, DiscussionTools), I should see the block message, not an unrelated error.

We'll want a fresh start on dependency analysis. The Ruprecht projet was an early proof of concept.

In T223726#9278202, @Aklapper wrote:

Would declining tasks also imply requesting to archive the code repository?

The issue seems to be that the parsoid endpoints send a full URL in redirect responses, e.g. when redirecting from a page URL to a revision URL, e.g. /page/html/Dog to /page/html/Dog/12345. Since the target URL uses an external domain name, it will not be routed to the parsoid cluster, where the Parsoid extension is enabled on MediaWiki. The request will instead be routed to the public API cluster, where the relevant endpoints are unavailable, resulting in a 404 response.

In T350219#9297828, @hnowlan wrote:

This issue may also be affecting other public-facing services like https://en.wikipedia.org/w/rest.php/en.wikipedia.org/v3/transform/wikitext/to/html

This seems to only fail for messages that have non-wikitext content. This works fine, even though it's overwritten: https://fr.wikipedia.org/w/rest.php/v1/page/MediaWiki:Abusefilter/html

I'll take a look in a bit

I haven't done this yet, but I would recommend limiting these to autoconfirmed instead

I made a patch that allows groups to be specified in a gadget definition, e.g.

Ah, I was unaware of this mechansim in Gadgets. Thank you for the explanation.

In T349348#9278359, @JJMC89 wrote:

In T349348#9278076, @daniel wrote:

Making the purge right implcite should not cause permission errors. To the contrary, it should make sure that the permission is always granted.

Special:ListGroupRights no longer had purge for any group, and the API does not have it included in the list of a user's rights.

Pywikibot checks the if the user has the purge right before making a POST to the purge API.