Maniphest T206253

Some ApiParse requests fail due to fatal MWException: "not supported for content model"
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	Krinkle
	Oct 4 2018, 6:45 PM

Description

Error

Request ID: W6M@7grAMG8AAAerQtQAAABL
Request: GET, /w/api.php?...prop=wikitext..&contentformat=text/plain

exception

MWException: Format text/plain is not supported for content model wikitext

#0 /srv/mediawiki/php-1.32.0-wmf.20/includes/content/TextContentHandler.php(46): ContentHandler->checkFormat(string)
#1 /srv/mediawiki/php-1.32.0-wmf.20/includes/content/AbstractContent.php(155): TextContentHandler->serializeContent(WikitextContent, string)
#2 /srv/mediawiki/php-1.32.0-wmf.20/includes/api/ApiParse.php(463): AbstractContent->serialize(string)
#3 /srv/mediawiki/php-1.32.0-wmf.20/includes/api/ApiMain.php(1587): ApiParse->execute()
#4 /srv/mediawiki/php-1.32.0-wmf.20/includes/api/ApiMain.php(531): ApiMain->executeAction()
#5 /srv/mediawiki/php-1.32.0-wmf.20/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#6 /srv/mediawiki/php-1.32.0-wmf.20/api.php(87): ApiMain->execute()
#7 /srv/mediawiki/w/api.php(3): include(string)

Notes

API request fatal rate is increased due to this issue.

It appears that this is a matter of a certain feature not being available yet, or intentionally not provided. That's probably fine, at least in the short-term, but it needs to not crash the request to avoid false alarms. As a stop-gap solution it can probably be caught somewhere along the way and simply reject explicitly so as to mark it a user error (rather than server error).

Subject	Repo	Branch	Lines +/-
ApiParse: Ignore 'contentformat' param when the query is based on page	mediawiki/core	master	+14 -3
ApiParse: Fail gracefully when the requested format is not supported	mediawiki/core	master	+22 -1
content: Throw MWContentSerializationException instead of normal MWException when asserting format	mediawiki/core	master	+26 -4

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Declined		dchen	T118706 Conduct heuristic evaluation of image upload and insert flow in VisualEditor
Open		None	T115858 Design improvements for mw.ForeignStructuredUpload.BookletLayout
Open		None	T115865 Insert image in content immediately after it's uploaded, skipping the "General settings" step
Duplicate		None	T115864 Figure out if the description of the image can be used as the caption on-wiki
Open	Feature	None	T53032 When inserting an image, set its caption by default to be the Commons image description
Open	BUG REPORT	None	T12863 Links on commons upload summaries do not link to commons
Open		None	T5498 Image history is confusing
Stalled		None	T96384 Integrate file revisions with description page history
Open	Feature	None	T39534 Wikimedia Commons should support searching by color
Duplicate		None	T39535 Wikimedia Commons should support filtering by color
Resolved		None	T19503 Provide metadata support on Wikimedia Commons
Open		None	T122038 Moving/Deleting a video file does not move/delete subtitles files, nor does it inform the file mover of their existence
Open		None	T135221 Make TimedText content an integral part of the File page
Open	Feature	None	T2167 Use a dedicated interface for adding meta-data like interwiki links, rather than wikitext
Open		None	T132072 Integrate page meta-data as a new content model revision slot for consistency and atomicity
Resolved		None	T51662 VisualEditor: Use Multimedia/Wikidata's proposed rich structured meta-data in the image insertion dialog
Resolved		None	T68108 [Epic] Store media information for files on Wikimedia Commons as structured data
Open	Feature	None	T14963 allow per-page exceptions to spam blacklist
Open		None	T203157 Make the spam whitelist its own slot
Open	Feature	None	T56140 Move TemplateData to its own JSON-content namespace and associate with Template-namespace, or to its own TemplateData content model and revision slot
Stalled		None	T174043 Deploy Multi-Content Revisions
Open		None	T174022 Implement multi-content revisions
Open		None	T174032 Make relevant API modules aware of MCR
Open		None	T200572 Make ApiParse API module aware of MCR
Resolved	PRODUCTION ERROR	Func	T206253 Some ApiParse requests fail due to fatal MWException: "not supported for content model"

Event Timeline

Krinkle created this task.Oct 4 2018, 6:45 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 4 2018, 6:45 PM

What seems to be going on is that it's using a content format intended for interpretation of input text (of which there isn't any in that request) to try and format the output. It looks like ApiEditPage likely has the same issue.

The direct solution would be to remove the passing of a format when calling Content->serialize(). But since both of these modules are going to need some major work soon for MCR anyway, I'm probably not going to bother doing that as a separate patch.

Recently triggered via urls such as https://en.wikipedia.org/w/api.php?action=parse&page=Example&contentformat=application/json&prop=wikitext&section=2.

Suppressing in Logstash/new-errors is difficult due the message not having a normal form without variance by user input, and the exception file/line of TextContentHandler.php(46) is too wide.

Using ApiParse.php(463): AbstractContent->serialize(string) for now, seems to be the most specific, whilst still potentially ignoring other errors in the same kind of API call.

Krinkle moved this task from Apr 2019 / 1.33.wmf.25+ to Older on the Wikimedia-production-error board.Oct 16 2018, 11:30 PM

Krinkle renamed this task from Some ApiParse reqs fatal due to contentformat to Some ApiParse requests fail due to fatal MWException: "not supported for content model".Nov 4 2018, 8:44 PM

Krinkle added a project: Platform Engineering.

Still seen in the logs for 1.33-wmf.1. Stack trace changed, hence it resurfaced. Exposed from the same urls as T206253#4672796.

MWException: Format text/javascript is not supported for content model wikitext

#0 /srv/mediawiki/php-1.33.0-wmf.1/includes/content/TextContentHandler.php(116): ContentHandler->checkFormat(string)
#1 /srv/mediawiki/php-1.33.0-wmf.1/includes/content/ContentHandler.php(145): TextContentHandler->unserializeContent(string, string)
#2 /srv/mediawiki/php-1.33.0-wmf.1/includes/api/ApiParse.php(222): ContentHandler::makeContent(string, Title, string, string)
#3 /srv/mediawiki/php-1.33.0-wmf.1/includes/api/ApiMain.php(1570): ApiParse->execute()
#4 /srv/mediawiki/php-1.33.0-wmf.1/includes/api/ApiMain.php(531): ApiMain->executeAction()
#5 /srv/mediawiki/php-1.33.0-wmf.1/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#6 /srv/mediawiki/php-1.33.0-wmf.1/api.php(87): ApiMain->execute()
#7 /srv/mediawiki/w/api.php(3): include(string)

CCicalese_WMF added a parent task: T200572: Make ApiParse API module aware of MCR.Nov 5 2018, 3:14 PM

CCicalese_WMF edited projects, added Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)), Platform Team Legacy (Later); removed Platform Engineering.Nov 5 2018, 3:22 PM

Krinkle mentioned this in Blog Post: Production Excellence #5: November 2018.Dec 12 2018, 4:40 AM

Krinkle mentioned this in Blog Post: Production Excellence #6: December 2018.Jan 22 2019, 2:12 AM

WDoranWMF moved this task from Later to Mop Column on the Platform Team Legacy board.Jul 5 2019, 3:45 PM

WDoranWMF removed a project: Platform Team Legacy (Later).

• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:08 PM

WDoranWMF triaged this task as Low priority.Sep 3 2019, 6:43 PM

WDoranWMF edited projects, added Platform Team Workboards (Clinic Duty Team); removed Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)).

daniel moved this task from Inbox to Backlog on the Platform Team Workboards (Clinic Duty Team) board.Oct 8 2019, 9:57 AM

• AMooney changed the task status from Open to Stalled.Mar 12 2020, 1:22 PM

• AMooney edited projects, added Platform Engineering; removed Platform Team Workboards (Clinic Duty Team).

WDoranWMF moved this task from Inbox to Icebox on the Platform Engineering board.Mar 12 2020, 1:29 PM

WDoranWMF edited projects, added Platform Engineering (Icebox); removed Platform Engineering.Mar 23 2020, 2:05 PM

Restricted Application edited projects, added Platform Engineering; removed Platform Engineering (Icebox). · View Herald TranscriptMar 23 2020, 2:05 PM

WDoranWMF moved this task from Icebox to Icebox on the Platform Engineering board.Mar 23 2020, 2:18 PM

WDoranWMF edited projects, added Platform Engineering (Icebox); removed Platform Engineering.

brennen mentioned this in T255700: "Bad content model: expected wikitext but got javascript" while saving/stashing an edit.Jun 24 2020, 6:21 PM

Still seen in Logstash.

In T206253#6488993, @Krinkle wrote:

Still seen in Logstash.

It's trivial to fix I believe, if someone is committing to review patch for that

Aklapper removed a subscriber: Anomie.Oct 16 2020, 5:02 PM

The previous comments don't explain who or what (task?) exactly this task is stalled on ("If a report is waiting for further input (e.g. from its reporter or a third party) and can currently not be acted on"). Hence resetting task status.

(Smallprint, as general orientation for task management: If you wanted to express that nobody is currently working on this task, then the assignee should be removed and/or priority could be lowered instead. If work on this task is blocked by another task, then that other task should be added via Edit Related Tasks... → Edit Subtasks. If this task is stalled on an upstream project, then the Upstream tag should be added. If this task requires info from the task reporter, then there should be instructions which info is needed. If this task needs retesting, then the TestMe tag should be added. If this task is either out of scope and nobody should ever work on this, or nobody else managed to reproduce the problem described in this task, then this task should have the "Declined" status. If the task is valid but should not appear on some team's workboard, then the team project tag should be removed while the task has another active project tag.)

Umherirrender mentioned this in T297899: MWException: Format application/json is not supported for content model wikitext.Dec 17 2021, 3:01 PM

Krinkle merged a task: T297899: MWException: Format application/json is not supported for content model wikitext.Jan 11 2022, 11:30 PM

Krinkle added subscribers: cjming, Umherirrender, Zabe.

Platform team: I'm moving this back to request re-triage as it is still a current issue. Feel free to undo. (It seemed like it may've been unintentional to move an active production error to an icebox?)

Ammarpad unsubscribed.Jan 12 2022, 5:59 AM

Change 766322 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@master] ApiParse: Fail gracefully when the requested format is not supported

https://gerrit.wikimedia.org/r/766322

gerritbot added a project: Patch-For-Review.Feb 28 2022, 5:01 AM

Change 766563 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@master] Throw MWContentSerializationException instead of normal MWException

https://gerrit.wikimedia.org/r/766563

Change 766563 abandoned by Func:

[mediawiki/core@master] content: Throw MWContentSerializationException instead of normal MWException when asserting format

Reason:

per comment

https://gerrit.wikimedia.org/r/766563

Change 767799 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@master] ApiParse: Ignore contentmodel param when the query is based on page

https://gerrit.wikimedia.org/r/767799

Change 766322 merged by jenkins-bot:

[mediawiki/core@master] ApiParse: Fail gracefully when the requested format is not supported

https://gerrit.wikimedia.org/r/766322

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.1; 2022-03-21-early).Mar 19 2022, 8:00 PM

Change 767799 merged by jenkins-bot:

[mediawiki/core@master] ApiParse: Ignore 'contentformat' param when the query is based on page

https://gerrit.wikimedia.org/r/767799

Technical the last patch set was a breaking change for an undocumented feature. The parameter contentformat is documented as "Content serialization format used for the input text. Only valid when used with text.", but the format was also applied for non-text parses like page=. Such transformation can only happen for ContentHandler with two or more formats. Core does not have such a ContentHandler, but Wikibase.

At the moment requests like shows the format change
https://www.wikidata.org/w/api.php?action=parse&page=Property:P31&prop=wikitext&contentformat=application/vnd.php.serialized
https://www.wikidata.org/w/api.php?action=parse&page=Property:P31&prop=wikitext&contentformat=application/json

While invalid formats are still failing
https://www.wikidata.org/w/api.php?action=parse&page=Property:P31&prop=wikitext&contentformat=text/unknown

With the patch set all requests returning the same result, which is identical as requested without the parameter
https://www.wikidata.org/w/api.php?action=parse&page=Property:P31&prop=wikitext

As replacement it seems prop=revision with the contentformat is usable.

I just want to write it down, but I think there is no need for clarification. Requesting the content of wikidata item as wikitext seems like a bad use case.

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.3; 2022-03-22); removed MW-1.39-notes (1.39.0-wmf.1; 2022-03-21-early).Mar 21 2022, 6:00 PM

Maintenance_bot removed a project: Patch-For-Review.Mar 21 2022, 6:12 PM

Umherirrender mentioned this in T304348: EntityContentDataCodec: InvalidArgumentException: Unsupported encoding format: text/plain.Mar 21 2022, 8:20 PM

Umherirrender mentioned this in T279585: Content serialization failed: Failed to decode as application/json (when parsing edit summary via API).Aug 4 2022, 10:58 PM

Soda mentioned this in T321446: ProofreadPage index JSON access not working.Oct 23 2022, 2:39 AM

This also broke a documented (and used) feature of ProofreadPage, where the fields of an Index namespace page can be serialised as JSON.

Documentation of the broken feature: https://www.mediawiki.org/wiki/Extension:ProofreadPage/Index_data_API

Issue for the breakage: T321446