Return 415 Media Type not Supported errors for pdfs and other types of unsupported formats in the citoid back end.
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	Mvolz
	May 22 2024, 11:55 AM

Description

Partially done in: https://gerrit.wikimedia.org/r/c/mediawiki/services/citoid/+/1031870

This now returns a 415 error in citoid; however restbase/hyperswitch doesn't return the native error message in citoid so it just says

 {
  "type": "https://mediawiki.org/wiki/HyperSwitch/errors/unknown_error",
  "method": "get",
  "uri": "/en.wikipedia.org/v1/data/citation/mediawiki/https%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F9%2F98%2FColoring_page_for_Wikipedia_Day_2019_in_NYC.pdf"
}

Needs follow up to:

Detect content-type in citoid first (currently this is done only in Zotero)
Return specific content-type in error, i.e. 'application/pdf'
Allow restbase/hyperswitch to correctly pass through the error itself. Slightly complicated by restbase decomm being in progress.

Details

	Subject	Repo	Branch	Lines +/-
	Check content-type	mediawiki/services/citoid	master	+206 -200
	Add documentation of 415 error to spec	mediawiki/services/citoid	master	+7 -1

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T362379 Several major news websites (NYT, NPR, Reuters...) block citoid
Open	None	T364900 Add enhanced logging to Citoid
Open	None	T364903 [SPIKE] Determine how specific we can be about logging why Citoid is failing
Open	None	T214038 Better UI flow for PDFs
Open	None	T365583 Return 415 Media Type not Supported errors for pdfs and other types of unsupported formats in the citoid back end.

Event Timeline

Mvolz created this task.May 22 2024, 11:55 AM

Mvolz mentioned this in T364903: [SPIKE] Determine how specific we can be about logging why Citoid is failing.May 22 2024, 11:57 AM

You can now see 415 errors seperate from 404s: https://grafana.wikimedia.org/d/NJkCVermz/citoid?orgId=1&refresh=5m&from=1716376689141&to=1716380289141&viewPanel=13

Mvolz added a parent task: T214038: Better UI flow for PDFs.May 22 2024, 12:20 PM

I deployed a quick change today that reports a 415 if Zotero reports an unsupported media type (T365583) It also supposedly prevents us from re-scraping the page, avoiding downloading the pdf twice once Zotero fails.

On the plus side, looking at those saturation panels, you can see it immediately reduced our network transmission, as well as CPU and memory usage (CPU was most dramatic) which is nice: https://grafana.wikimedia.org/d/NJkCVermz/citoid?orgId=1&refresh=5m&from=1716376689141&to=1716380289141

Weirdly, the total request volume jumped seemingly in direct response: https://grafana.wikimedia.org/d/NJkCVermz/citoid?orgId=1&refresh=5m&from=1716376689141&to=1716380289141&viewPanel=13

We have much fewer 500s (good!) but I was more expecting 404s to drop and 415 to rise in equal measure. But instead, 404s rose. Because previously a 415 should have been a 404. And I expected 200s to remain roughly stable, but those have also jumped. I can't really explain this.

My two theories are: metrics are broken and now we're double counting some incoming requests, or:

Third party suddenly getting 415s doesn't know how to handle them and makes the request again? But that wouldn't explain why we're getting more 200s?

Change #1034917 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Add documentation of 415 error to spec

https://gerrit.wikimedia.org/r/1034917

gerritbot added a project: Patch-For-Review.May 22 2024, 1:32 PM

Lofhi awarded a token.May 22 2024, 3:22 PM

Mvolz moved this task from Backlog to Service on the Citoid board.May 28 2024, 1:28 PM

Mvolz added a project: Platform Engineering.

VPuffetMichel moved this task from To Triage to Triaged on the VisualEditor board.Jun 4 2024, 10:37 PM

Change #1034917 merged by jenkins-bot:

[mediawiki/services/citoid@master] Add documentation of 415 error to spec

https://gerrit.wikimedia.org/r/1034917

Maintenance_bot removed a project: Patch-For-Review.Jun 6 2024, 3:30 PM

ppelberg moved this task from Incoming to Code Review on the Editing-team (Kanban Board) board.Jul 2 2024, 4:22 PM

I'm not sure how long it will take to fix the hyper switch issue. We might need to switch from restbase to the api gateway first.

At this point we could definitely return a generic unsupported format error to the user in response to 415s though.

In T365583#9946857, @Mvolz wrote...

@Mvolz, three questions in response to this update...

1) What work would be involved to, as you described, "...detect content-type in citoid first."?

2) How – if at all – does allowing, "...restbase/hyperswitch to correctly pass through the error itself." impact our ability to detect the content-type that caused the error to be activated and subsequently, offer people feedback specific to the content-type they're trying to cite?

3) To be doubly sure I'm understanding the state of this work, can you please share what – if anything – about the Current state section below is missing/inaccurate?

Current state

A. We are now logging when Citoid returns a 415 error in response to people attempting to generate a citation for a media format/type Citoid does not currently support.

See this in action by visiting this Grafana dashboard.

B. For each 415 error, we are NOT yet able to detect the content-type that caused the error to be activated

See question "1)" above for what work needs to be done to offer this support.

C. Because of "B.", we are NOT yet able to offer people feedback that is specific to the content-type they are trying to use Citoid to cite (T364594).

Note: as Marielle described in T365583#9946857, we could offer people, "...a generic unsupported format error..." in the time between now and when "B." is addressed. I've filed T369547 for this interim work.

ppelberg mentioned this in T368802: Identify patterns in Citoid requests/traffic .Jul 8 2024, 7:55 PM

ppelberg mentioned this in T368988: Investigate Citoid feature use.Jul 9 2024, 7:50 PM

In T365583#9962638, @ppelberg wrote:

In T365583#9946857, @Mvolz wrote...

@Mvolz, three questions in response to this update...

1) What work would be involved to, as you described, "...detect content-type in citoid first."?

A patch in the backend, but I would do this after 2) is resolved.

2) How – if at all – does allowing, "...restbase/hyperswitch to correctly pass through the error itself." impact our ability to detect the content-type that caused the error to be activated and subsequently, offer people feedback specific to the content-type they're trying to cite?

Blocked by T361576. Basically, restbase handling of 415s is broken, but we need to switch to Gateway from restbase anyway so shouldn't invest time in fixing this but rather on switching to gateway (which needs to be done imminently).

3) To be doubly sure I'm understanding the state of this work, can you please share what – if anything – about the Current state section below is missing/inaccurate?

Current state

A. We are now logging when Citoid returns a 415 error in response to people attempting to generate a citation for a media format/type Citoid does not currently support.

See this in action by visiting this Grafana dashboard.

B. For each 415 error, we are NOT yet able to detect the content-type that caused the error to be activated

See question "1)" above for what work needs to be done to offer this support.

C. Because of "B.", we are NOT yet able to offer people feedback that is specific to the content-type they are trying to use Citoid to cite (T364594).

Note: as Marielle described in T365583#9946857, we could offer people, "...a generic unsupported format error..." in the time between now and when "B." is addressed. I've filed T369547 for this interim work.