Page MenuHomePhabricator

Mailman cannot correctly decode GB2312-superset mails labelled as GB2312 (non-standard behavior)
Open, MediumPublic

Description

Per the WHATWG (also W3C, choose the one you like) _[Encoding](https://encoding.spec.whatwg.org/)_ Technical Recommendation, all MIME text labelled as gb2312 should be treated as gbk (and consequently, a gb18030-gbk decoder) to properly handle mixtures from the later supersets. However, Mailman does not seem to use this behavior for such admixtures and spits back the raw base64 text, as noticed in zhwp's VPT.

This issue needs to be fixed in two steps:

  1. Aliasing. gb2312 should at least be aliased to gbk.
  2. Making a "union" decoder. WHATWG's TR uses a joint decoder of gb18030 and gbk, or in more simple terms a gb18030 decoder that also understands gbk (cp936)'s single-byte euro sign (U+20AC) at 0x80.

In addition to the GB's, Mailman should probably check for more of such aliasing problems highlighted in WHATWG's TR; after all, the wild web has so much non-standard behavior that browser makers did end up writing a "how to work with nonstandard things" guide.

Event Timeline

Restricted Application added subscribers: Cosine02, Aklapper. · View Herald TranscriptAug 23 2017, 1:07 AM
Arthur2e5 updated the task description. (Show Details)Aug 23 2017, 1:11 AM

Does this seem to be an issue with the OTRS system?

我说,OTRS的问题去OTRS那边提issue啊,phabricator这边也管不到啊。还有,我在VPT上提的问题是mailman的,中文维基unblock没用OTRS。

Restricted Application added a subscriber: jeblad. · View Herald TranscriptAug 25 2017, 6:08 PM
Arthur2e5 renamed this task from OTRS cannot correctly decode GB2312-superset mails labelled as GB2312 (non-standard behavior) to Mailman cannot correctly decode GB2312-superset mails labelled as GB2312 (non-standard behavior).Aug 25 2017, 6:25 PM
Arthur2e5 updated the task description. (Show Details)
jeblad removed a subscriber: jeblad.Aug 25 2017, 9:22 PM
Restricted Application added a project: Operations. · View Herald TranscriptNov 19 2019, 10:56 AM
crusnov triaged this task as Medium priority.Nov 19 2019, 4:26 PM
ema moved this task from Backlog to General on the Wikimedia-Mailing-lists board.Jun 24 2020, 1:51 PM