Page MenuHomePhabricator
Create Task
Maniphest T114332

mailman - convert all language templates to UTF-8 and remove illegal chars
Closed, ResolvedPublic
Actions

Description

convert all mailman global language templates for listinfo pages (back) to UTF-8, remove all illegal characters (that break puppet), add correct characters instead, re-declare them as UTF-8 files

The fix described on T114289#1691038 for German needs to be applied to multiple other languages as well.

affected were Korean, German (linked tickets) and the following:

~/wmf/puppet/modules/mailman/files/templates$ find . -name listinfo.html -exec file {} \; | grep -v UTF-8
./ro/listinfo.html: HTML document, ASCII text
./ca/listinfo.html: HTML document, ISO-8859 text
./nl/listinfo.html: HTML document, ASCII text
./es/listinfo.html: HTML document, ISO-8859 text
./it/listinfo.html: HTML document, ASCII text
./fi/listinfo.html: HTML document, ISO-8859 text
./id/listinfo.html: HTML document, ASCII text
./en/listinfo.html: HTML document, ASCII text

Details

SubjectRepoBranchLines +/-
mailman: convert language templates, pt.2operations/puppetproduction+2 -3
mailman: convert language templates, ca,es,fi,froperations/puppetproduction+30 -25
Customize query in gerrit

Related Objects
Search...

Event Timeline

Dzahn created this task.Oct 1 2015, 12:25 AM
Dzahn raised the priority of this task from to Medium.
Dzahn updated the task description. (Show Details)
Dzahn added a project: Wikimedia-Mailing-lists.
Dzahn added subscribers: Dzahn, • JohnLewis, Platonides.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2015, 12:25 AM
gerritbot subscribed.Oct 1 2015, 12:26 AM

Change 242749 had a related patch set uploaded (by Dzahn):
mailman: convert language templates, ca,es,fi,fr

https://gerrit.wikimedia.org/r/242749

gerritbot added a project: Patch-For-Review.Oct 1 2015, 12:26 AM
Dzahn added subtasks: T114289: listinfo template encoding issues: German, T72180: HTML description page of otrs-ko mailing list broken (fix global Korean listinfo template).Oct 1 2015, 12:26 AM
Dzahn updated the task description. (Show Details)Oct 1 2015, 12:28 AM
Dzahn set Security to None.
Restricted Application added a subscriber: revi. · View Herald TranscriptOct 1 2015, 12:28 AM
Dzahn added a comment.Oct 1 2015, 12:30 AM

I say convert "back" because this is partially from https://gerrit.wikimedia.org/r/#/c/157708/ but that was a fix before the mailman upgrade back in 2014.

gerritbot added a comment.Oct 1 2015, 12:37 AM

Change 242749 merged by Dzahn:
mailman: convert language templates, ca,es,fi,fr

https://gerrit.wikimedia.org/r/242749

Dzahn mentioned this in rOPUPff3f62f46137: mailman: convert language templates, ca,es,fi,fr.Oct 1 2015, 12:37 AM
gerritbot added a comment.Oct 1 2015, 12:51 AM

Change 242771 had a related patch set uploaded (by Dzahn):
mailman: convert language templates, pt.2

https://gerrit.wikimedia.org/r/242771

gerritbot added a comment.Oct 1 2015, 12:52 AM

Change 242771 merged by Dzahn:
mailman: convert language templates, pt.2

https://gerrit.wikimedia.org/r/242771

Dzahn mentioned this in rOPUPca2136a78e73: mailman: convert language templates, pt.2.Oct 1 2015, 12:54 AM
Dzahn added a comment.Oct 1 2015, 1:00 AM

done.

./fr/listinfo.html: HTML document, UTF-8 Unicode text
./de/listinfo.html: HTML document, UTF-8 Unicode text
./hu/listinfo.html: HTML document, UTF-8 Unicode text
./zh/listinfo.html: HTML document, UTF-8 Unicode text
./pt_BR/listinfo.html: HTML document, UTF-8 Unicode text
./ta/listinfo.html: HTML document, UTF-8 Unicode text
./ro/listinfo.html: HTML document, ASCII text
./ca/listinfo.html: HTML document, UTF-8 Unicode text
./ja/listinfo.html: HTML document, UTF-8 Unicode text
./sv/listinfo.html: HTML document, UTF-8 Unicode text
./pl/listinfo.html: HTML document, UTF-8 Unicode text
./ar/listinfo.html: HTML document, UTF-8 Unicode text
./nl/listinfo.html: HTML document, ASCII text
./sk/listinfo.html: HTML document, UTF-8 Unicode text
./uk/listinfo.html: HTML document, UTF-8 Unicode text
./ko/listinfo.html: HTML document, UTF-8 Unicode text
./cs/listinfo.html: HTML document, UTF-8 Unicode text
./he/listinfo.html: HTML document, UTF-8 Unicode text
./lt/listinfo.html: HTML document, UTF-8 Unicode text
./vi/listinfo.html: HTML document, UTF-8 Unicode text
./ru/listinfo.html: HTML document, UTF-8 Unicode text
./no/listinfo.html: HTML document, UTF-8 Unicode text
./hi/listinfo.html: HTML document, UTF-8 Unicode text
./es/listinfo.html: HTML document, UTF-8 Unicode text
./it/listinfo.html: HTML document, ASCII text
./fi/listinfo.html: HTML document, UTF-8 Unicode text
./id/listinfo.html: HTML document, ASCII text
./hr/listinfo.html: HTML document, UTF-8 Unicode text
./tr/listinfo.html: HTML document, UTF-8 Unicode text
./fa/listinfo.html: HTML document, UTF-8 Unicode text
./pt/listinfo.html: HTML document, UTF-8 Unicode text
./en/listinfo.html: HTML document, ASCII text

Where it says ASCII text it's also UTF-8 but doesn't use any characters outside ASCII. I ran a loop across this to convert them all with iconv -t utf8 and there is no more change here.

Also, after doing T114289#1691038 on the server side, now puppet runs complete without crying about invalid byte sequence in UTF-8.

Dzahn closed this task as Resolved.Oct 1 2015, 1:00 AM
Dzahn claimed this task.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits