mailman: listinfo template encoding
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Dzahn
	Aug 28 2015, 4:17 PM

Description

current mailman class and templates on newer puppet and distro:

Warning: /Stage[main]/Mailman::Listserve/Exec[dpkg-reconfigure mailman]: Skipping because of failed dependencies
Error: invalid byte sequence in UTF-8
Error: /Stage[main]/Mailman::Webui/File[/etc/mailman/fr/listinfo.html]/content: change from {md5}758d1040b52a79891ea3fe4ae206b3ef to {md5}4334e7ddc4c307abc6158e221fc962f9 failed: invalid byte sequence in UTF-8

~/wmf/puppet/modules/mailman/files/templates$ find . -name listinfo.html -exec file {} \;
./fr/listinfo.html: HTML document, ISO-8859 text
./de/listinfo.html: HTML document, ISO-8859 text
./hu/listinfo.html: HTML document, UTF-8 Unicode text
./zh/listinfo.html: HTML document, UTF-8 Unicode text
./pt_BR/listinfo.html: HTML document, UTF-8 Unicode text
./ta/listinfo.html: HTML document, UTF-8 Unicode text
./ro/listinfo.html: HTML document, ASCII text
./ca/listinfo.html: HTML document, ISO-8859 text
./ja/listinfo.html: HTML document, UTF-8 Unicode text
./sv/listinfo.html: HTML document, UTF-8 Unicode text
./pl/listinfo.html: HTML document, UTF-8 Unicode text
./ar/listinfo.html: HTML document, UTF-8 Unicode text
./nl/listinfo.html: HTML document, ASCII text
./sk/listinfo.html: HTML document, UTF-8 Unicode text
./uk/listinfo.html: HTML document, UTF-8 Unicode text
./ko/listinfo.html: HTML document, ISO-8859 text
./cs/listinfo.html: HTML document, UTF-8 Unicode text
./he/listinfo.html: HTML document, UTF-8 Unicode text
./lt/listinfo.html: HTML document, UTF-8 Unicode text
./vi/listinfo.html: HTML document, UTF-8 Unicode text
./ru/listinfo.html: HTML document, UTF-8 Unicode text
./no/listinfo.html: HTML document, UTF-8 Unicode text
./hi/listinfo.html: HTML document, UTF-8 Unicode text
./es/listinfo.html: HTML document, ISO-8859 text
./it/listinfo.html: HTML document, ASCII text
./fi/listinfo.html: HTML document, ISO-8859 text
./id/listinfo.html: HTML document, ASCII text
./hr/listinfo.html: HTML document, UTF-8 Unicode text
./tr/listinfo.html: HTML document, UTF-8 Unicode text
./fa/listinfo.html: HTML document, UTF-8 Unicode text
./pt/listinfo.html: HTML document, UTF-8 Unicode text
./en/listinfo.html: HTML document, ASCII text

Details

	Subject	Repo	Branch	Lines +/-
	mailman: listinfo templates from ISO-8859 to UTF-8	operations/puppet	production	+51 -51

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	faidon	T84041 Replace all instances of lighttpd with nginx
Resolved	faidon	T84053 mailman - replace lighttpd
Resolved	LSobanski	T111653 Encrypt all the things
Resolved	faidon	T82576 Enable STARTTLS (both inbound and outbound) on lists
Resolved	JanZerebecki	T55259 Add Forward Secrecy to all HTTPS sites
Resolved	Dzahn	T90351 Improve SSL of lists.wikimedia.org
Resolved	Dzahn	T83541 Upgrade Exim to >=4.73
Duplicate	None	T97492 Upgrade to Mailman 3.0
Resolved	Dzahn	T110141 TTL back up to normal 1H
Resolved	• MZMcBride	T27231 Mailman mailing list archiver truncates if a line begins with "From"
Resolved	None	T66818 Mitigate strict DMARC policy on the mailing lists
Resolved	Dzahn	T80945 Get rid of all Ubuntu Lucid (10.04) installs
Resolved	Dzahn	T82698 shutdown sodium after mailman has migrated to jessie VM
Resolved	Dzahn	T105756 Mailman Upgrade (Jessie & Mailman 2.x) and migration to a VM
Resolved	Dzahn	T109925 apply regular lists role on fermium and confirm no issues
Resolved	Dzahn	T110695 mailman: listinfo template encoding

Event Timeline

Dzahn created this task.Aug 28 2015, 4:17 PM

Dzahn raised the priority of this task from to Needs Triage.

Dzahn updated the task description. (Show Details)

Dzahn added a project: Wikimedia-Mailing-lists.

Dzahn subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 28 2015, 4:17 PM

Dzahn added a parent task: T109925: apply regular lists role on fermium and confirm no issues.Aug 28 2015, 4:18 PM

Dzahn mentioned this in T109925: apply regular lists role on fermium and confirm no issues.

/wmf/puppet/modules/mailman/files/templates$ iconv -f ASCII -t UTF-8 fr/listinfo.html > fr_listinfo.html
iconv: illegal input sequence at position 4683

/wmf/puppet/modules/mailman/files/templates$ for templ in de ca ko es fi fr; do iconv -f ISO-8859-1 -t UTF-8 ${templ}/listinfo.html > ${templ}_listinfo.html ; done

/wmf/puppet/modules/mailman/files/templates$ for templ in de ca ko es fi fr; do mv ${templ}_listinfo.html ./${templ}/listinfo.html; git add ./${templ}/listinfo.html; done

Change 234565 had a related patch set uploaded (by Dzahn):
mailman: listinfo templates from ISO-8859 to UTF-8

https://gerrit.wikimedia.org/r/234565

gerritbot added a project: Patch-For-Review.Aug 28 2015, 4:36 PM

Change 234565 abandoned by Dzahn:
mailman: listinfo templates from ISO-8859 to UTF-8

Reason:
only the French one caused actual issues and that is fixed. some of the other ones look more broken after conversion, so won't do that

https://gerrit.wikimedia.org/r/234565

only fixed the French one:

https://gerrit.wikimedia.org/r/#/c/234589/

that was the only one actually causing puppet fail.

fixed now. error is gone

Dzahn closed this task as Resolved.Aug 28 2015, 6:17 PM

Dzahn claimed this task.