Page MenuHomePhabricator

Ensure file.encoding is UTF-8 for all JVMs in the Analytics Cluster
Closed, ResolvedPublic5 Story Points

Description

T128295 was annoying!

We don't know why the JVM processes locale was messed up, but if we manually set file.encoding for all JVM processes, this wouldn't happen. Ideally we could set this for all JVMs, but we might have to manually set it in all of the individual JVM CLIs.

Event Timeline

Ottomata created this task.Mar 2 2016, 4:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 2 2016, 4:53 PM

According to: http://javarevisited.blogspot.com/2012/01/get-set-default-character-encoding.html it seems it's possible to set an environment variable:

export JAVA_TOOL_OPTIONS=-Dfile.encoding="UTF-8"

And the jvm should pick that up on each run. I tested this on my local machine and it worked.

Milimetric triaged this task as High priority.

Change 276010 had a related patch set uploaded (by Ottomata):
Set file.encoding=UTF-8 for all java processes in analytics cluster

https://gerrit.wikimedia.org/r/276010

Ottomata moved this task from Next Up to In Code Review on the Analytics-Kanban board.
Ottomata moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Ottomata claimed this task.

Change 276010 merged by Ottomata:
Set file.encoding=UTF-8 for all java processes in analytics cluster

https://gerrit.wikimedia.org/r/276010

Cool! Now on nodemanager restart:

Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
Milimetric set the point value for this task to 5.Mar 10 2016, 5:14 PM
Nuria closed this task as Resolved.Mar 22 2016, 7:14 PM