Page MenuHomePhabricator

Scap3 doesn't support non us-ascii characters in config templates for service deployments
Closed, ResolvedPublic


When I'm trying to deploy a service where the config.yaml.j2 template contains non us-ascii characters, I'm getting the following error in Scap logs:

12:49:02 [scb2001.codfw.wmnet] Unhandled error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/scap/", line 339, in run
    exit_status = app.main(app.extra_arguments)
  File "/usr/lib/python2.7/dist-packages/scap/", line 147, in main
    getattr(self, stage)()
  File "/usr/lib/python2.7/dist-packages/scap/", line 189, in config_deploy
  File "/usr/lib/python2.7/dist-packages/scap/", line 84, in __init__
    self._template = self._env.get_template(name)
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 791, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 765, in _load_template
    template = self.loader.load(self, name, globals)
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 125, in load
    code = environment.compile(source, name, filename)
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 540, in compile
    source = self._parse(source, name, filename)
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 459, in _parse
    return Parser(self, source, name, encode_filename(filename)).parse()
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 32, in __init__ = environment._tokenize(source, name, filename, state)
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 490, in _tokenize
    source = self.preprocess(source, name, filename)
  File "/usr/lib/python2.7/dist-packages/jinja2/", line 484, in preprocess
    self.iter_extensions(), text_type(source))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2930: ordinal not in range(128)

Scap should support Unicode in the configs and vars.

Event Timeline

Pchelolo created this task.
Stashbot subscribed.

Mentioned in SAL (#wikimedia-operations) [2018-07-02T12:57:55Z] <ppchelko@deploy1001> Started deploy [changeprop/deploy@82fd280]: Revert: Move static blacklisting to change-prop T198386 Can't deploy yet due to scap bug T198621

Mentioned in SAL (#wikimedia-operations) [2018-07-02T12:59:21Z] <ppchelko@deploy1001> Finished deploy [changeprop/deploy@82fd280]: Revert: Move static blacklisting to change-prop T198386 Can't deploy yet due to scap bug T198621 (duration: 01m 26s)

For completeness, the deployment fails due to this patch (which is the one containing non-us-ascii chars).

thcipriani claimed this task.
thcipriani subscribed.

D1077 should have resolved this problem and it is now deployed to production. I'll close this task for now, please reopen if problem persists in production.

Thank you. Just made a deployment and verified it's working correctly with non-ascii configs