Page MenuHomePhabricator

Scap3 doesn't support non us-ascii characters in config templates for service deployments
Closed, ResolvedPublic

Description

When I'm trying to deploy a service where the config.yaml.j2 template contains non us-ascii characters, I'm getting the following error in Scap logs:

12:49:02 [scb2001.codfw.wmnet] Unhandled error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/scap/cli.py", line 339, in run
    exit_status = app.main(app.extra_arguments)
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 147, in main
    getattr(self, stage)()
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 189, in config_deploy
    overrides=overrides
  File "/usr/lib/python2.7/dist-packages/scap/template.py", line 84, in __init__
    self._template = self._env.get_template(name)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 791, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 765, in _load_template
    template = self.loader.load(self, name, globals)
  File "/usr/lib/python2.7/dist-packages/jinja2/loaders.py", line 125, in load
    code = environment.compile(source, name, filename)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 540, in compile
    source = self._parse(source, name, filename)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 459, in _parse
    return Parser(self, source, name, encode_filename(filename)).parse()
  File "/usr/lib/python2.7/dist-packages/jinja2/parser.py", line 32, in __init__
    self.stream = environment._tokenize(source, name, filename, state)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 490, in _tokenize
    source = self.preprocess(source, name, filename)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 484, in preprocess
    self.iter_extensions(), text_type(source))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2930: ordinal not in range(128)

Scap should support Unicode in the configs and vars.

Event Timeline

Pchelolo triaged this task as Medium priority.Jul 2 2018, 12:55 PM
Pchelolo created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 2 2018, 12:55 PM
Stashbot added a subscriber: Stashbot.

Mentioned in SAL (#wikimedia-operations) [2018-07-02T12:57:55Z] <ppchelko@deploy1001> Started deploy [changeprop/deploy@82fd280]: Revert: Move static blacklisting to change-prop T198386 Can't deploy yet due to scap bug T198621

Pchelolo removed mobrovac as the assignee of this task.Jul 2 2018, 12:58 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-02T12:59:21Z] <ppchelko@deploy1001> Finished deploy [changeprop/deploy@82fd280]: Revert: Move static blacklisting to change-prop T198386 Can't deploy yet due to scap bug T198621 (duration: 01m 26s)

For completeness, the deployment fails due to this patch (which is the one containing non-us-ascii chars).

thcipriani closed this task as Resolved.Jul 11 2018, 1:46 PM
thcipriani claimed this task.
thcipriani added a subscriber: thcipriani.

D1077 should have resolved this problem and it is now deployed to production. I'll close this task for now, please reopen if problem persists in production.

Thank you. Just made a deployment and verified it's working correctly with non-ascii configs