Page MenuHomePhabricator

scap-rebuild-cdbs failed: <AttributeError> 'bool' object has no attribute 'encode'
Closed, ResolvedPublic

Description

Beta cluster scap runs died ( https://integration.wikimedia.org/ci/job/beta-scap-eqiad/86094/console ). First failure happened at 3am UTC:

03:05:38 l10n merge:  86% (ok: 338; fail: 0; left: 54)                                   
03:05:38 l10n merge:  86% (ok: 339; fail: 0; left: 53)                                   
03:05:38 03:05:38 Failure processing (u'/srv/mediawiki/php-master/cache/l10n', u'l10n_cache-jbo.cdb', True)
03:05:38 Traceback (most recent call last):
03:05:38   File "/srv/deployment/scap/scap/scap/tasks.py", line 429, in update_l10n_cdb_wrapper
03:05:38     return update_l10n_cdb(*args)
03:05:38   File "/srv/deployment/scap/scap/scap/utils.py", line 252, in context_wrapper
03:05:38     return func(*args, **kwargs)
03:05:38   File "/srv/deployment/scap/scap/scap/tasks.py", line 405, in update_l10n_cdb
03:05:38     writer.put(key.encode('utf-8'), value.encode('utf-8'))
03:05:38 AttributeError: 'bool' object has no attribute 'encode'
03:05:38 03:05:38 Unhandled error:
03:05:38 Traceback (most recent call last):
03:05:38   File "/srv/deployment/scap/scap/scap/cli.py", line 275, in run
03:05:38     exit_status = app.main(extra_args)
03:05:38   File "/srv/deployment/scap/scap/scap/main.py", line 268, in main
03:05:38     cache_dir, use_cores, True, self.arguments.mute)
03:05:38   File "/srv/deployment/scap/scap/scap/utils.py", line 252, in context_wrapper
03:05:38     return func(*args, **kwargs)
03:05:38   File "/srv/deployment/scap/scap/scap/tasks.py", line 208, in merge_cdb_updates
03:05:38     itertools.repeat(trust_mtime))), 1):
03:05:38   File "/usr/lib/python2.7/multiprocessing/pool.py", line 659, in next
03:05:38     raise value
03:05:38 AttributeError: 'bool' object has no attribute 'encode'
03:05:38 03:05:38 scap-rebuild-cdbs failed: <AttributeError> 'bool' object has no attribute 'encode'
03:05:38 
03:05:38 scap-rebuild-cdbs: 100% (ok: 0; fail: 7; left: 0)                               
03:05:38 scap-rebuild-cdbs: 100% (ok: 0; fail: 7; left: 0)                               
03:05:38 
03:05:38 03:05:38 7 hosts had scap-rebuild-cdbs errors
03:05:38 03:05:38 Finished scap-rebuild-cdbs (duration: 02m 32s)

Seems one of the jbo language file has some scary value.

Event Timeline

hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar subscribed.

I have pick the jbo file from: deployment-bastion.deployment-prep.eqiad.wmflabs:/srv/mediawiki/php-master/cache/l10n/upstream/l10n_cache-jbo.cdb.json

Parsed it with:

import json
with open('l10n_cache-jbo.cdb.json', 'r') as f:
    d = json.load(f)

for k, v in d.iteritems():
    if type(v) != unicode:
        print "Not unicode:\n\tK: %s (%s)\n\tV: %s (%s)" % (
            k, type(k),
            v, type(v),
            )
Not unicode:
	K: messages:apihelp-query+globalblocks-param-ids (<type 'unicode'>)
	V: False (<type 'bool'>)

And indeed the json file has:

{
"messages:apihelp-query+globalblocks-param-ids": false
}

That comes from GlobalBlocking extension. Not much has changed there though... Note the l10n file is from Jan 12 02:55 UTC.

I have nuked the whole l10n cache under /srv/mediawiki-staging and triggered https://integration.wikimedia.org/ci/job/beta-scap-eqiad/86154/

hashar claimed this task.

Fixed by nuking the l10n cache.

That is happening again. Will nuke l10n cache and see whether it fix it.