Drafttopic breaks ?features parameter
Closed, ResolvedPublic

Description

awight@ores1001:/srv/deployment/ores/deploy$ curl http://localhost:8081/v3/scores/enwiki/123456/drafttopic?features
{
  "error": {
    "code": "internal server error",
    "message": "Traceback (most recent call last):\n  File \"./ores/wsgi/routes/v3/util.py\", line 102, in process_score_request\n    return format_v3_score_response(score_response)\n  File \"./ores/wsgi/routes/v3/util.py\", line 65, in format_v3_score_response\n    return util.jsonify({response.context.name: context_doc})\n  File \"./ores/wsgi/util.py\", line 21, in jsonify\n    return flask_jsonify(normalize_json(doc))\n  File \"/srv/deployment/ores/deploy/venv/lib/python3.5/site-packages/flask_jsonpify.py\", line 77, in jsonpify\n    return current_app.response_class(__pad(__dumps(*args, **kwargs)),\n  File \"/srv/deployment/ores/deploy/venv/lib/python3.5/site-packages/flask_jsonpify.py\", line 30, in __dumps\n    indent=indent)\n  File \"/srv/deployment/ores/deploy/venv/lib/python3.5/site-packages/flask/json.py\", line 126, in dumps\n    rv = _json.dumps(obj, **kwargs)\n  File \"/usr/lib/python3.5/json/__init__.py\", line 237, in dumps\n    **kw).encode(obj)\n  File \"/usr/lib/python3.5/json/encoder.py\", line 200, in encode\n    chunks = list(chunks)\n  File \"/usr/lib/python3.5/json/encoder.py\", line 429, in _iterencode\n    yield from _iterencode_dict(o, _current_indent_level)\n  File \"/usr/lib/python3.5/json/encoder.py\", line 403, in _iterencode_dict\n    yield from chunks\n  File \"/usr/lib/python3.5/json/encoder.py\", line 403, in _iterencode_dict\n    yield from chunks\n  File \"/usr/lib/python3.5/json/encoder.py\", line 403, in _iterencode_dict\n    yield from chunks\n  File \"/usr/lib/python3.5/json/encoder.py\", line 403, in _iterencode_dict\n    yield from chunks\n  File \"/usr/lib/python3.5/json/encoder.py\", line 403, in _iterencode_dict\n    yield from chunks\n  File \"/usr/lib/python3.5/json/encoder.py\", line 403, in _iterencode_dict\n    yield from chunks\n  File \"/usr/lib/python3.5/json/encoder.py\", line 436, in _iterencode\n    o = _default(o)\n  File \"/srv/deployment/ores/deploy/venv/lib/python3.5/site-packages/flask/json.py\", line 83, in default\n    return _json.JSONEncoder.default(self, o)\n  File \"/usr/lib/python3.5/json/encoder.py\", line 179, in default\n    raise TypeError(repr(o) + \" is not JSON serializable\")\nTypeError: array([ 4.96419271e-03,  1.32506789e-01,  5.33425850e-02,  5.65699527e-02,\n       -4.18918677e-02, -4.70650656e-02,  1.36600294e-02, -1.15882606e-01,\n       -3.08872691e-02,  5.15907689e-02,  8.93780892e-02, -8.38178668e-02,\n        8.50824724e-03,  1.07053522e-01, -1.17251748e-01,  7.95875014e-02,\n       -2.15621413e-02,  6.34894120e-02, -5.04471628e-02, -1.58115320e-01,\n        1.57767430e-02, -2.66787880e-03, -3.34147403e-02,  8.47275048e-02,\n        4.91766679e-02, -4.79977256e-02, -9.77857489e-02,  3.64382560e-02,\n       -8.11660499e-04, -1.64268895e-02,  5.03347296e-02, -2.01244689e-02,\n       -1.87622873e-03, -2.32120982e-02,  3.47761188e-02, -5.96866273e-02,\n        2.45500531e-02,  1.13321940e-02,  7.37534908e-03, -4.53663876e-02,\n       -3.63962274e-03, -6.79642527e-02,  8.93124865e-02, -1.71882228e-02,\n       -1.15345737e-02, -7.14020311e-02, -8.78317314e-02,  3.14197206e-03,\n       -3.79146107e-02,  4.63090863e-02,  1.27461751e-02, -1.56924599e-02,\n       -1.31912231e-02,  9.26824202e-03,  3.04413511e-02, -5.00223260e-02,\n       -1.41794305e-02,  1.77395469e-02, -5.70303934e-02, -1.12757365e-01,\n       -8.30237405e-02,  6.61637156e-02, -3.08211477e-02, -4.61854098e-02,\n        3.22174608e-02,  7.70627741e-02, -7.16349284e-02,  4.04438220e-03,\n       -1.25804700e-02,  3.01800109e-02, -2.84568385e-02, -3.40511924e-04,\n        2.54492509e-02,  5.13444867e-02, -7.97932943e-02, -1.06829192e-01,\n        7.12529233e-02,  2.05977590e-02, -2.85888136e-03,  2.63404512e-02,\n        2.85055596e-02,  2.37799527e-02, -1.05091563e-02,  6.02470532e-02,\n       -4.29607159e-02, -1.12907544e-01, -9.88833779e-02,  1.29720052e-01,\n        4.54572711e-02, -3.41716566e-03,  1.14826403e-02, -3.38102642e-03,\n       -1.38458118e-02, -7.69016199e-02,  6.76515813e-02, -2.79628017e-02,\n        6.10999392e-02, -3.01144249e-02, -8.88222142e-03, -1.39020953e-02,\n        2.17927632e-02, -5.65694173e-02, -2.24365770e-02,  4.83802662e-02,\n        5.35690575e-03, -8.48742033e-02,  6.47082412e-02, -3.95904006e-02,\n        8.84792261e-02, -4.70736320e-02, -7.29562860e-02,  7.26254112e-02,\n       -9.92198743e-02,  4.25659983e-03,  7.97600997e-02,  2.51609401e-02,\n        2.52423203e-02, -3.18681148e-03,  6.69290643e-02,  3.06332237e-02,\n       -1.19734915e-01,  8.10589707e-03, -3.16119278e-02,  2.06534402e-02,\n        1.06934665e-02, -5.22118284e-03, -5.37754527e-02, -6.17472331e-03,\n        1.67846680e-02,  1.02078622e-01,  2.73777811e-02, -1.17814984e-01,\n       -8.82032963e-02, -1.36634592e-02,  1.51025203e-02, -6.47099813e-02,\n        2.85387541e-02, -5.45730591e-02, -6.44240798e-02,  7.04267401e-02,\n       -3.75320702e-02, -3.82904588e-02, -2.34289337e-03, -1.78091484e-02,\n       -5.02488655e-02, -2.64274196e-02, -2.64318366e-02, -4.23080712e-02,\n       -1.62530196e-02, -1.91307737e-02,  1.02815059e-01,  7.72970099e-02,\n       -4.59415703e-02, -7.70970395e-04, -2.65687641e-02, -6.39814410e-02,\n       -8.96852728e-02, -6.62663422e-02, -1.06286835e-02, -8.67143598e-02,\n        7.39198651e-02,  7.62377287e-02, -1.87302640e-02, -5.85734049e-02,\n       -3.28368137e-02, -1.00634190e-01,  2.57600483e-02, -6.11652575e-02,\n        7.27183024e-02, -3.91904597e-02, -2.31938948e-02, -7.77459396e-02,\n       -4.33590538e-02, -7.52702680e-02,  8.08057283e-02,  3.94479852e-02,\n        6.47939046e-02, -9.93729976e-02, -1.04073039e-01, -5.41566548e-02,\n       -7.94715212e-02, -9.64470579e-02,  3.27817683e-02,  5.56073105e-02,\n       -1.89780185e-02,  2.75032980e-03,  6.08916701e-02, -1.91149795e-03,\n        6.79208856e-02,  8.22773984e-02, -3.73529802e-02, -2.39263166e-02,\n        2.44483279e-02,  1.69832999e-02, -2.82889918e-02,  3.06970697e-02,\n        1.14904036e-02, -3.75858775e-02, -4.97479355e-02, -1.09246371e-01,\n        1.68257596e-02,  7.56290503e-02, -6.32377089e-02, -1.16330866e-02,\n       -4.41107666e-02, -2.50904853e-02, -9.67865659e-03, -1.32054379e-01,\n       -2.43771202e-02,  7.44200589e-05, -3.95925421e-02, -9.57984255e-03,\n       -2.94842636e-03, -4.62103057e-02, -1.39320775e-02,  5.91227214e-02,\n        1.32833916e-01,  2.73582057e-02, -1.28467493e-01, -6.46030359e-02,\n       -3.85273716e-02,  1.61507590e-02,  1.11779999e-02, -8.50958573e-03,\n        7.15002428e-02, -7.29928268e-02,  3.54946203e-02,  1.66390402e-02,\n        7.21987005e-02, -8.72053180e-02,  4.57499989e-02, -6.16492556e-02,\n       -7.01686792e-03,  2.73351837e-02,  4.53726785e-02,  1.76707485e-02,\n        2.37038596e-02, -1.23055441e-02,  1.87535872e-03, -3.73727899e-02,\n        2.36594217e-02,  2.32684487e-02, -1.60230001e-02, -9.11736739e-02,\n       -2.32613547e-02, -2.60202508e-04, -8.28849391e-02, -2.62344093e-02,\n       -3.99151518e-02,  1.04723479e-03, -9.36185603e-02,  6.07192725e-02,\n       -2.97771253e-02,  5.89604963e-02,  5.51479407e-02, -6.89697266e-03,\n        4.24456680e-02, -2.14634946e-02, -7.19728303e-02,  3.88322797e-03,\n        5.79601087e-02,  1.41984371e-02, -4.62999846e-02,  5.99108244e-03,\n        3.67204767e-02,  3.44864025e-02, -1.44773784e-02, -1.15856238e-01,\n       -1.12701951e-01,  2.61447304e-02,  1.33045933e-02,  8.67893152e-02,\n        7.56872746e-03,  1.41258909e-02,  6.03297719e-02, -5.26645727e-02,\n       -1.74860369e-02, -7.40727207e-02, -6.56888192e-02,  5.51570424e-02,\n       -7.51783137e-02,  4.44465771e-03,  7.99581963e-02,  9.06618353e-02,\n        1.75648070e-02, -1.88405556e-02, -4.60376405e-02,  3.39730246e-02,\n        1.42286334e-01,  3.19500304e-02, -8.53967499e-02, -6.34465803e-02,\n       -1.27141049e-01, -1.03317395e-02, -1.17444490e-02, -5.00349078e-02,\n       -1.03342157e-02, -2.66440542e-02, -1.59285696e-02,  2.77597528e-02]) is not JSON serializable\n"
  }
}
awight created this task.May 9 2018, 10:49 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 9 2018, 10:49 PM
Halfak added a subscriber: Halfak.May 11 2018, 8:26 PM

Oooh. I see what's going on here. We're going to need to find a reasonable way to render a big vector. In this case, I think we should render a FeatureVector with a placeholder with some details until we can think of something better.

I'm puzzled enough by the code that I should probably take this one just to put my head on straight. Having a hard time figuring out where features get rendered, it seems that we're storing them by raw value without the Feature container.

Right. I think we normally will normalize the vectors to be python float and in this case, they are coming out as a numpy float.

After trying a few insanely complicated things, I realized this is all we need: https://github.com/wiki-ai/revscoring/pull/401

awight claimed this task.Jun 4 2018, 11:26 AM

Mentioned in SAL (#wikimedia-cloud) [2018-06-04T15:44:49Z] <awight> ORES: Fix T194322

awight closed this task as Resolved.Jun 7 2018, 1:39 PM
Vvjjkkii renamed this task from Drafttopic breaks ?features parameter to h9caaaaaaa.Jul 1 2018, 1:10 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii triaged this task as High priority.
Vvjjkkii removed awight as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot assigned this task to awight.
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot closed this task as Resolved.
CommunityTechBot renamed this task from h9caaaaaaa to Drafttopic breaks ?features parameter.
CommunityTechBot added a subscriber: Aklapper.