Page MenuHomePhabricator
Create Task
Maniphest T154168

Quantity changes broke ORES
Closed, ResolvedPublic
Actions

Description

Form https://ores.wikimedia.org/v2/scores/wikidatawiki/?models=damaging&revids=421063984:

{
  "scores": {
    "wikidatawiki": {
      "damaging": {
        "scores": {
          "421063984": {
            "error": {
              "message": "KeyError: Failed to process datasource.wikibase.revision.item: 'upperBound'\nTraceback (most recent call last):\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/revscoring/dependencies/functions.py\", line 244, in _solve\n    value = dependent(*args)\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/revscoring/dependencies/dependent.py\", line 52, in __call__\n    return self.process(*args, **kwargs)\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/revscoring/features/wikibase/datasources/revision_oriented.py\", line 115, in _process_item\n    item.get(content=item_doc or {'aliases': {}})\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/pywikibase/itempage.py\", line 50, in get\n    data = super(ItemPage, self).get(*args, **kwargs)\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/pywikibase/wikibasepage.py\", line 86, in get\n    c = Claim.fromJSON(claim)\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/pywikibase/claim.py\", line 95, in fromJSON\n    claim.type, lambda value: value)(value)\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/pywikibase/claim.py\", line 38, in <lambda>\n    'quantity': lambda value: WbQuantity.fromWikibase(value),\n  File \"/srv/deployment/ores/venv/lib/python3.4/site-packages/pywikibase/wbquantity.py\", line 62, in fromWikibase\n    upperBound = eval(wb['upperBound'])\nKeyError: 'upperBound'\n",
              "type": "CaughtDependencyError"
            }
          }
        },
        "version": "0.1.2"
      }
    }
  }
}

Details

SubjectRepoBranchLines +/-
Update wheelsmediawiki/services/ores/deploymaster+1 -1
Change pywikibase wheel to 0.0.3 and use a newer versionresearch/ores/wheelsmaster+0 -0
Update quantity with the recent changes in Wikibasepywikibot/wikibasemaster+83 -32
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT56318 Quantity datatype (tracking)
 OpenNoneT133042 Quantity datatype precision (tracking)
 ResolvedLadsgroupT154168 Quantity changes broke ORES

Event Timeline

Ladsgroup created this task.Dec 27 2016, 12:16 AM
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptDec 27 2016, 12:16 AM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald Transcript
Ladsgroup triaged this task as Unbreak Now! priority.Dec 27 2016, 12:20 AM
Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald TranscriptDec 27 2016, 12:20 AM
gerritbot subscribed.Dec 27 2016, 12:32 AM

Change 329310 had a related patch set uploaded (by Ladsgroup):
Update quantity with the recent changes in Wikibase

https://gerrit.wikimedia.org/r/329310

gerritbot added a project: Patch-For-Review.Dec 27 2016, 12:32 AM
Ladsgroup added a comment.Dec 27 2016, 12:53 AM

^ will fix this. Tested in ores-experiment. http://ores-experiment.wmflabs.org/v2/scores/wikidatawiki/?models=damaging&revids=421063984

gerritbot added a comment.Dec 27 2016, 12:54 AM

Change 329310 merged by jenkins-bot:
Update quantity with the recent changes in Wikibase

https://gerrit.wikimedia.org/r/329310

gerritbot added a comment.Dec 27 2016, 1:25 AM

Change 329312 had a related patch set uploaded (by Ladsgroup):
Change pywikibase wheel to 0.0.3 and use a newer version

https://gerrit.wikimedia.org/r/329312

gerritbot added a comment.Dec 27 2016, 1:26 AM

Change 329312 merged by Ladsgroup:
Change pywikibase wheel to 0.0.3 and use a newer version

https://gerrit.wikimedia.org/r/329312

Ladsgroup added a comment.Dec 27 2016, 1:31 AM

Okay, It seems there are some undeployed changes to wheels repo, I need to deploy all (like revscoring, etc.) or none. That's complex. I can revert other wheels patches and re-revert when the deployment is done but I really don't like it :(

gerritbot added a comment.Dec 27 2016, 1:45 AM

Change 329315 had a related patch set uploaded (by Ladsgroup):
Update wheels

https://gerrit.wikimedia.org/r/329315

gerritbot added a comment.Dec 27 2016, 1:47 AM

Change 329315 merged by Ladsgroup:
Update wheels

https://gerrit.wikimedia.org/r/329315

Ladsgroup added a comment.Dec 27 2016, 2:13 AM

Now, https://phabricator.wikimedia.org/diffusion/1915/repository/master/ is stuck :/

Ladsgroup mentioned this in T154172: phabricator is slow to update mirrors.Dec 27 2016, 3:56 AM
Ladsgroup mentioned this in rORESWHEELS63fe882cb7d6: Change pywikibase wheel to 0.0.3 and use a newer version.Dec 27 2016, 4:57 AM
Stashbot subscribed.Dec 27 2016, 5:00 AM

Mentioned in SAL (#wikimedia-releng) [2016-12-27T05:00:13Z] <Amir1> deploying 5230e7d in ores beta node (T154168)

Ladsgroup mentioned this in rORESDEPLOY228b9b4ff925: Update wheels.Dec 27 2016, 5:01 AM
Ladsgroup added a comment.Dec 27 2016, 5:03 AM

Deployed in beta, got it fixed:

{
  "scores": {
    "fawiki": {
      "damaging": {
        "scores": {
          "56789": {
            "prediction": false,
            "probability": {
              "false": 0.9956156179791043,
              "true": 0.004384382020895765
            }
          }
        },
        "version": "0.1.2"
      }
    }
  }
}

Going to prod...

Stashbot added a comment.Dec 27 2016, 5:06 AM

Mentioned in SAL (#wikimedia-operations) [2016-12-27T05:06:06Z] <Amir1> starting deploy of ores:228b9b4 in canary nodes (T154168)

Stashbot added a comment.Dec 27 2016, 5:14 AM

Mentioned in SAL (#wikimedia-operations) [2016-12-27T05:14:37Z] <Amir1> starting deploy of ores:228b9b4 in all nodes (T154168)

Stashbot added a comment.Dec 27 2016, 5:25 AM

Mentioned in SAL (#wikimedia-operations) [2016-12-27T05:25:27Z] <Amir1> finished deploy of ores:228b9b4 in all nodes (T154168)

Ladsgroup added a comment.Dec 27 2016, 5:26 AM

And now everything is fine, I'll monitor #wikimedia-operations channel in IRC for any issues and keep an eye on failed job ratio too.

Ladsgroup closed this task as Resolved.Dec 27 2016, 5:26 AM
Ladsgroup moved this task from Proposed to Done on the Wikidata-Former-Sprint-Board board.
Ladsgroup moved this task from Parked to Completed on the Machine-Learning-Team (Active Tasks) board.
Stashbot added a comment.Dec 27 2016, 5:44 AM

Mentioned in SAL (#wikimedia-operations) [2016-12-27T05:44:10Z] <Amir1> ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/PopulateDatabase.php --wiki=wikidatawiki (T154168)

Ladsgroup added a parent task: T133042: Quantity datatype precision (tracking).Dec 27 2016, 5:54 AM
Ladsgroup added a comment.Dec 27 2016, 6:15 AM

Incident report: https://wikitech.wikimedia.org/wiki/Incident_documentation/20161227-ores

Ladsgroup mentioned this in rORESWHEELS13d356e48a86: Change pywikibase wheel to 0.0.3 and use a newer version Bug: T154168 Change-Id….Nov 7 2018, 6:34 PM
Ladsgroup mentioned this in rORESWHEELS5ef56513d7eb: Change pywikibase wheel to 0.0.3 and use a newer version Bug: T154168 Change-Id….Nov 7 2018, 7:19 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL