Page MenuHomePhabricator

Graphoid "No graph found." failures - 11 Dec 2019
Closed, ResolvedPublic

Description

Graphoid started alerting at 06:42 UTC

I don't see a higher error rate on https://logstash.wikimedia.org/goto/4ff40949e4939b48686eb94e2c72ae0f, but I do see an increase in mwapi errors in https://grafana.wikimedia.org/d/000000021/service-graphoid?orgId=1&from=now-3h&to=now-4m

Restarting graphoid on scb1001 didn't have any affect

Icinga test failing:

{

"name": "graphoid",
"hostname": "scb1001",
"pid": 252,
"level": 30,
"domain": "mediawiki.org",
"format": "png",
"title": "Extension:Graph/Demo",
"revid": "0",
"id": "2e25518b199b22ab9043f7ce9a0cd1370b27d77a",
"apicall": {
  "format": "json",
  "formatversion": "2",
  "action": "graph",
  "title": "Extension:Graph/Demo",
  "hash": "2e25518b199b22ab9043f7ce9a0cd1370b27d77a"
},
"apiRetError": {
  "code": "invalidhash",
  "info": "No graph found.",
  "docref": "See https://www.mediawiki.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes."
},
"levelPath": "info/mwapi-error",
"request_id": "2b6ccc86-1be6-11ea-9005-d5a835bbce52",
"request": {
  "url": "/mediawiki.org/v1/png/Extension%3AGraph%2FDemo/0/2e25518b199b22ab9043f7ce9a0cd1370b27d77a",
  "headers": {
    "user-agent": "ServiceChecker-WMF/0.1.2",
    "x-request-id": "2b6ccc86-1be6-11ea-9005-d5a835bbce52"
  },
  "method": "GET",
  "params": {
    "0": "/mediawiki.org/v1/png/Extension:Graph/Demo/0/2e25518b199b22ab9043f7ce9a0cd1370b27d77a"
  },
  "query": {},
  "remoteAddress": "208.80.154.84",
  "remotePort": 38690
},
"msg": "Message not supplied",
"time": "2019-12-11T07:16:37.395Z",
"v": 0

}

{

"name": "graphoid",
"hostname": "scb1001",
"pid": 222,
"level": 30,
"domain": "ru.wikipedia.org",
"format": "png",
"title": "Фитнес",
"revid": "0",
"id": "5d34a4b83fa0ffc21c0549858e11ee58864945aa.png",
"apicall": {
  "format": "json",
  "formatversion": "2",
  "action": "graph",
  "title": "Фитнес",
  "hash": "5d34a4b83fa0ffc21c0549858e11ee58864945aa"
},
"apiRetError": {
  "code": "invalidhash",
  "info": "No graph found.",
  "docref": "See https://ru.wikipedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes."
},
"levelPath": "info/mwapi-error",
"request_id": "730865e5-e724-4837-8e2a-87f521672dbc",
"request": {
  "url": "/ru.wikipedia.org/v1/png/%D0%A4%D0%B8%D1%82%D0%BD%D0%B5%D1%81/0/5d34a4b83fa0ffc21c0549858e11ee58864945aa.png",
  "headers": {
    "x-request-id": "730865e5-e724-4837-8e2a-87f521672dbc",
    "content-length": "0"
  },
  "method": "GET",
  "params": {
    "0": "/ru.wikipedia.org/v1/png/Фитнес/0/5d34a4b83fa0ffc21c0549858e11ee58864945aa.png"
  },
  "query": {},
  "remoteAddress": "10.64.0.104",
  "remotePort": 46476
},
"msg": "Message not supplied",
"time": "2019-12-11T07:13:27.495Z",
"v": 0

}

Event Timeline

jijiki created this task.Dec 11 2019, 7:37 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 11 2019, 7:37 AM
jijiki triaged this task as Unbreak Now! priority.Dec 11 2019, 7:38 AM
jijiki updated the task description. (Show Details)
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptDec 11 2019, 7:38 AM
jijiki updated the task description. (Show Details)Dec 11 2019, 7:41 AM
akosiaris closed this task as Resolved.Dec 11 2019, 8:17 AM
akosiaris claimed this task.
akosiaris added a subscriber: akosiaris.

Graphoid is to be undeployed mid of next quarter. With that in mind, the alerts were because some one edited the monitored page. I 've reverted the changes with an appropriate message and everything has recovered. https://www.mediawiki.org/w/index.php?title=Extension:Graph/Demo&action=history