There seems to be a large number of API calls to wikidata creating a high number of deadlocks like this (450/hour):
{
"_index": "logstash-2015.09.04",
"_type": "mediawiki",
"_id": "AU-ZFLBnIkWKhegMyoko",
"_score": null,
"_source": {
"message": "Wikibase\\Repo\\Store\\SQL\\EntityPerPageTable::{closure}\t10.64.32.28\t1213\tDeadlock found when trying to get lock; try restarting transaction (10.64.32.28)\tINSERT INTO `wb_entity_per_page` (epp_entity_id,epp_entity_type,epp_page_id,epp_redirect_target) VALUES ('16507538','item','22922124',NULL)",
"@version": 1,
"@timestamp": "2015-09-04T15:58:33.340Z",
"type": "mediawiki",
"host": "mw1221",
"level": "ERROR",
"tags": [
"syslog",
"es",
"es",
"normalized_message_trimmed"
],
"channel": "wfLogDBError",
"url": "/w/api.php",
"ip": "10.64.32.107",
"http_method": "POST",
"server": "www.wikidata.org",
"referrer": null,
"uid": "4943efb",
"process_id": 21645,
"wiki": "wikidatawiki",
"db_server": "10.64.32.28",
"db_name": "wikidatawiki",
"db_user": "wikiuser",
"method": "DatabaseBase::reportQueryError",
"errno": 1213,
"error": "Deadlock found when trying to get lock; try restarting transaction (10.64.32.28)",
"sql1line": "INSERT INTO `wb_entity_per_page` (epp_entity_id,epp_entity_type,epp_page_id,epp_redirect_target) VALUES ('16507538','item','22922124',NULL)",
"fname": "Wikibase\\Repo\\Store\\SQL\\EntityPerPageTable::{closure}",
"normalized_message": "Wikibase\\Repo\\Store\\SQL\\EntityPerPageTable::{closure}\t10.64.32.28\t1213\tDeadlock found when trying to get lock; try restarting transaction (10.64.32.28)\tINSERT INTO `wb_entity_per_page` (epp_entity_id,epp_entity_type,epp_page_id,epp_redirect_target) VALUE"
},
"sort": [
1441382313340
]}
They started on Friday 4 Sept at around 10:50 and continues 5 hours later. They are not causing issues on the infrastructure, but they seem to be interfering with themselves. We need to identify the cause: if it is legitimate traffic, change how the inserts are done; if it is abnormal traffic, maybe avoid it in the first place.