The service is not reporting errors to logstash. Instead, it seems to be returning the logstash-formatted error back to the caller, as exemplified in this RESTBase log entry:
{ "_index": "logstash-2018.04.27", "_type": "restbase", "_id": "AWMGnwseh452DCx2bntT", "_version": 1, "_score": null, "_source": { "err_root_req_headers_content-length": "0", "err_levelPath": "error/events/emit", "line": "257", "pid": 177, "type": "restbase", "err_message": "400", "err_root_req_uri": "/www.wikidata.org/v1/page/html/Q52220064/670276832", "file": "/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/preq/index.js", "err_root_req_method": "get", "err_root_req_headers_x-request-class": "external", "host": "restbase1014", "@version": "1", "err_root_req_headers_user-agent": "CrossrefEventDataBot", "err_root_req_headers_x-client-ip": "52.214.31.76", "gelf_level": "3", "err_stack": "restbase: 400\n at Request.P.try.bind.then.err.retry.HTTPError.status (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/preq/index.js:257:23)\n at Request.tryCatcher (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/bluebird/js/release/util.js:16:23)\n at Promise._settlePromiseFromHandler (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/bluebird/js/release/promise.js:512:31)\n at Promise._settlePromise (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/bluebird/js/release/promise.js:569:18)\n at Promise._settlePromise0 (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/bluebird/js/release/promise.js:614:10)\n at Promise._settlePromises (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/bluebird/js/release/promise.js:693:18)\n at Promise._fulfill (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/bluebird/js/release/promise.js:638:18)\n at Request._callback (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/bluebird/js/release/nodeback.js:45:21)\n at Request.self.callback (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/request/request.js:186:22)\n at emitTwo (events.js:106:13)\n at Request.emit (events.js:191:7)\n at Request.<anonymous> (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/request/request.js:1163:10)\n at emitOne (events.js:96:13)\n at Request.emit (events.js:188:7)\n at IncomingMessage.<anonymous> (/srv/deployment/restbase/deploy-cache/revs/fbce5204d61e0eae55fc8ab72f1ccb3976d4a87d/node_modules/request/request.js:1085:12)\n at IncomingMessage.g (events.js:292:16)", "err_root_req_headers_x-forwarded-for": "52.214.31.76, 10.20.0.178, 10.20.0.178, 10.20.0.167, ::ffff:10.64.32.107", "source_host": "10.64.48.133", "level": "ERROR", "err_headers_server": "TornadoServer/4.4.2", "message": "400", "err_request_id": "a3a8e634-4a04-11e8-bb7c-03466a8b4324", "version": "1.0", "normalized_message": "400", "tags": [ "es", "gelf", "normalized_message_untrimmed" ], "err_body": "[{\"wiki\": \"\", \"recvFrom\": \"kafka1002.eqiad.wmnet\", \"uuid\": \"a629cbb84a0411e89896141877334a4e\", \"timestamp\": 1524824446, \"revision\": 14035058, \"event\": {\"message\": \"Failed sending event <Event a3cff637-4a04-11e8-b297-8741ff9b83e8 of schema (u'resource_change', 1)>. KafkaTimeoutError: \", \"code\": \"validation\", \"revision\": 1, \"rawEvent\": \"{\\\"meta\\\": {\\\"domain\\\": \\\"www.wikidata.org\\\", \\\"uri\\\": \\\"http://www.wikidata.org/api/rest_v1/page/html/Q52220064\\\", \\\"topic\\\": \\\"resource_change\\\", \\\"request_id\\\": \\\"a3a8e634-4a04-11e8-bb7c-03466a8b4324\\\", \\\"schema_uri\\\": \\\"resource_change/1\\\", \\\"dt\\\": \\\"2018-04-27T10:20:42.343Z\\\", \\\"id\\\": \\\"a3cff637-4a04-11e8-b297-8741ff9b83e8\\\"}, \\\"tags\\\": [\\\"restbase\\\"]}\", \"schema\": \"resource_change\"}, \"schema\": \"EventError\"}, {\"wiki\": \"\", \"recvFrom\": \"kafka1002.eqiad.wmnet\", \"uuid\": \"b812c7f84a0411e89896141877334a4e\", \"timestamp\": 1524824476, \"revision\": 14035058, \"event\": {\"message\": \"Failed sending event <Event a3cff638-4a04-11e8-b3bc-d43f8df4e0e1 of schema (u'resource_change', 1)>. KafkaTimeoutError: \", \"code\": \"validation\", \"revision\": 1, \"rawEvent\": \"{\\\"meta\\\": {\\\"domain\\\": \\\"www.wikidata.org\\\", \\\"uri\\\": \\\"http://www.wikidata.org/api/rest_v1/page/html/Q52220064/670276832\\\", \\\"topic\\\": \\\"resource_change\\\", \\\"request_id\\\": \\\"a3a8e634-4a04-11e8-bb7c-03466a8b4324\\\", \\\"schema_uri\\\": \\\"resource_change/1\\\", \\\"dt\\\": \\\"2018-04-27T10:20:42.343Z\\\", \\\"id\\\": \\\"a3cff638-4a04-11e8-b3bc-d43f8df4e0e1\\\"}, \\\"tags\\\": [\\\"restbase\\\"]}\", \"schema\": \"resource_change\"}, \"schema\": \"EventError\"}]", "@timestamp": "2018-04-27T10:21:16.361Z", "err_headers_content-type": "text/html; charset=UTF-8", "err_name": "restbase", "err_status": 400, "err_root_req_headers_x-request-id": "a3a8e634-4a04-11e8-bb7c-03466a8b4324", "err_headers_date": "Fri, 27 Apr 2018 10:20:42 GMT" } }
As a result, the only way to know what is going on with the service is to log onto the nodes and tail the log files which does not scale and represents a considerable loss of time in case of incidents.