Page MenuHomePhabricator

KafkaSSE: Cannot write SSE event, the response is already finished
Closed, ResolvedPublic

Description

Might be nothing, but noticed it browsing in Kibana:

{
  "name": "Error",
  "stack": "Error: Cannot write SSE event: the response is already finished.\n    at KafkaSSE._error (/srv/service/node_modules/kafka-sse/lib/KafkaSSE.js:835:17)\n    at _start.catch (/srv/service/node_modules/kafka-sse/lib/KafkaSSE.js:335:26)\n    at tryCatcher (/srv/service/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/srv/service/node_modules/bluebird/js/release/promise.js:547:31)\n    at Promise._settlePromise (/srv/service/node_modules/bluebird/js/release/promise.js:604:18)\n    at Promise._settlePromise0 (/srv/service/node_modules/bluebird/js/release/promise.js:649:10)\n    at Promise._settlePromises (/srv/service/node_modules/bluebird/js/release/promise.js:725:18)\n    at _drainQueueStep (/srv/service/node_modules/bluebird/js/release/async.js:93:12)\n    at _drainQueue (/srv/service/node_modules/bluebird/js/release/async.js:86:9)\n    at Async._drainQueues (/srv/service/node_modules/bluebird/js/release/async.js:102:5)\n    at Immediate.Async.drainQueues [as _onImmediate] (/srv/service/node_modules/bluebird/js/release/async.js:15:14)\n    at runCallback (timers.js:705:18)\n    at tryOnImmediate (timers.js:676:5)\n    at processImmediate (timers.js:658:5)",
  "message": "Cannot write SSE event: the response is already finished."
}
hostnameeventstreams-production-6fd7b8788c-267l4
k8s.hostkubernetes1007.eqiad.wmnet
k8s.labels.appeventstreams
docker.container_idcb9bd94e87105e6d0b626d24351a9c6e1dd72e5501fdf9129dfe591b98e1895b

Document link

Event Timeline

I am pretty sure this is normal; this happens when the HTTP connection is closed (either by the client or the server timeout), but the KafkaSSE consume loop had a pending message to send to the client.

Milimetric triaged this task as High priority.
Milimetric moved this task from Incoming to Ops Week on the Analytics board.
Milimetric added a project: Analytics-Kanban.
Milimetric subscribed.

Maybe we should change the level of this to warn.

Change 626201 had a related patch set uploaded (by Milimetric; owner: Milimetric):
[mediawiki/services/eventstreams@master] Update KafkaSSE dependency

https://gerrit.wikimedia.org/r/626201

Change 626201 merged by Milimetric:
[mediawiki/services/eventstreams@master] Update KafkaSSE dependency

https://gerrit.wikimedia.org/r/626201

Change 626223 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventstreams - bump to image version 2020-09-09-201733-production

https://gerrit.wikimedia.org/r/626223

(we decided to deploy this next week, and we can do so by simply merging the change above (https://gerrit.wikimedia.org/r/626223))

Change 626223 merged by Milimetric:
[operations/deployment-charts@master] eventstreams - bump to image version 2020-09-09-201733-production

https://gerrit.wikimedia.org/r/626223