KafkaSSE tests work locally, but don't run well in Jenkins/Travis because a Kafka broker is spawned.
Goal is to test integration with Kafka client node-rdkafka
We want to identify changes to Kafka SSE that break integration with node-rdkafka
[nuria] jenkins spwning a broker might not be the best way. Can we provision a docker environment with all that is needed and use it locally w/o involving jenkins?
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Declined | Xqt | T125197 Give hint to the current socketIO_client in ImportError of rcstream.py | |||
| Declined | None | T91393 RCStream is not accessible from python client due to using socket-io 1.0 while only socket-io 0.9 is offered | |||
| Declined | None | T68232 Upgrade RCStream backend to use socket.io 1.0 protocol | |||
| Resolved | Ottomata | T130651 EventStreams | |||
| Resolved | JAllemandou | T150436 Improve tests for KafkaSSE |
Several Options:
Or:
I am merely brain dumping my understanding for this task:
node-rdkafka are nodejs bindings for librdkafka. It can be compiled either against the system package or a source package. The canonical repository is https://github.com/wikimedia/node-rdkafka
It seems KafkaSSE canonical repository is on Differential https://phabricator.wikimedia.org/diffusion/WKSE/
There is some discussion going on T151469 to eventually get Kafka to spawn in jobs and rely on it to run integration tests.
For KafkaSSE, I guess you would install node-rdkafka from npmjs.org, start Kafka and run tests against it. If one want to test against versions that runs on production that would be a different Job which would have to fetch the version of node-rdkafka currently deployed and compiled against system package. Then Nuria pointed that last part would be better suited for a staging environment.
I just landed Joseph's docker KafkaSSE test patch. We could make some improvements to it to make it better for running tests during developments, but it should be good as is for Jenkins...I think.