Page MenuHomePhabricator
Create Task
Maniphest T91393

RCStream is not accessible from python client due to using socket-io 1.0 while only socket-io 0.9 is offered
Closed, DeclinedPublic
Actions

Description

INFO:urllib3.connectionpool:Starting new HTTP connection (1): stream.wikimedia.org
DEBUG:urllib3.connectionpool:"GET /socket.io/?transport=polling&EIO=3&t=1425388662485-0 HTTP/1.1" 404 13
WARNING:root:stream.wikimedia.org:80/socket.io [waiting for connection] unexpected status code (404)
DEBUG:urllib3.connectionpool:"GET /socket.io/?transport=polling&EIO=3&t=1425388663760-0 HTTP/1.1" 404 13
DEBUG:urllib3.connectionpool:"GET /socket.io/?transport=polling&EIO=3&t=1425388664889-0 HTTP/1.1" 404 13
DEBUG:urllib3.connectionpool:"GET /socket.io/?transport=polling&EIO=3&t=1425388666018-0 HTTP/1.1" 404 13
DEBUG:urllib3.connectionpool:"GET /socket.io/?transport=polling&EIO=3&t=1425388667145-0 HTTP/1.1" 404 13
DEBUG:urllib3.connectionpool:"GET /socket.io/?transport=polling&EIO=3&t=1425388668273-0 HTTP/1.1" 404 13
DEBUG:urllib3.connectionpool:"GET /socket.io/?transport=polling&EIO=3&t=1425388669399-0 HTTP/1.1" 404 13

neverending

Related Objects
Search...

StatusSubtypeAssignedTask
 DeclinedXqtT125197 Give hint to the current socketIO_client in ImportError of rcstream.py
 DeclinedNoneT91393 RCStream is not accessible from python client due to using socket-io 1.0 while only socket-io 0.9 is offered
 DeclinedNoneT68232 Upgrade RCStream backend to use socket.io 1.0 protocol
 ResolvedKrinkleT86803 Rewrite RCStream in Node.js
 DeclinedKrinkleT121722 Deploy node-rcstream
 ResolvedOttomataT130651 EventStreams
 Duplicate• PcheloloT137379 Replace kafka-node with more mature bindings, ideally using librdkafka
 Resolved• PcheloloT140284 Develop a node kafka driver based on librdkafka
 ResolvedOttomataT145095 Kasocki Prototype
 Resolved• NuriaT145099 Write node-rdkafka event.stats callback that reports stats to statsd
 ResolvedOttomataT145140 Run Kasocki tests in Jenkins via Differential commits
 ResolvedOttomataT148470 Kafka SSE Prototype
 ResolvedOttomataT148651 Create KafkaSSE diffusion repository
 ResolvedOttomataT148779 Prepare eventstreams (with KafkaSSE) for deployment
 ResolvedOttomataT143925 Productionize and deploy Public EventStreams
 DeclinedOttomataT145805 Support node cluster sticky-session in service-runner
 DeclinedOttomataT148043 Fix consumer.disconnect() node-rdkafka bug
 ResolvedOttomataT149736 Bikeshed what events should be exposed in public EventStreams API
 ResolvedOttomataT167670 Expose revision-create in EventStreams
 ResolvedOttomataT167180 Emit revision-score event to EventBus and expose in EventStreams
 ResolvedHalfakT172566 Include label-specific schemas with model_info
 ResolvedJAllemandouT150436 Improve tests for KafkaSSE
 DeclinedNoneT150439 Tests for swagger spec stream routes in EventStreams
 ResolvedOttomataT152030 RecentChanges in Kafka
 OpenNoneT152731 Implement server side filtering for EventStreams (if we should)
 ResolvedOttomataT153117 EventStreams documentation
 ResolvedmfornsT154328 Set charset=utf-8 in Content-Type response header from sse.js client
 ResolvedOttomataT156919 Port RCStream clients to EventStreams
 ResolvedXqtT158943 Implement EventStreams client for pywikibot
 DuplicateNoneT160997 Migrate pywikibot from RCStream to EventStreams
 Resolved• CmjohnsonT170157 decommission rcs100[12]
 ResolvedXqtT170534 Decommission RCStream in pywikibot framework
 ResolvedOttomataT157492 Report number of stream connections to statsd
 ResolvedOttomataT158106 Update EventBus RCFeed config to use newly refactored settings
 ResolvedOttomataT158066 Create EventStreams swagger spec docs endpoint
 ResolvedOttomataT159179 Create /schema/:schema endpoint in eventbus service to serve schemas by schema_uri
 ResolvedOttomataT160030 old revision length missing in /v2/stream/recentchange
 ResolvedOttomataT160080 EventStreams Blog Post
 DeclinedOttomataT160748 Create /v2/schema/:schema_uri endpoint for eventstreams that proxies schemas from eventbus
 DeclinedOttomataT161027 Puppetize event schema topic configuration
 ResolvedOttomataT172356 Decom RCStream in Beta Cluster

Event Timeline

Petrb created this task.Mar 3 2015, 1:18 PM
Petrb raised the priority of this task from to Needs Triage.
Petrb updated the task description. (Show Details)
Petrb added a project: EventStreams.
Petrb subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 3 2015, 1:18 PM
Petrb updated the task description. (Show Details)EditedMar 3 2015, 1:18 PM
Petrb set Security to None.
Sample code:

#!/usr/bin/python
import logging
import socketIO_client
import json

logging.basicConfig(level='DEBUG')

class WikiNamespace(socketIO_client.BaseNamespace):
    def on_reconnect(self, *args):
        self.emit('subscribe', '*')
    def on_connect(self):
        self.emit('subscribe', '*')


socketIO = socketIO_client.SocketIO('stream.wikimedia.org', 80)
socketIO.define(WikiNamespace, '/rc')

socketIO.wait()
Petrb added a comment.Mar 3 2015, 1:20 PM
This comment was removed by Petrb.
Petrb triaged this task as Unbreak Now! priority.Mar 3 2015, 1:27 PM
Krinkle subscribed.Mar 3 2015, 3:49 PM

The 404 errors look like a client trying to communicate using the socket-io 1.0 protocol. RCStream (at the moment) uses the more widespread socket-io 0.9 protocol. See T68232. If using a popular library, you may have to install an older version of it. Or use an alternate built-in (e.g. one of the php libraries out there ships a Version1 and Version09 client class).

Aklapper renamed this task from RCStream is not accessible from python to RCStream is not accessible from python client due to using socket-io 1.0 while only socket-io 0.9 is offered.Mar 6 2015, 10:25 AM
Aklapper lowered the priority of this task from Unbreak Now! to Medium.
XZise mentioned this in T85716: LiveRCPageGenerator socketIO_client upstream issues.Mar 10 2015, 11:08 AM
XZise subscribed.

I can confirm that. The entry on pypi mentions that the latest version (0.6.1 as of this comment) supports socket.io 1.0. I installed 0.5.5 and I don't get the 404 error.

XZise mentioned this in rPWBC7412dbfb4946: [FIX] socketIO: Use older version to support WMF.Mar 10 2015, 11:17 AM
XZise mentioned this in T85717: LiveRCPageGenerator using socketIO_client stall the builds regularly.Mar 12 2015, 5:34 PM
Krinkle moved this task from Backlog to Backlog on the EventStreams board.Jul 23 2015, 7:30 PM
Krinkle added a subtask: T68232: Upgrade RCStream backend to use socket.io 1.0 protocol.Jul 23 2015, 8:26 PM
Ricordisamoa subscribed.Jul 23 2015, 9:51 PM
jayvdb added a project: Pywikibot.Oct 15 2015, 11:50 AM
jayvdb subscribed.

This is a pretty high risk for Python apps. v0.5.5 was released Feb 17 , and we're stuck on that version.
Since then there have been fixes , and we cant update the library to get those or future fixes.

Restricted Application added a subscriber: pywikibot-bugs-list. · View Herald TranscriptOct 15 2015, 11:50 AM
Petrb added a comment.Oct 15 2015, 2:16 PM

Since we have whole server (that means 1 dedicated public IP) for this service, why not just running a newer version on a different port so that it still works with software that already use it?

jayvdb moved this task from Backlog to Upstream issues on the Pywikibot board.Oct 16 2015, 7:06 AM
jayvdb added a comment.Oct 17 2015, 12:17 PM
In T91393#1727795, @jayvdb wrote:

This is a pretty high risk for Python apps. v0.5.5 was released Feb 17 , and we're stuck on that version.
Since then there have been fixes , and we cant update the library to get those or future fixes.

Case in point:
https://github.com/invisibleroads/socketIO-client/issues/95

Afandian subscribed.Oct 26 2015, 4:38 PM
Krinkle changed the status of subtask T68232: Upgrade RCStream backend to use socket.io 1.0 protocol from Open to Stalled.Dec 17 2015, 12:24 AM
Xqt added a parent task: T125197: Give hint to the current socketIO_client in ImportError of rcstream.py.Jan 29 2016, 9:42 AM
Krinkle closed this task as Declined.Jan 30 2017, 10:52 PM

I'm closing this task and T68232 since it's unlikely this will happen. Rather than upgrading RCSteam to Socket.IO v1.0 (which would be a breaking change for existing RCStream consumers), I'll instead help push forward T130651: EventStreams.

Krinkle mentioned this in T68232: Upgrade RCStream backend to use socket.io 1.0 protocol.Jan 30 2017, 10:52 PM
Krinkle closed subtask T68232: Upgrade RCStream backend to use socket.io 1.0 protocol as Declined.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL