Page MenuHomePhabricator
Create Task
Maniphest T213582

ORES Python client drowning in TIME_WAIT sockets
Closed, ResolvedPublic
Actions

Description

A researcher using the ORES Python client with a parallelism of 20 is swamped by TCP sockets in the TIME_WAIT state, and very few "ESTABLISHED" connections. It's possible that we're not closing the socket properly.

I'll make sure we're using context managers to guarantee that resources are released regardless of exception or success.

Related Objects

Event Timeline

awight created this task.Jan 11 2019, 9:42 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 11 2019, 9:42 PM
awight mentioned this in T205909: ORES command line service sometimes hangs.Jan 11 2019, 9:43 PM
awight mentioned this in rORES5dd020439d13: Make sure the ORES client always closes sockets.Jan 11 2019, 10:15 PM
awight moved this task from Parked to Review on the Machine-Learning-Team (Active Tasks) board.Jan 12 2019, 12:22 AM

There were some cleanups to make on our side, and a synchronization issue with the research client. I've replaced the ORES consumer part of their pipeline with our standard score_revisions utility and will wait for feedback.

Halfak moved this task from Review to Completed on the Machine-Learning-Team (Active Tasks) board.Jan 14 2019, 5:37 PM
Ladsgroup closed this task as Resolved.Jan 23 2019, 3:11 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL