Page MenuHomePhabricator

ORES doesn't block hammering IPs
Closed, ResolvedPublic


From this:

ladsgroup@ores2001:~$ echo 'STATS FULL' | nc -w1 orespoolcounter2001.codfw.wmnet 7531
uptime: 13 days, 313h 30m 49s
total processing time: 2m 56.459259s
average processing time: 0.000618s
gained time: 0.000000s
waiting time: 0.007007s
waiting time for me: 0.007007s
waiting time for anyone: 0.000000s
waiting time for good: 0.000000s
wasted timeout time: 0.000000s
total_acquired: 285463
total_releases: 285463
hashtable_entries: 0
processing_workers: 0
waiting_workers: 0
connect_errors: 0
failed_sends: 0
full_queues: 0
lock_mismatch: 0
lock_while_waiting: 0
release_mismatch: 0
processed_count: 285463

ORES locks but nothing goes to "full_queues". I was able to run 15 threads at the same time to prod and nothing happened

Event Timeline

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptSep 19 2018, 6:41 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

This reason was that it just locks the IP and then immediately release it:

Change 461451 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/services/ores/deploy@master] Bump ORES to HEAD

Change 461451 merged by Ladsgroup:
[mediawiki/services/ores/deploy@master] Bump ORES to HEAD

Mentioned in SAL (#wikimedia-releng) [2018-09-19T18:56:56Z] <Amir1> ores:76fe25a goes to beta (T204862)

Mentioned in SAL (#wikimedia-operations) [2018-09-19T20:05:14Z] <ladsgroup@deploy1001> Started deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862)

Mentioned in SAL (#wikimedia-operations) [2018-09-19T20:26:27Z] <ladsgroup@deploy1001> Finished deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862) (duration: 21m 13s)

Ladsgroup moved this task from Incoming to Done on the User-Ladsgroup board.Oct 2 2018, 8:19 PM
Ladsgroup closed this task as Resolved.Nov 1 2018, 8:00 PM