Maniphest T204862

ORES doesn't block hammering IPs
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Ladsgroup
	Sep 19 2018, 6:41 PM

Description

From this:

ladsgroup@ores2001:~$ echo 'STATS FULL' | nc -w1 orespoolcounter2001.codfw.wmnet 7531
uptime: 13 days, 313h 30m 49s
total processing time: 2m 56.459259s
average processing time: 0.000618s
gained time: 0.000000s
waiting time: 0.007007s
waiting time for me: 0.007007s
waiting time for anyone: 0.000000s
waiting time for good: 0.000000s
wasted timeout time: 0.000000s
total_acquired: 285463
total_releases: 285463
hashtable_entries: 0
processing_workers: 0
waiting_workers: 0
connect_errors: 0
failed_sends: 0
full_queues: 0
lock_mismatch: 0
lock_while_waiting: 0
release_mismatch: 0
processed_count: 285463

ORES locks but nothing goes to "full_queues". I was able to run 15 threads at the same time to prod and nothing happened

Details

	Subject	Repo	Branch	Lines +/-
	Bump ORES to HEAD	mediawiki/services/ores/deploy	master	+1 -1

Customize query in gerrit

Related Objects

Mentioned In: rORESDEPLOY76fe25a346a7: Bump ORES to HEAD
Mentioned Here: rORESDEPLOY76fe25a346a7: Bump ORES to HEAD

Event Timeline

Ladsgroup created this task.Sep 19 2018, 6:41 PM

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptSep 19 2018, 6:41 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

This reason was that it just locks the IP and then immediately release it: https://github.com/wikimedia/ores/pull/262

Ladsgroup moved this task from Parked to Pending deployment on the Machine-Learning-Team (Active Tasks) board.Sep 19 2018, 6:53 PM

Change 461451 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/services/ores/deploy@master] Bump ORES to HEAD

https://gerrit.wikimedia.org/r/461451

gerritbot added a project: Patch-For-Review.Sep 19 2018, 6:54 PM

Ladsgroup mentioned this in rORESDEPLOY76fe25a346a7: Bump ORES to HEAD.Sep 19 2018, 6:54 PM

Change 461451 merged by Ladsgroup:
[mediawiki/services/ores/deploy@master] Bump ORES to HEAD

https://gerrit.wikimedia.org/r/461451

Mentioned in SAL (#wikimedia-releng) [2018-09-19T18:56:56Z] <Amir1> ores:76fe25a goes to beta (T204862)

Mentioned in SAL (#wikimedia-operations) [2018-09-19T20:05:14Z] <ladsgroup@deploy1001> Started deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862)

Mentioned in SAL (#wikimedia-operations) [2018-09-19T20:26:27Z] <ladsgroup@deploy1001> Finished deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862) (duration: 21m 13s)

Ladsgroup moved this task from Pending deployment to Completed on the Machine-Learning-Team (Active Tasks) board.Sep 19 2018, 8:36 PM

Ladsgroup moved this task from Incoming to Done on the User-Ladsgroup board.Oct 2 2018, 8:19 PM

Ladsgroup closed this task as Resolved.Nov 1 2018, 8:00 PM

ORES doesn't block hammering IPsClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

ORES doesn't block hammering IPs
Closed, ResolvedPublic
Actions