all(imap_unordered()) bad trick
ActivePublic
Actions

Authored by hashar on Jun 7 2019, 9:29 PM.

Tags

Referenced Files

	F29379878: raw.txt
	Jun 7 2019, 9:29 PM

Subscribers

awight

	# For https://gerrit.wikimedia.org/r/#/c/integration/quibble/+/513515/3/quibble/test.py

	from time import sleep
	from multiprocessing import Pool
	from random import random

	SLEEPER_RETURN = None
	#SLEEPER_RETURN = True

	def sleeper(i):
	print("Starting %s" % i)
	sleep(random())
	print("Done %s" % i)
	return SLEEPER_RETURN

	tasks = [x for x in range(1, 9)]

	def parallel_run(tasks):
	with Pool(processes=2) as pool:
	return all(pool.imap_unordered(sleeper, tasks))

	parallel_run(tasks)
	print("end of script")

Event Timeline

When sleeper returns None, all() returns on the first sleeper that completes. Thus the output is:

Starting 1
Starting 2
Done 2
Starting 3
end of script

When sleeper returns True (uncomment line8: SLEEPER_RETURN = True):

Starting 1
Starting 2
Done 1
Starting 3
Done 3
Starting 4
Done 2
Starting 5
Done 4
Starting 6
Done 5
Starting 7
Done 6
Starting 8
Done 7
Done 8
end of script

all(imap_unordered()) bad trickActivePublicActions

Event Timeline

all(imap_unordered()) bad trick
ActivePublic
Actions