Terminate Executor Using Threadpoolexecutor From Concurrent.futures Module
I am trying to terminate a ThreadPool based on values returned from long running request. I wish to terminate the ThreadPool once the sum of the request return values reaches MIN_R
Solution 1:
I changed the code around to append only futures with results if MIN_REQUIRED_VALUE not reached and loop through all pending futures and cancel them if MIN_REQUIRED_VALUE is reached.
You can notice I added num_requests to check number of requests submitted and it turns out to be exactly 6 in this case which is expected.
If anyone has a better way to do this would be good to see.
from concurrent.futures import ThreadPoolExecutor, as_completed
from time import sleep
NUM_REQUESTS = 1000
MIN_REQUIRED_VALUE = 50deflong_request(id):
sleep(1)
return {"data": {"value": 10}}
defcheck_results(results):
total = 0for result in results:
total += result["data"]["value"]
return total
defmain():
futures = []
responses = []
num_requests = 0with ThreadPoolExecutor(max_workers=10) as executor:
for request_index inrange(NUM_REQUESTS):
future = executor.submit(long_request, request_index)
# Future list
futures.append(future)
for future in as_completed(futures):
# --- Changed Logic Below ---
total = check_results(responses)
if total > MIN_REQUIRED_VALUE:
for pending_future in futures:
pending_future.cancel()
else:
num_requests += 1
responses.append(future.result())
return num_requests
if __name__ == "__main__":
requests = main()
print("Num Requests: ", requests)
Post a Comment for "Terminate Executor Using Threadpoolexecutor From Concurrent.futures Module"