Bolt threads not properly closing after graph.run or graph.execute #106
Description
I have an async program that runs graph.execute and graph.run calls. I am using a semaphore to limit the number of concurrent tasks at once. My neo4j server is set to have 100x the number of concurrent tasks for its thread pool maximum and I am setting my max_connections for the ConfigBuilder at 10x the number of concurrent tasks. When I run a graph.execute or graph.run I make sure to await on them, so each task should only have one live connection at a time.
Despite this I very quickly hit AuthenticationError("There are no available threads to serve this request at the moment. You can retry at a later time or consider increasing max thread pool size for bolt connector(s).") or UnexpectedMessage("unexpected response for RUN: Ok(Failure(Failure { metadata: BoltMap { value: {BoltString { value: \"code\" }: String(BoltString { value: \"Neo.TransientError.Request.NoThreadsAvailable\" }), BoltString { value: \"message\" }: String(BoltString { value: \"There are no available threads to serve this request at the moment. You can retry at a later time or consider increasing max thread pool size for bolt connector(s).\" })} } }))"). The higher my number of concurrent tasks the faster I will hit these errors, this leads me to believe that there is some lag time with closing the connections despite the await statements. At 1024 concurrent tasks it takes about 15 seconds, at 512 concurrent tasks it occurs within a couple of minutes.