Cancelling ping_server task potentially kills _conn_lost_cb execution.

[galindus]

Hi there,

We were trying to implement automatic reconnection to STAN when the connection was lost, and ran into an issue with the conn_lost_cb.

If you have a couple of awaitable, like calling stan_client.connect, the callback stops executing. This is due to, the whole task that is responsible for invoking conn_lost_cb (_ping_server_task) gets cancelled during the clean up of the client.

Someone with more experience in the code-base and more knowledge of asyncio could surely improve the change. But this is sufficient to be able to implement a retry mechanism.

The fix we have introduced is a shield around _close_due_to_ping_failure to allow the callback to full finish its execution.

The issue can easily be reproduced, by introducing an awaitable call inside conn_lost_cb, that is long enough for task to register it has been cancelled:

async def conn_lost_cb(err):
  await asyncio.sleep(0.1, loop=self.loop)
  print("This message is often not shown")

Notice that this does not only affect _conn_lost_cb, but potentially be any of the clean up work in _close_due_to_ping_failure after the _ping_server_task cancellation.

Cheers