Add deadline to break the connection. We use the default of 5 seconds.
After this the backend is marked unhealthy and not used for some time.

Fixes #467