plugin/forward: TCP conns can be closed (#1651)

* plugin/forward: TCP conns can be closed

Only when we read and get a io.EOF we know the conn is closed (for TCP).
If this is the case Dial (again) and retry. Note that this new
connection can also be closed by the upstream, we may want to add a
DialForceNew or something to get a new TCP connection..

Simular to #1624, *but* this is by (TCP) design. We also don't have to
wait for a timeout which makes it easier to reason about.

* Move to forward.go

* doesnt need changing

plugin/forward/forward.go

@@ -7,6 +7,7 @@ package forward
 import (
 	"crypto/tls"
 	"errors"
+	"io"
 	"time"
 
 	"github.com/coredns/coredns/plugin"
@@ -87,7 +88,19 @@ func (f *Forward) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg
 			ctx = ot.ContextWithSpan(ctx, child)
 		}
 
-		ret, err := proxy.connect(ctx, state, f.forceTCP, true)
+		var (
+			ret *dns.Msg
+			err error
+		)
+		stop := false
+		for {
+			ret, err = proxy.connect(ctx, state, f.forceTCP, true)
+			if err != nil && err == io.EOF && !stop { // Remote side closed conn, can only happen with TCP.
+				stop = true
+				continue
+			}
+			break
+		}
 
 		if child != nil {
 			child.Finish()