* Add part 1 watch functionality. (squashed)

* add funcs for service/endpoint fqdns

* add endpoints watch

* document exposed funcs

* only send subset deltas

* locking for watch map

* tests and docs

* add pod watch

* remove debugs prints

* feedback part 1

* add error reporting to proto

* inform clients of server stop+errors

* add grpc options param

* use proper context

* Review feedback:
 * Removed client (will move to another repo)
 * Use new log functions
 * Change watchChan to be for string not []string
 * Rework how k8s plugin stores watch tracking info to simplify
 * Normalize the qname on watch request

* Add blank line back

* Revert another spurious change

* Fix tests

* Add stop channel.
Fix tests.
Better docs for plugin interface.

* fmt.Printf -> log.Warningf

* Move from dnsserver to plugin/pkg/watch

* gofmt

* remove dead client watches

* sate linter

* linter omg

In the case of ttl <= 0, `cacheMisses` should increase its counter.
Signed-off-by: Jun Li <lijun.git@gmail.com>

* Replace gh-release with curl

This fix tries to address the issue raised in 1895 where
gh-release is deprecated.

As GitHub provides API that could be accessed easily,
and gh-release itself is just a wrapper to the API,
this fix replaces gh-release with the direct interaction
with GitHub API, so that it could be maintained by coredns
team.

This fix fixes 1895.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

* Added `-o /dev/null`
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

* retry creating dnstest.Server if tcp or udp port is in use

* changes according to review

Correctly set the path for the presubmits, fix some typos and make
the goimport target do the linting to the lint target can be removed.

Also don't make it a fatal error because gofmt changes between releases.

After several experiments at SoundCloud we found that the current
minimum read timeout of 10ms is too low. A single request against a
slow/unavailable authoritative server can cause all TCP connections to
get closed. We record a 50th percentile forward/proxy latency of <5ms,
and a 99th percentile latency of 60ms. Using a minimum timeout of 200ms
seems to be a fair trade-off between avoiding unnecessary high
connection churn and reacting to upstream failures in a timely manner.

This change also renames hcDuration to hcInterval to reflect its usage,
and removes the duplicated timeout constant to make code comprehension
easier.

The prometheus naming convention states only counters should have a
`_total` suffix, so that gagues and counters can be easily
distinguished.

This removes:
* reverse plugin from plugin.cfg
* https_google option from proxy
* the -log flag

The default dns.Response implementation of a dns.ResponseWriter will
panic if RemoteAddr() is called after the connection to the client has
been closed already. The current cache implementation doesn't create a
new request+responsewriter during an asynchronous prefetch, but
piggybacks on the request triggering the prefetch.

This change copies the RemoteAddr first, so that it's safe to use it
later during the actual prefetch request.

A better implementation would be to completely decouple the prefetch
request from the client triggering a request.

Reduce map usage and combine two maps into one; add isDuplicate that
tests and adds to make things more readable.

* - ensure description of 'append' match what is encoded.

* - fix description based on review comments

add a test to see if we copy the rcode correctly. Some minor cleanup in
import ordering and renaming NewUpstream to New as we already are in the
upstream package.

* - implement an auto-tunable dialTimeout for fallback.

* - fix gofmt

* - factorized timeout computation with readTimeout / updated readme /

* - fix comment

Fix typo corendns -> coredns in coredns.1

Bump the version, allows #691 to get merged.

* plugin/file: fix local CNAME lookup

Issue #1864 explains it will, when we serve the child zone as well we
should just recursive into ourself (upstream self). Thus relax the
IsSubDomain check in file/lookup.go and just query (even if the query
will hit a remote server).

I've looped over all other plugins that do something similar (CNAME
resolving) and they didn't do the IsSubDomain check; therefor I've
removed it from *file* as well.

Added test in file_upstream_test that shows this failed before but now
results in a reply.

Fixes #1864

* self does not need to be exported

* Fix test

We don't know if we had a valid reply. Check this.

Remove the code and remove the call in etcd and kubernetes handlers.
This does mean we should not add dups in the first place, which means
adding maps in backend_lookup to prevent dups from begin added.

This should cut down on the allocations because dnsutil.Dedup is very
expensive by converting everything to strings, we avoid doing that now.

Add "Why is this needed".

I can get a description of what the PR does from the code changes,
answering the why question is more important.

* Current stage of the log files. Test need to be done as well as formatting of times.

* Finished testing. All altered classes test pass along with my additions

* Updated the replacer package to print the units as well. May take out.

* Changed the time units to be within the rules. Fixed the test as well.

* Fixed some tests, updated the readme, fixed the replacer class.

* Updates of standardizing only to seconds in response duration. Need to revert README.

* Reverted readme.

* Added a small test in new replacer.

* Changed replacer to inline the strconv for duration.

These must be alligned on 8 bytes, in Go this means putting them first
in the struct (AFAICT).

This fix update Makefile.release for docker image build
on different platforms.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

* Docker: drop alpine

Create a multistage docker build image that uses debian to install certs
and then create the final image by using FROM: scratch. This creates a
(slightly) smaller images and drops busybox and alpine.

* Even less copying

Uppercase these test cases as well.

Uppercase all these test errors as well. And extend the presubmit to
check for these in the future. Also do a slightly smarter grep to only
get t.<something>. as (because dump regexp) this also grep over non test
files.

No other fixes; this only adds the check.

Fix the casing *and* fix use of context as we were still referencing
the non-std lib context - no wondering how this could have worked...

We use "_" every where else.

(Might add presubmit to test for this)

Signed-off-by: Mario Kleinsasser <mario.kleinsasser@gmail.com>

Rework the documentation
Signed-off-by: Mario Kleinsasser <mario.kleinsasser@gmail.com>

* plugin/forward: erase expired connection by timer

 - in previous implementation, the expired connections resided in
   cache until new request to the same upstream/protocol came. In
   case if the upstream was unhealthy new request may come long time
   later or may not come at all. All this time expired connections
   held system resources (file descriptors, ephemeral ports). In my
   fix the expired connections and related resources are released
   by timer
 - decreased the complexity of taking connection from cache. The list
   of connections is treated as stack (LIFO queue), i.e. the connection
   is taken from the end of queue (the most fresh connection) and
   returned to the end (as it was implemented before). The remarkable
   thing is that all connections in the stack appear to be ordered by
   'used' field
 - the cleanup() method finds the first good (not expired) connection
   in stack with binary search, since all connections are ordered by
   'used' field

* fix race conditions

* minor enhancement

* add comments

* Implement deprecation notice for 1.1.4

This still allows all the config to be parsed, but noops it:

* -log; always set the log to stdout; no matter what.
* https_google; removed from the proxy implementation.
* reverse plugin: set to deprecated.

* Whole of reverse can go

* Remove test for deprecated plugin

enable alias and add one, so that "/plugin: forward" adds a label
called plugin-forward to the issue.

enable branches that automatically delete merged branches.

Fix the upstream unit test to actually test it.

* ADD: ignoreemptyservice option for kubernetes plugin

* Modify documentation and rename option to add space

* UPD: Add unit tests

* UPD: gofmt

* Add unit test for ignore emptyservice

* gofmt

* xfr tests failed

* Rename emptyservice to empty_service

The DoH work (#1619) made changes to pkg/nonwriter.Writer that in
hindsight were not backwards compatible; it added override for the
LocalAddr() and RemoteAddr(). Instead of rolling back that PR, this PR
reverts those changes and creates a DoHWriter for use in the
https-server.go side of things.

This was only caught in the integration test making this hard to catch,
so we add a upstream_file_test.go that tries (doesn't work yet) to test
this in the unit tests as well. Esp. helpful when 'git bisecting'.

Fixes #1826

* Add links to whoami plugin reading material

* create See Also section