- compliance with metadata contract. If metadata value is empty, ignore rewrite - like if metadata does not exist. (#1953)

* - add support of metadata values for edns0 local variables

* - comments from review.

* - simplify label check. Add UT

* - enhance check for Labels, add UT
- remove IsMetadataSet

* - edns0 variable - if variable is not found just ignore the rewrite.

Fix documentation and remove the unused From method.
Signed-off-by: Miek Gieben <miek@miek.nl>

Fix documentation and touch up plugin/forward/README.md
Signed-off-by: Miek Gieben <miek@miek.nl>

* DoH: put in pkg/doh

Factor out the DoH stuff into its own package, add function to request
a DoH response. This can be used by forward (and maybe proxy) to
implement DoH client support.
Signed-off-by: Miek Gieben <miek@miek.nl>

* lint
Signed-off-by: Miek Gieben <miek@miek.nl>

* ... and make it compile
Signed-off-by: Miek Gieben <miek@miek.nl>

* plugin/forward: add prefer_udp option

* updated according to code review

 - fixed linter warning
 - removed metric parameter in Proxy.Connect()

Automatically submitted.

* Doc updates

Make the name section fit on one line.
Signed-off-by: Miek Gieben <miek@miek.nl>

* Regen docs
Signed-off-by: Miek Gieben <miek@miek.nl>

* plugin/metadata: adjust doc to latest code.
Signed-off-by: Miek Gieben <miek@miek.nl>

Automatically submitted.

Allow plugins to dump messages in text pcap to the log. The forward
plugin does this when a reply does not much the query.

If the debug plugin isn't loaded Hexdump and Hexdumpf are noop.
Signed-off-by: Miek Gieben <miek@miek.nl>

* plugin/cache: add extra test for FORMERR

Add extra test that test for not caching a formerr.
Signed-off-by: Miek Gieben <miek@miek.nl>

* govet
Signed-off-by: Miek Gieben <miek@miek.nl>

Exporting Zone.File to avoid getters and setters

Updating getter and setter for Zone.File to be less racy

Renaming GetFile to File in zone plugin

* request: add LocalIP

Fix TODO that was added: Add LocalIP and test the Clear() method.
Signed-off-by: Miek Gieben <miek@miek.nl>

* Move to Errorf

PR feedback and move to Errorf instead Fatalf.
Signed-off-by: Miek Gieben <miek@miek.nl>

This was done anyway, but only deep in the functions, just do this
everywhere; allows for shorter code and request.Request allows for
caching as well.

Cleanups, make it more Go like.
* remove unneeded switches
* remove testdir (why was this there??)
* simplify the logic
* remove unneeded variables
* put short functions on a single line
* fix documentation.
* spin off wire funcs in wire.go, make them functions.
Signed-off-by: Miek Gieben <miek@miek.nl>

k8s' client-go has been updated to v8.0.0 (1.11). This fix
updates client-go dependency so that it is in sync.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

This revert 17d807f0 and re-adds the metadata plugin as a plugin that
just sets a label to a value function.

Add package documentation on how to use the metadata package. Make it
clear that any caching is up to the Func implemented.

There are now - no in tree users. We could add the request metadata by
default under names that copy request.Request, i.e

request/ip - remote IP
request/port - remote port

Variables.go has been deleted.
Signed-off-by: Miek Gieben <miek@miek.nl>

Cache IP's and ports as well.
Signed-off-by: Miek Gieben <miek@miek.nl>

Signed-off-by: Miek Gieben <miek@miek.nl>

When running `dep prune` explicitly, the following message show up:
```
dep prune
Pruning is now performed automatically by dep ensure.
```

However, after the explicit `dep prune`, there are still many files deleted. (Guess `dep ensure` is not complete yet).

This fix did a `dep prune` to clean up unneeded files.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

Signed-off-by: Miek Gieben <miek@miek.nl>

delete unused tests and fix import lines.
Signed-off-by: Miek Gieben <miek@miek.nl>

* dep ensure -update
Signed-off-by: Miek Gieben <miek@miek.nl>

* Add new files
Signed-off-by: Miek Gieben <miek@miek.nl>

* Update dependencies and add etcdv3 client

* Update etcd plugin to support etcd v3 clients

Fixes #341

These are used in the rewrite plugin, makes sense to have a common place
for them.
Signed-off-by: Miek Gieben <miek@miek.nl>

* plugin/metadata: some cleanups

Name to provider.go as that's what being defined right now in the file.
Use request.Request because that's done in variables.go anyway. Name the
main storage M, because there is no further meaning behind.

Remove superfluous methods
Signed-off-by: Miek Gieben <miek@miek.nl>

* Fix test
Signed-off-by: Miek Gieben <miek@miek.nl>

Add that we support https and do a regen.
Signed-off-by: Miek Gieben <miek@miek.nl>

* plugin/metadata: add metadata plugin

* plugin/metadata: Add MD struct, refactor code, fix doc

* plugin/metadata: simplify metadata key

* plugin/metadata: improve setup_test

* Support of metadata by rewrite plugin. Move calculated variables to metadata.

* Move variables from metadata to pkg, add UTs, READMEs change, metadata small fixes

* Add client port validation to variables_test

* plugin/metadata: improve README

* plugin/metadata: rename methods

* plugin/metadata: Update Metadataer interface, update doc, cosmetic code changes

* plugin/metadata: move colllisions check to OnStartup(). Fix default variables metadataer.

* plugin/metadata: Fix comment for method setValue

* plugin/metadata: change variables order to fix linter warning

* plugin/metadata: rename Metadataer to Provider

* Fix max-age in http server

Move the minMsgTTL to dnsutil and rename it MinimalTTL, move some
constants there as well.
Use these new function in server_https to correctly set the max-age
HTTP header.

Fixes: #1823

* Linter

* Add part 1 watch functionality. (squashed)

* add funcs for service/endpoint fqdns

* add endpoints watch

* document exposed funcs

* only send subset deltas

* locking for watch map

* tests and docs

* add pod watch

* remove debugs prints

* feedback part 1

* add error reporting to proto

* inform clients of server stop+errors

* add grpc options param

* use proper context

* Review feedback:
 * Removed client (will move to another repo)
 * Use new log functions
 * Change watchChan to be for string not []string
 * Rework how k8s plugin stores watch tracking info to simplify
 * Normalize the qname on watch request

* Add blank line back

* Revert another spurious change

* Fix tests

* Add stop channel.
Fix tests.
Better docs for plugin interface.

* fmt.Printf -> log.Warningf

* Move from dnsserver to plugin/pkg/watch

* gofmt

* remove dead client watches

* sate linter

* linter omg

In the case of ttl <= 0, `cacheMisses` should increase its counter.
Signed-off-by: Jun Li <lijun.git@gmail.com>

* Replace gh-release with curl

This fix tries to address the issue raised in 1895 where
gh-release is deprecated.

As GitHub provides API that could be accessed easily,
and gh-release itself is just a wrapper to the API,
this fix replaces gh-release with the direct interaction
with GitHub API, so that it could be maintained by coredns
team.

This fix fixes 1895.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

* Added `-o /dev/null`
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

* retry creating dnstest.Server if tcp or udp port is in use

* changes according to review

Correctly set the path for the presubmits, fix some typos and make
the goimport target do the linting to the lint target can be removed.

Also don't make it a fatal error because gofmt changes between releases.

After several experiments at SoundCloud we found that the current
minimum read timeout of 10ms is too low. A single request against a
slow/unavailable authoritative server can cause all TCP connections to
get closed. We record a 50th percentile forward/proxy latency of <5ms,
and a 99th percentile latency of 60ms. Using a minimum timeout of 200ms
seems to be a fair trade-off between avoiding unnecessary high
connection churn and reacting to upstream failures in a timely manner.

This change also renames hcDuration to hcInterval to reflect its usage,
and removes the duplicated timeout constant to make code comprehension
easier.

The prometheus naming convention states only counters should have a
`_total` suffix, so that gagues and counters can be easily
distinguished.