Is there a way to make Unbound log slow DNS queries/slow upstreams?
According to Grafana/Prometheus, the average recursion time is between 2-4 seconds, which is way too high for DNS resolution imo.
Odd thing is, I haven't actually noticed DNS resolution being that slow, and if I manually do a dig at the upstream servers, things are resolved extremely quickly, as they should. I don't know if I haven't noticed due to caching or if the unbound exporter is reporting incorrect recursion time :\
That's *very* long. Enable query request *and* response log and you can see straight away which ones take time to resolve (even visually), then you can get exact metrics using timestamps. Maybe a problem with DNSSEC resolution? It's requires way more requests than regular resolution, and then if your system clock is off the signatures will fail. Also make the cache as large as possible.
@kravietz
Is DNSSEC super sensitive to time?
I'm running my internal Unbound DNS resolver on K8S, and after your comment I checked to see if any of my K8S node's time was out of sync, and low and behold, one node was indeed 3 minutes out of sync.
When I restarted Unbound to make the verbosity change, it came up on a different node so it's possible it was just the time issue.
It is sensitive - most modern authoritative DNS servers will do fully automated DNSSEC signing and key rolling at quite short intervals (like 1h) so if time is off at the validating nameserver it will think the signature is expired/not yet valid.
The same applies to apt package signatures (~1 day) and TLS certificate validation (~3 months)
@kravietz I just changed verbosity to 4, which spits out how long a query took, and now everything is resolving fast again (174ms) after restarting unbound to apply the config change 😓
I guess I'll leave verbosity high for a while and see if the recursion time increases again.