I use a local Unbound resolver with similar blacklists as PiHole uses. I've just ran statistics on its reply log, which conveniently records time spent resolving and on average it's 0.092 sec so 92 ms, median however is even lower at 54 ms.
Because a local resolver has to do all the recursive resolution, I guess it's the speed and latency of your broadband that has *huge* impact here as all tiny delays add up.