> you retrofit insulation
Been there, doesn't work 😉 Old buildings have hundreds of thermal bridges where energy escapes.
On the other hand, the improvements in this field are largely asymptotic - if you got your house built up to 15 kWh/m2 standard ("passive house") you don't really need to move any further for the next 100 years.
An average UK house has ~100 kWh/m2 usage there's certainly massive room for improvement.
No black & white answers here, need to look case per case.