@trooper you may want to drill in to the location setting some more. There is a sneaky setting called “significant locations” that is described at https://support.apple.com/en-us/HT207056.
Not sure your threat model but if you don’t want that location info stored on your phone then best to go in and explicitly disable it. Even with location services disabled that setting stays on unless you are explicit about turning off.