I think the noise emissions of a successful launch already make it an unattractive and potentially hazardous (for your hearing) place to live, especially considering SpaceX' launch frequency.
"The US Government studied what regular sonic booms do to people during the supersonic jet craze in the 1960s and 70s! It drives them to murderous rage. While the 1964 OKC test had multiple events per day, they were limited to 2 psf."
With a difference of 5 dB, one approximation for a Starship flyback boom at 20 km is a ∼50% increase in loudness over the Concorde boom (where 9 dB represents a loudness doubling;)