>But then you go refactor something and oops you broke it without realizing it, some boring obvious edge case, or the like

I will start to care when integration tests are failing, because that is an actual bug. Then I will fix the bug and move over.