The Chesterton fence argument is that you need to understand why the fence is there. If you do understand it, and still remove it, it doesn't say that's bad.