Then you end up like some of our customers with log4j. We are consultants and notice that a cave for log4j comes out. We inform our customers that we have detected an issue under active exploit, and we performed an update to non vulnerable versions and want to deploy. Customer waffles for days and gets exploited before he decides to upgrade.
Threats are often only minutes away. We are currently away to slow and manual updates are slowing you down even more.
So do automated canarying! Deploy to 1000 machines, wait 60 seconds, deploy to 10,000, etc. All done in under 5 minutes while automatically rolling back or at least halting if any metrics look bad.
Canarying is by now not a very new practice. This is like doctors not washing their hands.
