By my experience engineers tend to way underestimate schedules and those are the two reasons. Best case scenario means no suprises come up during implementation, which is almost always false. Roughly double the best case estimate is usually about right. The second though can be a miscommunication, whether the estimate is an actual date when done or an estimate of the man-hours.
Another problem is that engineers also tend to estimate only the time for original implementation, not for design, testing, documentation, bug fixes, meetings, etc.
A long time ago I read about some research that showed that the original implementation makes up about 1/6 of the total time spent on the task. This has been roughly the case in my experience too -- so now I budget for at least 6× estimated implementation cost, more if it's a tricky problem.
A reductive approximation: engineers estimate at 50% confidence (on average it will be done on this date), management and leadership hear that at 90% confidence (it will be done by this date to near certainty). Making these assumptions explicit up front solves tons of problems.
This has been one great thing about moving from "just engineer" to "consultative engineering/architecture/etc" in my career - learning how to speak to the business needs over the technical needs
At the end of the day, Ford doesn't care about its IT environment - they care about building and selling cars, trucks, etc
The NASDAQ doesn't care about the OSes on their servers, they care that transactions complete properly
Learn to speak to the business case, and the tech takes care of itself (more-or-less)
