I think perhaps what is missing is an understanding of statistics. There are thousands of doors and stairways in castles. If the majority of stairways all go one way, or if the majority of doors all open one way in castles, then it is statistically impossible that this is by chance. There has to be an obvious reason. A reason obvious enough that most people building the castles saw it without too much thought.