This is what strikes me as well. You probably wouldn't be fighting with a sword, because at best those were side arms for knights in most circumstances in Europe. You'd likely defend a staircase from the top with a barricade, and/or by pushing something down it. Spears or other pole arms would be preferable for defending the space from the top. But as you mentioned, you might as well start a small fire at the bottom to smoke out defenders.

Worst case scenario as an attacker, you could just wait out people on the upper levels, they have limited supplies and you presumably control the gates and ground level.