Lubrication is, I think, and under-appreciated aspect here.
Trains got substantially more efficient circa 1850-1950, more miles-per-ton, and I believe the reason was essentially that they made hotter steam, because efficiency is bounded by the temperature difference (thanks to Carnot). And the limitation on how hot you could make steam, in any given decade, was that you needed the cylinder to still be lubricated at that temperature, and this technology slowly advanced.
I think this is one of the reasons internal combustion engines didn't appear sooner. They are necessarily quite hot, even more so for diesel, which is later (and more efficient).
Trains got substantially more efficient circa 1850-1950, more miles-per-ton, and I believe the reason was essentially that they made hotter steam, because efficiency is bounded by the temperature difference (thanks to Carnot). And the limitation on how hot you could make steam, in any given decade, was that you needed the cylinder to still be lubricated at that temperature, and this technology slowly advanced.
I think this is one of the reasons internal combustion engines didn't appear sooner. They are necessarily quite hot, even more so for diesel, which is later (and more efficient).