It is always a problem, in the same way that holding on to huge objects is always a problem. Let's think about this in terms of a memory map, so ignoring limits of physical memory. On 64bit systems we actually have 48bits of usable address space, so that's 256TB. Now, the OS is going to need some workspace, but I think we can ignore that for this argument. So if you're only going to run 4 threads then each one could have a huge stack of 32TB, but if we wanted to run a huge server with a thread per connection then maybe we can only have 256MB per thread.
Now, 256MB is pretty huge, but we probably want way more memory kept for heap storage and similar things (because managing lifetimes purely on the stack will be hard and might require a lot of copying of data), so it will be less than that. Now, you may ask, "Why not start with small stacks and make them bigger as needed?" It's a good question, but since our stacks are contiguous areas of memory and we don't know what's in them, we still have to space them out in our memory map allowing for as much growth as is needed.
We might solve some of these problems by introducing segmented stacks, but this is one of those problems that crosses so many language, runtime, and OS boundaries that it's been hard to do in the general case, and it feels like gently nudging people towards writing code that can be tail-optimised is easier, just as it's easier to push people to use async than it is to provide systems and APIs that would allow for blocking code and a huge number of threads.
Thanks, I didn't realise the usable address space was so "small", those numbers are well within reasonable usage so that's definitely too limiting to just say it's infinite and not worry about it.
Apart from address space, there’s also simply the question of wasted resources, because in tail-call-optimizable programs, that stack space is by definition not strictly needed (otherwise it couldn’t be optimized away), so you might unnecessarily be wasting many GBs of memory. Lastly, TCO can improve cache locality and thus potentially speed up the program.
Now, 256MB is pretty huge, but we probably want way more memory kept for heap storage and similar things (because managing lifetimes purely on the stack will be hard and might require a lot of copying of data), so it will be less than that. Now, you may ask, "Why not start with small stacks and make them bigger as needed?" It's a good question, but since our stacks are contiguous areas of memory and we don't know what's in them, we still have to space them out in our memory map allowing for as much growth as is needed.
We might solve some of these problems by introducing segmented stacks, but this is one of those problems that crosses so many language, runtime, and OS boundaries that it's been hard to do in the general case, and it feels like gently nudging people towards writing code that can be tail-optimised is easier, just as it's easier to push people to use async than it is to provide systems and APIs that would allow for blocking code and a huge number of threads.