You shouldn't need any "ecosystem" just a bootloader, TF-A and a kernel. The rest should be abstracted away for you by the kernel, unless you're doing something really heavily reliant on details of the HW, like implementing a smartphone functionality.

And that's how I use all of my SBCs (I just counted I have 7 different SoCs on my various boards and mobile devices). I put on a regular whatever distro on all of them (Arch Linux ARM in my case, but it can be anything. That's the point. The distro doesn't need to "support" my board.) it and I only figure out my own kernel and bootloader builds. Bootloader can be figured out once and forgotten about. Kernel just needs a quick update now and then, if I care. Nothing much harder than a make && make install with some env vars set and a corss-compiler installed on my workstation.

I don't need any special board specific software ecosystem provided by some foundation for normal desktop or server uses. It's already there (any GNU/Linux on ARM distribution must work), and it's generic. If the SoC is so bizarre that it can't just run U-Boot/TF-A/Linux/any distro in rootfs in that order, and requires something custom, including non-standard tools in userspace to control basic things like display output, etc., that may not be worth bothering with, but that's only really a domain of Raspberry Pi, and not common anywhere else.

Some distros provide generic kernel builds and a proper bootloader/device tree already, for many boards. So if someone is not interested in building their own, they don't need to. That may limit the choice of the distro though. But it's just an artificial limit.