That’s a completely separate concern, it’s not like a new language server is downloaded for each file you open. I don’t know if Zed has a “safe mode” like some other editors, if it doesn’t you should ask for that instead. Unless of course you never open untrusted files in a language you’re familiar with, which would make you extremely peculiar.
This kind of article or reddit post and discussion is how you know, at least for some people.
Anyway, you asked who would care. Now the topic has moved to "what to do about it", which is hardly an issue. Of course people who think Zed has a problem will not use it. That does not make it a non-problem.