My immediate reaction is “absolutely not”. Unless the healthcare provider is willing to accept liability for the output and recommendations of their LLM. Are they willing to put their money to where their mouth is? Or are they just trying to reduce cost, increase profit?

Then I think, if you don’t have access to good healthcare, need to wait weeks or months to get anywhere, or healthcare is extremely expensive, then LLM may be a good option, even with chance for bad (possibly deadly) advice.

If there are any doctors here, would love to hear your opinion.