I've been helping test it as well - your experience sounds identical to mine. I was initially very excited for it, but nowadays I don't really bother turning it on unless I feel the conversation will be a long one. Although I am very much looking forward to them rolling out the automated pending of orders based on what was said during the conversation.
LLM's have so much potential in medicine, and I think one of the most important applications they will have is the ability to ingest a patient's medical chart within their context and present key information to clinicians that would've otherwise been overlooked in the bloated mess that most EMR's are nowadays (including Epic).
There's been so many times where I've found critically important details hidden away as a sidenote in some lab/path note overlooked for years that very likely could've been picked up by an LLM. Just a recent example - a patient with repeated admissions over the years due to severe anemia, would usually be scoped and/or given a transfusion without much further workup and discharged once Hgb >7. Blood bank path note from 10 years ago mentions presence of warm autoantibodies as a sidenote; for some reason the diagnosis of AIHA is never mentioned nor carried forward in their chart. A few missed words which would've saved millions of dollars in prolonged admissions and diagnostic costs over the years.
Given everything I hear about LLMs for similar summary purposes including your description and that given above, it seems unlikely that the LLM would be all that likely to “notice” a side note in a huge chart. I agree that’d be great but I’m curious why you think it would necessarily pick up on that sort of thing.
> it seems unlikely that the LLM would be all that likely to “notice” a side note in a huge chart
I respectfully disagree - I think LLM's have already made significant advances in this area as shown in the various "needle in a haystack" demonstrations we've seen over the past couple years. I've already been impressed by the minute but relevant details they can "recall" after being fed very dense journal articles and the technology is only getting better. Also keep in mind that the raw text / "data" itself found in many patients' charts is not always that expansive (though it certainly can be for patients with recurrent admissions). It's more an issue of finding the actual information given that EMR's are a nightmare to navigate effectively.
Hallucinations are always a consideration too, but any implementation of the sort I mentioned before would certainly contain in-text backlinks to actual notes in the EMR. Epic already does this with their basic text search function. So I don't think hallucinations would be too problematic as clinicians should always be verifying this type of information at the source in good practice.
> A few missed words which would've saved millions of dollars in prolonged admissions and diagnostic costs over the years.
I don't mean to come off antagonistic here. But surely the more important benefit is the patient who would've avoided years of sickness and repeated hospital visits?
> But surely the more important benefit is the patient who would've avoided years of sickness and repeated hospital visits?
The patient experience is always important and maybe I could've been less implicit in what I wrote. I think I was focusing more on the collective/societal impact this would have, which I felt would resonate more with the readers here.
As a patient with an under-served condition I quite often focus on the financial rather than human cost of not having a better system of care when talking about it.
If someone’s going to object to improving the system it’s mostly likely going to be on grounds of cost.
I don't know, if it was really millions of dollars for a single patient - I wouldn't pay a few million dollars to avoid a few bouts of illness for a random member of my health insurance group scheme cohort, and that seems like the correct comparison to make. Increase the costs by another order of magnitude and I'd rather let them die.
But it's "millions of dollars in prolonged admissions", not just "prolonged admissions". The point is the financial cost, not the wellness of the patient.
LLM's have so much potential in medicine, and I think one of the most important applications they will have is the ability to ingest a patient's medical chart within their context and present key information to clinicians that would've otherwise been overlooked in the bloated mess that most EMR's are nowadays (including Epic).
There's been so many times where I've found critically important details hidden away as a sidenote in some lab/path note overlooked for years that very likely could've been picked up by an LLM. Just a recent example - a patient with repeated admissions over the years due to severe anemia, would usually be scoped and/or given a transfusion without much further workup and discharged once Hgb >7. Blood bank path note from 10 years ago mentions presence of warm autoantibodies as a sidenote; for some reason the diagnosis of AIHA is never mentioned nor carried forward in their chart. A few missed words which would've saved millions of dollars in prolonged admissions and diagnostic costs over the years.