It's not so much about trusting your doctor. For me, it's more about utter absurdity of the image of every single patient of a GP calling said doctor to ask this question, and thinking that the answer given is going to be any more specific for most of those patients than, say, guidance from the CDC.
Yes, if you have existing medical conditions that your doctor is aware of, they may be in a better position to point out any potential interactions between those conditions, any treatment you already receive, and a booster. But I'm guessing that this isn't a description of most of the population.
I'm also not sure that most doctor's offices really want every single patient calling in to ask "Should I get the booster?", and might simply be unable to deal with such a flood of questions.
There's also the little detail of millions of Americans not actually having "a doctor" in the traditional sense.
