Excellent observation. Either side can (and does) easily accuse their opponents of this.

Some don't like anti-immigrant, anti-gay, anti-climate rethoric. Others don't like trans-rights, anti-hate-speech, anti-christianity rethoric. For either side, those are not real problems which their opponents are concerned about.

That's an underappreciated aspect of current public discourse.