That is exactly what booing is, but citizens are allowed to boo. I can boo you, you can boo me. If you are booing me then I can walk away, and likewise you can walk away from me. If I'm booing you during a public performance that is indeed rude but then I need to be thrown out by security, which is perfectly allowed and expected.

Citizens, i.e each other, are not the problem when it comes to free speech, ever. The only entity which needs to be defended against is the entity that has a monopoly on violence, which is of course the government.