In a civilized, democratic society that's always true. But it doesn't disqualify the original point.
And it works both ways... Other people have spent trillions of US dollars on, for e.g., the war in the middle east. Some of my tax money went to that. I didn't have the ability to veto it.
There should be a reasonable public debate and then people vote. Very often it involves spending other peoples money. That's just how it works.
Getting money from society doesn't come with a grant of immunity to it's rules. Are you saying that it should?
If you use your money to start a business involving a giant tire fire in your back yard then society will, rightfully so, have something to say about it.