It's an example of some people deciding, not all the people deciding, specifically the people who got their first and then put up regulatory barriers in an effort to drive up housing prices. If you gave everyone in the Bay Area a vote as to whether they wanted to "keep the SF charm" or put up high density development so they could afford to move into the city proper, I think you'd get a different outcome.
It would be interesting to put zoning issues like building height and affordable housing policy up to the voters. When the BART was being built, many SF residents organized against it, fearing the "manhattanification" of SF. I see similar attitudes today; I think a lot of SF residents are fine with the small-town "charm" and lack of big-city services, even though they are paying big-city prices.
I honestly think a lot of people in SF don't understand how much better big cities can be; there's a reason you don't see many NY transplants in the Bay. In more ways than one, the Bay Area is in a bubble.
As someone who grew up in NYC I don't think many respondents here understand how depressing and lightless it is to live in an urban environment of 30 story buildings where you almost never see the sun. That which has drawn many people to San Francisco would be nearly completely destroyed if it were developed the way midtown manhattan is.
When comparing SF and NYC you have to be careful. Manhattan (mostly Midtown and the Financial District) is "sized" to handle an influx of almost 2 million commuters and visitors each day (225% increase). So when you see those skyscraper in Midtown Manhattan, you're looking at places designed to accommodate almost 1 million people per square mile in some of the census tracts.
San Francisco doesn't have that commuting pattern. It's daytime population is only about 200,000 larger than its night-time (less than 25% increase). A neighborhood designed to accommodate 50,000-75,000 people per square mile doesn't look like Midtown Manhattan. The near north side of Chicago is in that range, and it's very light and airy.
This is what a neighborhood of 60,000 residents per square mile looks like: http://goo.gl/maps/V8IS6. That's about 2.5x as dense as the Mission.
