If you live in San Francisco, or if you've lived here in the last few years, you've participated in this debate 100 times already. Why do we have an affordability crisis? Because we didn't build enough housing when we should have. Why didn't we build denser housing faster? Because the planning process here is purposely slow, because of height limits and NIMBYism, and because when it comes to development and change in general, San Franciscans have tended over the last 50 years to favor preservation and protection over free-market capitalism. This widely clicked-upon, rambling essay on TechCrunch last year said as much, noting that present-day progressives like 48Hills editor Tim Redmond and Supervisor David Campos don't believe in supply and demand when it comes to housing, and don't trust that we can get affordable housing built through traditional, private development means.

Whether or not you agree with them, this much is clear: The biggest reason we can't build enough affordable housing to meet the demand and maintain an economically diverse population is that the people running this city 30 and 40 years ago were even more resistant to dense in-fill development than the politicians working today, and were more concerned with maintaining the "character" of San Francisco as it was. Architectural and environmental concerns were used by the left to trump economics, and to thwart for-profit development.

There is something noble in that, and certainly those people probably still stand by their views, if they're still around. But as SPUR executive director Gabriel Metcalf succinctly puts it in this new CityLab piece, "progressive San Francisco had a fatal, Shakespearean flaw that would prove to be its undoing: It decided early on to be against new buildings. It decided that new development, with the exception of publicly subsidized affordable housing, was not welcome."

Like Kim-Mai Cutler did in the TechCrunch essay, Metcalf describes how the bulldozing of The Fillmore district in the 1960s and the anti-freeway movement helped inform the progressive politics that would combat "urban renewal" and take on the preservationist bent we know today.

And, as we all know, Silicon Valley came along to create a huge economic engine for the Bay Area and has, among other things like our vibrant food and music scenes made San Francisco such a desirable place to move to.

But unlike other cities which saw the writing on the wall over the last 30 years, watching more and more people arrive and knowing the city needed more housing, resistance remained.

Regardless of these realities, most San Francisco progressives chose to stick with their familiar stance of opposing new development, positioning themselves as defenders of the city’s physical character. Instead of forming a pro-growth coalition with business and labor, most of the San Francisco Left made an enduring alliance with home-owning NIMBYs. It became one of the peculiar features of San Francisco that exclusionary housing politics got labeled “progressive.” (Organized labor remained a major political force throughout this time period, and has allied with both pro-growth and anti-growth forces, depending on the issue.) Over the years, these anti-development sentiments were translated into restrictive zoning, the most cumbersome planning and building approval process in the country, and all kinds of laws and rules that make it uniquely difficult, time-consuming, and expensive to add housing in San Francisco.
...
As the city got more and more expensive, progressive housing policy shifted gradually to a sad, rearguard movement to protect the people already here from being displaced. No longer would San Francisco even try to remain open as a refuge for immigrants and radicals from around the world. The San Francisco Left could never come to terms with its central contradiction of being against the creation of more “places” that would give new people the chance to live in the city. Once San Francisco was no longer open to freaks and dissidents, immigrants and refugees, because it was deemed to be “full,” it could no longer fulfill its progressive values, could no longer do anything for the people who weren’t already here.

Do you agree with him? Is this our own version of What's the Matter With Kansas? in which people who should, for their own economic interest, have held more liberal views but became change-resistant and conservative instead?

It should be noted that Metcalf has argued against rent control in the past, but since he's been writing for the Atlantic's CityLab over the last couple years he's backed off that stance — see this January 2014 piece in which he advocated protecting the city's existing rent-controlled units as a top priority.

There is certainly an argument here that the very people who've wanted to keep SF safe and habitable for "freaks and dissidents" are the ones who have, out of disdain for monied interests, voted against new development, and against "new people" in general. Metcalf, who's lived in SF for 20 years, admits that "making it possible to add large amounts of housing supply in San Francisco would never have been enough by itself." But what would have happened if, back in the 70s and 80s, SF decided to get denser in certain places?

Maybe it's inevitable that this is a place that people hold dear, and fight for, and want to protect against interlopers. But, of course, most of us began here as interlopers ourselves.

Previously: Planning Study: SF Is Losing Affordable Housing Almost As Fast As We're Building It