Luxury developments can create local rent increases and displacement even if they reduce overall rents relative to where they would be without the development, and it's not surprising that tenant groups tend to resist simple upzoning. What's more, they are wasteful of resources and space, since luxury units are much larger and tend to have a lot of square footage-eating amenities. Inclusionary zoning (mandating some percentage of affordable units in new building) is an improvement, but doesn't generally create remotely enough units to satisfy demand.
Upzoning advocates argue that if we simply went on a wild private building spree, we'd eventually burn through all the luxury demand and actually reduce the price of midrange and affordable units (as opposed to only slowing the rate of increase). But that could take easily take decades, if it even works at all.