Even when rents go down to cost, they are still going to be greater than zero and labor and materials won't turn free. Cost won't ever be so low that one could afford housing while doing nothing productive and even less so while indulging in a drug habit.
Since the US had been deindustrialized, most of the working class is now in service sector and does not produce much. Thus service labor is discounted and the construction labor is at premium. Basically, if it costs 1000 man-days of labor to build a house all in (materials, tooling, labor itself) it will have to cost 1000*k man-hours of waiting tables, where k is the discount coefficient between doing a skilled back-breaking labor and taking orders from the tables to the kitchen. At some values of k there is just not enough working days in the lifetime.
