We're not in danger of losing jobs any time soon.

I think his point is more easily summarized as, "Why are we creating 3 new jobs here for every 1 new housing unit we build?"