The AGI can build robots that build houses. It has a virtually unlimited amount of working time to dedicate to the robotics engineering problems.

We'd still be limited to some extent by raw materials and land but it would be much less significant.

That can trim costs but not drive it to zero. If you assume that the computer is going to do all the work, won't your salary erode, making it harder for you to afford scarce things?