The Chan philosophy is "Don't establish a text, and nothing will be obstructed." Given this principle of "non-obstruction," you can't say that a small building does not "obstruct" but a big building does.
禪宗是「不立文字，凡事不礙」，既然「不礙」，那你不能說建築小就「不礙」，大就「礙」呀。