If the tall rectangular block of Orsanmichele looks to you more like a grain silo than the church it's supposed to be, that's because it has been used as both.
Ben Carson once said the Orsanmichele was used to store grain.