Northwest of the church are the small Corning Park and Albany's city hall, outside the historic district but on the Register as well. West of the church along the north side of State are the Guild House, its parish hall, and smaller, older commercial buildings. A block further is Lafayette Park and New York's state capitol, a National Historic Landmark like St. Peter's. Facing the building on the south side is a vacant lot and some more older commercial buildings, one of which houses the local extension of the Cornell University School of Industrial and Labor Relations. Opposite the church at the southeast corner of State and Lodge are the offices of Albany County's information-technology department and the state comptroller. More parking garages separate them and the large Times Union Center arena to their south. The church itself has walls of Schenectady bluestone with trim of New Jersey sandstone. Shed-roofed additions are on either side of the main block. A square tower rises from the southeast corner, with a narrow octagonal tower on its own southeast carrying a spiral staircase. In the rear is a small octagonal chancel. Decoration on the facades is French Gothic in character. A small projecting main entrance portico has a gabled top with a rosette in the apex topped by a cross. It is flanked by two small miniature versions set on stone pillars. At the corners octagonal stone spires rise, with round smooth columns supporting gables that rise to a conical roof with a folial finial. At the second story, set off by a large water table, are four narrow lancet arched windows, with a rosette in their transoms. A molded stone course connects their springlines. Above them is a large rosette window with a small lancet-arched window in the gable apex. At the roofline is a modillioned cornice. The area behind it on the roof is trimmed in copper; the rest of the roof shingled in slate. It is pierced by five gabled dormer windows on either side, also topped in copper. In the lower three of the tower's five stages is a single narrow lancet-arched window. The shorter fourth stage has three small windows recessed behind an intricate set of smooth round columns and Gothic arches. Above them the fifth stage has corresponding narrowed 9-foot-tall (3 m) louvered vents in a similar treatment. The belfry, with six tons (5.4 tonnes) of bells inside, is 30 feet (10 m) tall.