Carlton Hill station was situated at the intersection with Jackson Avenue in Rutherford, where the Erie Railroad's main line crossed, 10 miles away from New York City's Chambers Street Ferry Terminal.