Hillesøya is an old church site, with the Hillesøy Church being located on the island for a long time, until it was moved to Brensholmen in 1889. Hillesøya was the namesake for the municipality of Hillesøy that existed from 1855 until its dissolution in 1964.