The parish church St Mary was built in the 1130s by Norman masons in the outer bailey of the ruined Roman fort. The once cruciform church was built for an Augustinian priory which was established within the castle in 1128. Other buildings would have been planned for the priory, although almost no trace of them survives. As the Augustinians moved to a new site at Southwick between 1147 and 1150 already, the buildings may never have been completed.

The Romanesque-Norman baptismal font is exceptionally well preserved. People and animals are entwined in tendrils all around - a detail.