The 15th-century flint building is a former ferry house on the River Wensum. The arch on which it is built was once a water gate in the medieval town wall. In Norman times, when the cathedral was under construction, a canal, running under the arch, was dug by monks between the river and the site to allow building stone and materials to be unloaded close by it.

Pull's Ferry is named after John Pull, who ran the ferry across the river between 1796 and 1841. The ferry operated until 1943.