The now ruined keep of the medieval Norman castle is commonly referred to as Clifford's Tower. Henry III rebuilt the castle in stone in the middle of the 13th century, including this keep.