The ruin is slightly fake. The original church was demolished in the early 19th Century, but a new building was build on the foundations. When that building was demolished in 1979 and this ruin was created using the part of the chapel that remained.
There are some supposed links with the Pilgrim Fathers, it is not unlikely that some of them went to church here. It was the church of the French community in those days.