Kasteel TerWorm or Ter Worm (Ter Worm Castle) was inhabited by noble families since the 14th century. The first known owner was the Lord of Strijthagen in 1476, when the castle was a moated building fortified by external walls outside the moat. The castle was destroyed by fire in 1550 but rebuilt in the same style. The rebuilding, completed by the Van Wijlre family, was done in brick and the building was painted white to disguise the difference in building materials. The castle remained in this family's possession until 1738.

At the end of the 18th century the castle was restored and the gardens laid out in French rococo style. Kasteel TerWorm became the centerpiece of a 1.500 ha estate with nine farms. In the late 19th Century the castle and the estate acquired its present neo-Gothic style.

From 1917 till 1988 the castle and its estate were in the hands of the Oranje Nassau mine, housing some of their staff. In the last decades of the twentieth century the castle became very run down until it was bought by the Van der Valk hotel chain. Kasteel TerWorm was restored ‘stone by stone’ in 1997-1999. Nowadays it houses a hotel and restaurant. The rococo garden with its orangery - carefully reconstructed based on excavations - is open for the public.