The history of Hackfort Castle is dating back to the year of 1322, when it was officially mentioned. In 1392 ‘House Hacforden’ was nothing more than just a residential tower. In the 1586 the castle was destroyed by the Spanish troops during the Eighty Year's War (the Dutch Independent War). After the liberation of the region by Prince Maurits van Oranje, the castle was rebuilt in 1598 by Borchard van Westerholt.

In 1788, Hackfort Castle underwent substantial renovations. The old gate house and outbuildings were demolished and the canals were filled in. The castle was transformed into an 18th-century manor house. Nowadays only the two thick cylindrical towers are reminders of the (former) castle. The castle is surrounded by forests and meadows and includes the watermill of Hackfort.

The castle remained for centuries in the possession of the family Van Westerholt. When the last Baroness Westerholt of Hackfort died in1981, Hackfort Castle and estate were donated to Natuurmonumenten (Netherlands Natural Heritage Society). The house, coachhouse (nowadays a restaurant) and watermill have since been restored and opened to the public.