Loja was probably captured by the Moors in the 8th century. Some walls of the Alcazaba, the Moorish fortress still remain.

The town was partially destroyed by the troops of Ferdinand III during the Reconquista in 1225, before finally falling into the hands of the Christians in 1486. The expulsion of the Jews began with the Catholic Monarchs' Edict of Alhambra (1492); the last Muslims were also expelled in the years around 1610. As a result of these events and the Moorish Revolt "Rebellion of the Alpujarras" of 1568, the town lost most of its inhabitants and had to be repopulated with Christians from the north and south of the Iberian Peninsula,

In 1491 work began on the church on the site of the town's main mosque.