The castle was built by the Normans around 1054 to protect the town. Later, (in the 17th century) it became the residence of the Acquaviva and a magnificent noble house.

This splendid monument rises on ancient megalithic walls dating back to 6th-4th century B.C. which are still visible at the base of the Main Tower (“Torre Maestra”),  the only remaining Norman tower. As a whole the castle is composed of a trapezoidal plan with the four corners oriented towards the cardinal points. In the corners the four towers stood out; the symbol of Conversano is the cylindrical tower erected under the will of Giulio Antonio Acquaviva in order to replace one of the four quadrangular towers of the castle.

The Counts used to entertain their guests in the splendid rooms of the castle; moreover there were several bedrooms, a kind of “cloakroom”, the kitchen and the chapel. Among the rooms of the castle, worthy of note is the wedding room of  Count Giangirolamo II Acquaviva d’Aragona (called the “Guercio of Puglia”). At the moment it is possible to visit only a few rooms since private citizens live in many of them. The castle houses the town art gallery where the 16th century paintings representing the cycle “ La Gerusalemme Liberata” ( an Italian work of art) by Finoglio are exhibited.