Traditionally known as the “Old Castle”, the present-day junction of ulica Beskidzka and ulica Zamkowa is the site of a former medieval house whose ruins were cleared in 1936.  It is assumed that the building belonged to the owners of Kozy Górne and served for both residential and defensive purposes. Until the early 18th century, Kozy Górne and Kozy Dolne had been separate administrative units and belonged to separate noble families. According to oral tradition, the remains of the walls which stood here before the Second World War constituted a fragment of a tower. The first mention of the village of Kozy dates back to 1325.  As the village was a knightly property, there must have existed a medieval residence of the owners of Kozy. The first confirmed owner of the village, mentioned in historical records of 1379, was Żegota, also known by the sobriquet Dwiekojski. It is assumed that he was a knight serving the ruler of the Land of Oświęcim, Duke Jan III. In the year of the duke’s death, the heir to the village of Dwiekozy, Przeszkon, also known as Przeszek or Przemek, bequeathed one grzywna of the annual rent for the village to the nascent Dominican monastery in Oświęcim. In the 18th century, the entire village became the property of the Jordan family who resided in a manor house which stood on the site of the palace. It is assumed that the castle in Kozy Górne lost its importance in that period, although its manor farm continued to function in the 18th and 19th centuries.