Efforts to build a church in Kozy-Gaje began in the early 1990s, and in 1993 the site for the construction of the church was consecrated by Bishop Tadeusz Rakoczy. The design was ready in 1996, the construction works started a year later, and the corner stone was laid in 2000. The new parish, which embraces the districts of Kozy-Gaje and Kozy-Zagroda, received the name Pentecost.  This is the third parish, after Bujaków and Małe Kozy, to be separated from the mother church of Saints Simon and Jude in Kozy.