Pentecost as the birthday of the church was a *cute* liturgical stance for awhile.  The Church became Church with the resurrection and appearance of Christ on Easter - our birthday is celebrated at the Easter Vigil (not Pentecost).
 This opinion is not universally held, however. A writer from an "Orthodox perspective" argues that the Church Fathers held that the Church had no beginning but that it existed before all things. See http://www.johnsanidopoulos.com/2009/06/is-pentecost-birthday-of-church.html  A similar argument is made from a Reformed perspective at  https://garynealhansen.com/ispentecostthebirthdayofthechurch/  and at http://www.reformedworship.org/article/march-1988/pentecost-birthday-church. 
Pentecost is only mentioned in one chapter in the New Testament. Paul, for example, does not refer to it in any of his letters. He is more focused on the death, resurrection, and especially the parousia.  John has Jesus giving the Spirit on Easter Night.