You might be right.  I think it is not just one thing, but a multitude of things that has brought about this decline.  The RCC's stance on contraception, the massive increase in divorce, feminism, and the societal upheavals of the 60's have each chipped away at the Church's authority.  Once that happened, it was a 50-year decline to where we are now.  

Did Vatican II have a positive effect on the Church?  Fifty years on and the Church is shrinking as parishoners stay away in droves.