You are misconstruing both atheism and a secular state. A secular state says religion has no place in the public sphere, aka they are all equal therefore none should dominate, what you do in your private life on your own time is your choice. Atheism is merely the non-belief in a supernatural god type entity. That is all. I have no idea what an atheist state is, but it definitely has nothing to do with banning religions, it is simply the rejection of  irrational faith based belief.

We are all inherently bias, nothing can be done about this, we all have odd emotional based beliefs and values, it is part of being human. There exists however, a difference between proudly espousing that and projecting it on others as a taxpayer funded public representative and a private citizen. I am not advocating banning or hatred toward any particular irrational belief, merely pointing out that as an elected mp you represent all constituents so tone down the outward religiosity; a question of respect.