Jesus, Mary, and Joseph were not refugees.  They were law abiding travelers heading to Bethlehem to be counted in the census and pay their taxes.  Jesus was carried to Bethlehem in Mary's womb.  Still Mary and Joseph considered him a soon to be new born child, a person.  Joseph and Mary recognized Jesus was a boy.  Jesus remained a boy and became a man.  He was persecuted and crucified because of his conservative beliefs.  No much has changed in 2000 years.  

 "And it came to pass in those days, that there went out a decree from Cæsar Augustus, that all the world should be taxed.
(And this taxing was first made when Cyrenius was governor of Syria.)
 And all went to be taxed, every one into his own city.
 And Joseph also went up from Galilee, out of the city of Nazareth, into Judæa, unto the city of David, which is called Bethlehem; (because he was of the house and lineage of David)
To be taxed with Mary his espoused wife, being great with child."