Slavery was a huge part of the Civil War.The young men fighting for both sides were duped. Rebels, by slave owners that wanted to expand slavery & slick,manipulative Union politicians,Lincoln being one of them.I would say that 90% of the  yankees were fighting to preserve the Union not to end slavery. You will come across so many quotes that state bluntly " I am not fighting for a ni$$er." As soon as their time was up, they left the army. The south was fighting an invading army that meant to rape & pillage,they were right. They were fed a load of crap about fighting for states rights. 
 Jim Crow laws were in almost every state of the Union. AS far as your white heritage it most likey came from the fact that Union soldiers raped & killed more slave women & as young as nine & gang raped.They considered them property, not people 
The  Union army was horrible to blacks. They used them for labor and digging up body's 4 relocation. DEVILS PUNCHBOWL,look it up est.20k died. Tired of BS.