White supremacism exists in the absence of statues; Antifa Leftism exists in the presence of statues.  Leftists have their own history of trying to "erase" history:  whether it's "indigenous rights" activists demanding that New York City museums take down statues of President Teddy Roosevelt, or anything of history they disagree with like the Civil War. those in favor of removing the statues are merely attempting to wipe away American history in pursuit of a Utopian vision that sees America as a place rooted in nastiness.  If you oppose tearing down and vandalizing historical statues, then you must be a white supremacist who honors their Confederacy's support of slavery.