Elucidating the regulatory mechanisms of human adipose tissues (ATs) evolution is essential for understanding human-specific metabolic regulation, but the functional importance and evolutionary dynamics of three-dimensional (3D) genome organizations of ATs are not well defined. Here, we compared the 3D genome architectures of anatomically distinct ATs from humans and six representative mammalian models. We recognized evolutionarily conserved and human-specific chromatin conformation in ATs at multiple scales, including compartmentalization, topologically associating domain (TAD), and promoter-enhancer interactions (PEI), which have not been described previously. We found PEI are much more evolutionarily dynamic with respect to compartmentalization and topologically associating domain. Compared to conserved PEIs, human-specific PEIs are enriched for human-specific sequence, and the binding motifs of their potential mediators (transcription factors) are less conserved. Our data also demonstrated that genes involved in the evolutionary dynamics of chromatin organization have weaker transcriptional conservation than those associated with conserved chromatin organization. Furthermore, the genes involved in energy metabolism and the maintenance of metabolic homeostasis are enriched in human-specific chromatin organization, while housekeeping genes, health-related genes, and genetic variations are enriched in evolutionarily conserved compared to human-specific chromatin organization. Finally, we showed extensively divergent human-specific 3D genome organizations among one subcutaneous and three visceral ATs. Together, these findings provide a global overview of 3D genome architecture dynamics between ATs from human and mammalian models and new insights into understanding the regulatory evolution of human ATs.