Goat's milk cheese is cheese made from milk that came from a goat. It has a different color, structure, and taste than that of regular cheese. It is usually white and breaks apart quickly. Because it breaks apart quickly it is usually stored in plastic containers with water.