Ernst was married to Magdalena until his death.  Together, they had three children: Anna Magdalena, Katharina Elisabeth and Hans-Martin Walter.
Besides toiling with his work, Ernst also enjoyed music and art, specifically Tibetan scroll art. Using scientific techniques, Ernst would research the pigments on the scrolls to learn about their geographic origin and age.
Ernst died on 4 June 2021 in Winterthur at the age of 87.