Title: Learning Basic Interpretable Factors from Temporal Signals via Physics Symmetry. Abstract: We have recently seen great progress in learning interpretable music representations, ranging from basic factors, such as pitch and timbre, to high-level concepts, such as chord, texture and melody contour. However, most methods rely heavily on music domain knowledge and it remains an open question how to learn interpretable and disentangled representations using inductive biases that are more general. In this study, we use \textit{physical symmetry} as a self-consistency constraint on the latent space. Specifically, it requires the prior model that characterises the dynamics of the latent states to be \textit{equivariant} with respect to a certain group transformation (say, translation and rotation). We show that our model can learn \textit{linear} pitch factor (that agrees with human music perception) as well as pitch-timbre disentanglement from unlabelled monophonic music audio. In addition, the same methodology can be applied to computer vision, learning the 3D Cartesian space as well as space-colour disentanglement from a simple moving object shot by a single fix camera. Furthermore, applying physical symmetry to the prior model naturally leads to \textit{representation augmentation}, a new learning technique which helps improve sample efficiency. 