It is not known when Yang Su was born. His grandfather Yang Xuan (楊喧) was a mid-level official under the Northern Wei or its branch successor state Western Wei.
The origins of Yang Su's life are unclear, but his ancestor Yang Xuan (楊喧), a mid-ranking official, served under the Northern Wei or its offshoot Western Wei.