Atherosclerosis is an inflammatory disease partly mediated by lipoproteins. The rupture of vulnerable atherosclerotic plaques and thrombosis are major contributors to the development of acute cardiovascular events. Despite various advances in the treatment of atherosclerosis, there has been no satisfaction in the prevention and assessment of atherosclerotic vascular disease. The identification and classification of vulnerable plaques at an early stage as well as research of new treatments remain a challenge and the ultimate goal in the management of atherosclerosis and cardiovascular disease. The specific morphological features of vulnerable plaques, including intraplaque hemorrhage, large lipid necrotic cores, thin fibrous caps, inflammation, and neovascularisation, make it possible to identify and characterize plaques with a variety of invasive and non-invasive imaging techniques. Notably, the development of novel ultrasound techniques has introduced the traditional assessment of plaque echogenicity and luminal stenosis to a deeper assessment of plaque composition and the molecular field. This review will discuss the advantages and limitations of five currently available ultrasound imaging modalities for assessing plaque vulnerability, based on the biological characteristics of the vulnerable plaque, and their value in terms of clinical diagnosis, prognosis, and treatment efficacy assessment.