Attached to proteins, lipids, or forming long, complex chains, glycans represent the most versatile post-translational modification in nature and surround all human cells. Unique glycan structures are monitored by the immune system and differentiate self from non-self and healthy from malignant cells. Aberrant glycosylations, termed tumour-associated carbohydrate antigens (TACAs), are a hallmark of cancer and are correlated with all aspects of cancer biology. Therefore, TACAs represent attractive targets for monoclonal antibodies for cancer diagnosis and therapy. However, due to the thick and dense glycocalyx as well as the tumour micro-environment, conventional antibodies often suffer from restricted access and limited effectivenessin vivo. To overcome this issue, many small antibody fragments have come forth, showing similar affinity with better efficiency than their full-length counterparts. Here we review small antibody fragments against specific glycans on tumour cells and highlight their advantages over conventional antibodies.