Angelica keiskeiKoidzumi (A. keiskei) is used as a traditional medicine, anti-aging agent, and health food, as well as to restore vitality. Xanthoangelol (xanol), a prenylated chalcone, is the predominant constituent ofA. keiskei. Oral squamous cell carcinoma (OSCC), the most common malignancy, has a high proliferation rate and frequent metastasis. However, it is unknown whether xanol has anti-OSCC effects on apoptosis, autophagy, and necroptosis. In the present study, we purified xanol fromA. keiskeiand demonstrated that it suppressed cell proliferation and induced cytotoxicity in human OSCC. Xanol triggered apoptotic cell death by regulating apoptotic machinery molecules but inhibited necroptotic cell death by dephosphorylating the necroptotic machinery molecules RIP1, RIP3, and MLKL in human OSCC. We also found that xanol inhibited the PI3K/AKT/mTOR/p70S6K pathway and induced autophagosome formation by enhancing beclin-1 and LC3 expression levels and reducing p62 expression levels. Furthermore, we showed that xanol prevented the metastatic phenotypes of human OSCC by inhibiting migration and invasion via the reduction of MMP13 and VEGF. Finally, we demonstrated that xanol exerted anticancer effects on tumorigenicity associated with its transformed properties. Taken together, these findings demonstrate the anticancer effects and biological mechanism of action of xanol as an effective phytomedicine for human OSCC.