Title: Accumulative Poisoning Defense with Memorization Discrepancy. Abstract: Adversarial poisoning attacks pose huge threats to various machine learning applications. Especially, the recent accumulative poisoning attacks show that it is possible to achieve irreparable harm on models via a sequence of imperceptible attacks followed by the trigger sample. Due to the limited data-level information in real-time data streaming, the current defensive methods are indiscriminate in handling the poison and clean samples. In this paper, we dive into the perspective of model dynamics and propose a novel information measure, namely, Memorization Discrepancy, to explore the defense via the model-level information. Through implicitly transferring changes in the data manipulation to that in model outputs, our Memorization Discrepancy constructed by the victim and historical models is aware of the imperceptible poison samples based on their distinct values from the clean samples. We thoroughly analyze its properties and accordingly propose a Discrepancy-aware Sample Correction (DSC) to defend against the accumulative poisoning attacks. Extensive experiments comprehensively characterize our proposed Memorization Discrepancy and verified the effectiveness of our DSC.