- Depending upon the model and the dataset, S$^3$T can result in lower training performance compared to full training.
- No code is provided, however the authors promise to release it with the final version.
The paper can be further improved by adding a limitations section that could summarize all potential trade-offs and discussing future directions to improve the same. It would also be nice to see some performance numbers in the abstract itself, such as S$^3$T improves the deletion rate by $1.6\times$ over SISA.
- Depending upon the model and the dataset, S$^3$T can result in lower training performance compared to full training.