- The experimental reports are lacking in many details about the experimental methodology, making it difficult to be confident that the claims are robust.
- The explanations throughout the paper should be clearer to fully communicate the ideas and experiments of the authors.
- The S2 hacking hypothesis is quite vague and the author do not present any deep understanding that would explain the mechanisms by which certain attention heads pay extra attention to the S2 token.
- In the experiments on the DoubleIO and other prompt variations, it is unclear at which token positions paths are being ablated, as this is unspecified by the original circuit.
- The authors write: “Given the algorithm inferred from the IOI circuit, it is clear that the full model should completely fail on this task”. However this is a misunderstanding of the original work. The IOI circuit was discovered using mean ablations that keep most of the prompt intact. Therefore Wang et al. don’t expect it generalize to different prompt formats.
- The authors write “In the base IOI circuit, the Induction, Duplicate Token, and Previous Token heads primarily attend to the S2 token” this is incorrect according to Section 3 of Wang et al., 2023. These heads are _active_ at the S2 token, but do not primarily attend to it.
- The authors write: "The proposed IOI circuit is shown to perform very well while still being faithful to the full model" In fact, the IOI circuit is known to have severe limitations, as shown in concurrent work by Miller et al. (2024) [[2]](https://arxiv.org/abs/2407.08734). Nitpicks:
- In Figure 2, it is not clear what Head 1, 2, 3 and 4 refer to.
- The paper should include Figure 2 from Wang et al. 2023 [[1]](https://arxiv.org/pdf/2211.00593#page=4.37) to make it easier to follow discussions about the circuit.
- The authors write “In the base IOI circuit, the Induction, Duplicate Token, and Previous Token heads primarily attend to the S2 token” this is incorrect according to Section 3 of Wang et al., 2023. These heads are _active_ at the S2 token, but do not primarily attend to it.