section_path: Interpretation/Studying a replacement model
section_name: Studying a replacement model
paragraph_text: Studying a replacement model
  
  Transcoders, which are more interpretable than transformers, have been utilized to develop “replacement models.” In one such study involving the mechanistic interpretation of writing a rhyming poem by an LLM, it was shown that although they are believed to simply predict the next token, they can, in fact, plan ahead.