keyword
IOI circuit
An IOI circuit, short for indirect object identification circuit, is a specific computational subnetwork within a transformer language model that mechanistically implements the ability to identify and predict the indirect object in a sentence. In the field of mechanistic interpretability, it refers to an identified subgraph of interconnected attention heads and pathways that cooperatively resolve references in natural language, such as completing a narrative prompt with the appropriate non-repeated individual when two names are introduced and one is later repeated as the subject. The circuit operates through distinct functional groups of attention heads, including duplicate token heads that identify repeated names, inhibition mechanisms that suppress those candidates, and name mover heads that extract and boost the probability of the correct indirect object for the final output token prediction.
1 item
