Skip Navigation

Anthropic can now track the bizarre inner workings of a large language model

www.technologyreview.com

Anthropic can now track the bizarre inner workings of a large language model

7 comments
  • The official Anthropic post/announcement

    Very interesting read

    The math guessing game (lol), the bullshitting of "thinking out loud", being able to identify hidden (trained) biases, looking ahead when producing text, following multi-step reasoning, analyzing jailbreak prompts, analysis of antihallucination training and hallucinations

    At the same time, we recognize the limitations of our current approach. Even on short, simple prompts, our method only captures a fraction of the total computation performed by Claude, and the mechanisms we do see may have some artifacts based on our tools which don't reflect what is going on in the underlying model. It currently takes a few hours of human effort to understand the circuits we see, even on prompts with only tens of words.

  • just a taste :

    (...) The team found that Claude used components independent of any language to answer a question or solve a problem and then picked a specific language when it replied. Ask it “What is the opposite of small?” in English, French, and Chinese and Claude will first use the language-neutral components related to “smallness” and “opposites” to come up with an answer. (...)

  • For some reason I don’t find it very bizarre. I’d even speculate that a random human mind isn’t any less weird. Surly, the pathways of my thoughts are often very bizarre. 😅

7 comments