Skip to content
MaungaAI news, explained

Research

A researcher sees AI as easier to understand; an expert calls it a hunch

Nick Cammarata, of Goodfire, argues in a 26,000-word essay that neural networks follow more logic than it seemed. Some in the industry welcome it. A Stanford researcher sees it as little more than intuition.

1 min readOnly reported by MIT Technology ReviewWritten with AI · machine-translated

Share: WhatsApp · Email

The essentials

  • Claimed by one partyNick Cammarata, a researcher at Goodfire, published a 26,000-word essay on how neural networks work inside, according to MIT Technology Review (MIT Technology Review).
  • Claimed by one partyCammarata says language models, like the ones behind chatbots, could be much easier to understand and monitor than people thought (MIT Technology Review).
  • Claimed by one partyChris Potts, of Stanford University, considers the essay little more than a hunch that doesn't make any of the problems easier (MIT Technology Review).

Why it matters

If confirmed, it would be good news for safety. Understanding what happens inside an AI is the first step to monitoring and correcting it. Until now, researchers had only managed to decode very small pieces of large models (MIT Technology Review).

The details

According to MIT Technology Review, Cammarata went back to Inception v1, a small model that recognizes images and that he studied in 2020. At the time, his team concluded that a single neuron, the smallest unit of the model, responded to both cars and cats because it mixed unrelated ideas. He now believes both images share a pattern: a V shape (MIT Technology Review).

What we don't know

  • Whether other teams reach the same conclusions when they examine the models on their own.
  • Whether what Cammarata saw in a small image model also holds for large language models.

Sources

This story is based on 1 sources. In the text, each fact names in brackets who reports it.

Was this story useful? Yes Not really

Spanish original