The Cognitive Revolution
Thinking in Silico: Goodfire CTO Dan Balsam on Concept Manifolds & a $1000/Month ML Research Agent
“I'm just always amazed when I think back to the kind of toy models of superposition is only like 3 years, right, ago now. And we have come so far.”
Open episode and seek to 04:08ConceptResearch
Report this moment“where the geometry components come in is that the features aren't like, sort of like naive, maybe like SAE build take on things is like, you could imagine that the model is encoding like a bunch of totally orthogonal concepts to each other”
Open episode and seek to 14:12ConceptResearch
Report this moment“I'm guessing it's like most, like the, the different days of the week, I would guess have a very high inner product in the sense, which would to me suggest that they all, the model kind of knows like this is a day of the week sort of thing.”
Open episode and seek to 18:19ConceptResearch
Report this moment