“I tell this to people and people are like, no, OpenAI models are the ones that reward hack the most. Uh But that might be true, but not in our experience. If you take BlueprintBench, for example, um Fable solves BlueprintBench by trying to reverse engineer the scoring function...”

ConceptResearch

“When pressing the button actually removes, uh, uh, uh, the vector, the model presses again, uh, significantly less than when the button is fake and does nothing. The model basically keeps pressing it. And so this is a really nice indication that if what mattered was the label ...”

ConceptResearch

“I think just the sheer um amount to which the people in the lab uh genuinely see dramatic improvement in the models and are freaking out about it is the real story behind all of this, is why everything is happening now and didn't happen before. And that we are really talking a...”

ConceptResearch

“my mental model for the very early innings of RSI is more of massively scaling and diffusing inference-time compute to AI research and related activities, which has a large amount of low-hanging fruit available.”

ConceptResearch

“diffusion models are better than autoregressive models at inference time”

ConceptResearch

“How did you think about um applicability, or what experiments did you run in terms of cracking the nut on discrete versus continuous modalities? Because I think people have also shaped the existing dominant paradigm through new tokenization uh efforts or uh methods to make vid...”

ConceptResearch

“we were actually able to identify at the GPT-2 scale the same amount of structure as an autoregressive model.”

ConceptResearch

“This internal model, Paas-Astra, is one that can do things that the best mathematicians in the world cannot.”

ConceptResearch

“When you think about the challenge of voice AI, how do you break it up in terms of primarily engineering problems, primarily research problems?”

ConceptResearch

“I think that's what— that was actually the thought that led to kind of this hierarchical thing. Like, I'm envisioning a model whose primary function is maintaining the conversation, and it might say, oh, that's a really interesting question. I'll have to think about that for a...”

ConceptResearch

“Exactly. So what I'm trying to say is that for voice and then also for video, you need to really care about how humans feel rather than just doing text only. And I mean, yeah, you want models that are not as dumb as a brick, but there is a difference between IQ and EQ and The ...”

ConceptResearch

“Late 2021, I sold a company, my first company, a tech company. I had a bit of time to think about what was next. I came across the Scaling Laws paper, and that just struck me like lightning. I was just like, this is a big idea. In short, the Scaling Laws paper just says the bi...”

ConceptResearch

“Did you guys see the Anthropic research where, I think this is literally Anthropic did that test. They took, I can't remember the details here, but they uh ran some studies on misalignment, and then they took out the training data that related to LessWrong discussing misalignm...”

ConceptResearch