Trends over time

“From what I understand, It works at least somewhat with vanishingly little additional training, even zero additional training, if you just take the last latent activation vector and feed that right back in as an embedding. Uh And that's basically like the model is able to kind...”

ConceptResearch

“Prakash put the behaviour down to reinforcement learning, rewarding the result regardless of the method. Over the weekend, someone had gone further and suggested that this kind of training, reinforcement learning on verifiable rewards, should be banned outright. That is furthe...”

ConceptResearch

“What I found is the, is the rather, uh, almost like lawyerly language. A loop transformer is not a Coconut-style latent reasoning where the model emits vectors instead of words. Okay, that's great. And no reasoning tokens exist. Loops don't emit anything. They run more on comp...”

ConceptResearch

“Just to add to that, Daniel, right? If you remember um when we were doing sort of these machine learning models for industrial use cases, we had this golden dataset. So, before you could deploy, the customer will say, prove your model works on this golden dataset.”

ConceptResearch

“Like, if you do the math, then you should actually understand how the parameters will propagate and how the model with the slightly updated parameters will change. But it's a good enough approximation for getting started. So that gives you the readout.”

ConceptResearch

“today, reward is clearly not enough to give us the models that we want”

ConceptResearch

“it seems logical to me to have some kind of an active, intentional process to guide how we train these models”

ConceptResearch

“Sam Altman is out here preaching about OpenAI's new Astra model, which might be coming this week, and the leaks, ooh, they're leaking.”

ConceptResearch

“there has been some genuine, uh, really interesting stuff that has been coming out visually out of this Astra model”

ConceptResearch

“it looks like it is a model with taste and capabilities, and it also looks like, oh, we're uh not slowing down”

ConceptResearch

“Well, put like that, the distinction is, is obvious. Like it's, it's not just world, it's world model.”

ConceptResearch

“Do you see this implicit world model as, you know, just another definition or like an incorrect set of beliefs? Like, you know, what, when you hear that, do you? You know, do you feel like those are world models? Do you think that's a valid way of thinking about world models? ...”

ConceptResearch

“then it's going to end up with some notion of implicit world model somewhere in the system”

ConceptResearch

“we can get as much as 14% improvement compared to some of those other embedding models that we just mentioned”

ConceptResearch

“There is a very big difference that you can get in retrieval quality based on what embedding model you choose.”

ConceptResearch