Trends over time

“In fact, as we've been discussing fine-tunes of open models like Cursor's Composer 2.5, they tend to be built around a KIMI base. On Wednesday, the KIMI K3 teaser started coming in a serious way. AI leaker Leo Synthwave wrote, I think KIMI K3 is going to shock some of the Chin...”

ConceptOpen Models

“As for features, KIMI-K3 supports a million-token context window and native image inputs alongside text. It uses a mixture of experts architecture, which has become standard for both open source and proprietary models since it was introduced by DeepSeek. And the benchmarks, we...”

ConceptOpen Models

“clearly a very strong new pre-training run that could give a solid base model for future iterations as well”

ConceptOpen Models

“But it's not as cheap as free and probably too big to run locally unless you're like some sort of billionaire who fights robots in their spare time. You know, robots like this.”

ConceptOpen Models

“It could be hitting next week. And wouldn't that make sense, Gavin? As we see, everybody goes open source, open weights, open this. It's always 6 months behind. And now look at this.”

ConceptOpen Models

“That model was a larger and more capable version of their Open Model family, addressing the demand for powerful and cheap US-based models”

ConceptOpen Models

“And yet, as regular listeners well know, benchmarks don't usually tell much of a story. Unfortunately, although there aren't all that many first impressions, the ones that we have aren't great. Professor Ethan Mollick wrote, Happy to see a new open weights model, but so far In...”

ConceptOpen Models

“The world needs open LLMs from as many players as possible. Given TLM's talent, know-how, funding, and compute, they will soon reach Opus if not Fable-5 level. So basically Yunfan's argument is that this should be treated as the first, not last, model that TML is going to put ...”

ConceptOpen Models

“a good scientific prior over what is known, and therefore a good base camp to build upon”

ConceptOpen Models

“Yeah. And we use, like, we use Neutron quite a bit because we have a partnership with Nvidia. And I think there's like 30 trillion tokens that go into the pre- and post-training for that model.”

ConceptOpen Models

“Have you all— So in the process of these reasoning tokens, you are also creating what are arguably probably just rather useful datasets themselves. Have you thought about independently releasing some of those datasets open source, even in the absence of the reasoning model, wh...”

ConceptOpen Models

“So, um, all right, so open source, like open source code has faced a similar situation. Open source code still fully exists. Is there any, like, is there any, like, precedent or You know, how is it— how and why is this different? Is it because it's just the scale? It's, it's b...”

ConceptOpen Models

“I think the open source code world was different. These are applications. that can do powerful, dangerous things”

ConceptOpen Models

“Um, CH, uh, Zimis, uh, or Zimis on YouTube says, interesting, some of these companies are very pro-open source though. Microsoft with .NET, Google in general. And I—”

ConceptOpen Models

“I would imagine that just the training volume is ridiculous, but the product side release here was a bit of a mess. And next, going back to models, there's a story, Chinese AI models gain ground with US companies as costs surge. So this is a bit of a trend check where it's sho...”

ConceptOpen Models

“Keep in mind also that China is going to be clamping down on open source at some point in the next like year or 2. So many of these trends are I think properly viewed as transients, where it just becomes untenable for, you know, like cyber weapon models or models that can even...”

ConceptOpen Models

“increasingly you're going to want to like look at a model that was made, you know, with the hardware that you trust or software you trust”

ConceptOpen Models