Apparently they are scanning OSS and popular software at scale and disclosing the vulnerabilities they found: https://cvd.z.ai/
Most of these are under embargo, but it seems there are a lot of CVE here from a wide range of popular software, many considered critical or high.
I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago?
> ... Anthropic's Project Glasswing is supposed to find them quite a while ago?
That was my thought too. For all of Anthropic's talk about their "adversaries", it seems Z.AI have been quietly offering fixes for single shot Remote Code Execution flaws in US software (Safari / WebKit) that Apple and Glasswing / Mythos missed, and that Apple would not attribute to GLM.
This is absolutely still shy of Sol and Fable, but only just by a hair. Ridiculous results. There's still not a compelling economic reason to drop OpenAI courtesy of the ludicrous reset addiction that's taken place, but it feels like we're on the precipice.
How are you all toying with running this kind of thing in a mega quantized way locally? Two weeks out from released weights, but this is still just GLM 5.2 with post-training magic.
OpenAI and Anthropic are both seeking trillion IPOs, while Chinese labs are pumping out open-weight models that are free for US providers to host and monetize.
These Chinese models cost less of US SOTA models to run, even if they are less capable. Providers can just run them, offer cheap tokens, and pocket the margin.
I just don't see how you justify a trillion valuation for US AI labs when the underlying models are being commoditized this fast.
Whether AI works or use useful or not isn’t even the question anymore. It can fulfil every promise Sam Altman has been making and will still make no financial sense to justify these valuations.
I take it from [1] (transcript of recent DeepSeek CEO discussion with investors) that DeepSeek would disagree on the immediate catastrophic impact to the likes of OpenAI or Anthropic. The reason is even though technology parity mostly exists, only OpenAI, Anthropic et al have the inference capacity to gain market share and generate revenue. Chinese vendors don't have the chips needed to scale up inference and gain market share, and the DeepSeek CEO doesn't think this would happen in optimistic circumstances in the next 3 years, but thinks it might be possible in 5 years.
In summary, regardless of country of origin, availability of inference capacity is the moat protecting the likes of OpenAI and Anthropic, not technology superiority.
I have already begun winding down my spend on claude and OAI to make room for infra budget. Anecdotal, but I have no doubt a lot of others are doing the same, I very much agree the US players have major issues looming. What an exciting time to be alive!
No time like the present to pull out and reduce your exposure. I brought this up in my employer's forums 4 months ago and honestly it's been clear even before then. In particular, the upcoming IPOs of both oAI and Anthropic will likely be disastrous for the public - the floor is falling from under them and I don't know if they can be scrappy and work with fewer resources - their internal culture may not support this. We all knew in our hearts they're a commodity - just see how easily you can switch between the 2 of them - and now there are 10 more options costing a fraction.
When Xi Jinping did the announcement of their open weights push, they might as well cancelled their IPOs....
Yes buuuut…. I do quite a bit of day trading (maybe closer to scalping) for the first few hours the market is open, everyday. Anecdotally: despite everyone knowing its valuation was ridiculous, I rode that SpaceX train pretty hard and made a pretty penny.
I close-out all my positions by end-of-trading everyday… so when the day came when there was a very clear and very scary indicator during early trading hours, quickly followed by SpaceX’s catastrophic fall right after opening bell, that was the end of my involvement….
And I fully expect oAI and anthro to be the same way. They’re being propped up with private loans, subsidies, and other tricky bookkeeping techniques. You would think their CEOs would pivot away from their current public personas. Ironically, they are like a poor man’s Elon Musk… and that doesn’t bode well for their companies
Another interesting potential market here will be 'LLM in a box'. All the hardware and other tooling in a prebuilt, but modular, package ready to go. Pay one up-front cost, get a system running [whatever open LLM] with a token rate of [x], optionally configured to be immediately ready for distributed usage. Basically the opposite of cloud stuff: no rent, no dependency, 100% guaranteed uptime, guaranteed security/privacy (at least subject to your own actions), and so on.
Palantir already offers a "turnkey AI datacenter", i.e. a rack with "NVIDIA Blackwell Ultra systems with eight NVIDIA Blackwell Ultra GPUs and NVIDIA Spectrum-X™ Ethernet networking for AI training and inference".
It is said that it comes with all hardware and software required to run inference or training with an open weights LLM.
The existence of this product, which competes with cloud-based offerings like those of OpenAI and Anthropic, is presumably the reason why the Palantir CEO criticized very harshly some time ago the business model of OpenAI/Anthropic.
While I doubt that the ethics of Palantir is any better than of OpenAI/Anthropic, in this particular case I have to agree with Alex Karp about "Sovereign AI", i.e. that only losers will make their business completely dependent on an external entity like OpenAI or Anthropic, who are certainly not trustworthy.
It is just a dedicated computer system, which should be managed by its owner, like any other on-prem servers.
I doubt that it has a good price/performance ratio, but it is a solution for those who feel that they do not want to search, buy, assemble, install and configure every HW/SW component.
“100% guaranteed downtime when you least can afford it and the support tickets are your problem.”
We’ve a hybrid shop, including hosting our own ML infra, and we save a ton from cloud spend with local ML. Easily one million USD over past three years. But it’s not “free”, you are shifting a lot of labor into your plate.
I think at this point the question is: will the US government be willing and capable to justify the trillion dollar valuation for _one_ of the companies via regulatory capture? The US has a workforce of 170m, so 1.7 trillion would come down to 10k per person, or a discounted cashflow at 3% of 25 USD per month - not including private use, students etc.
Most are not necessarily free to host and monetize. At least one of them has a license that says if you are re-hosting the model then you need a license with that company that made the model.
Which is not unreasonable. Just hosting it in the EU and promising not to retain / sell the data let's you charge a healthy extra and compete in many areas other players can't.
It is impossible to justify the absurd private valuations they have given themselves in collusion with investors.
I wish they had tried to IPO because then we’d see the judgement of the market on this. But that’s why they didn’t this year. How long can they keep up the charade that their models are uniquely valuable and on the path to AGI?
I'm sure US billionaires will find a way to extract those trillions from the public. They're smart, they can handle it. After all, they can ask AI for advice on how to do it.
"I just don't see how you justify a trillion valuation for US AI"
- military applications
- financial applications
- medical
- applied science
In all those cases it is achievable for those who have needed training data, and Chinese are not going to get them easily. US AI Labs are showing: give us the data, we will do wonders, promising "singularity"-level future achievements.
The thing that blows me away is it does this at one quarter the total parameter count of K3 (and 40% active parameter count). There's plenty of room at the bottom.
> How are you all toying with running this kind of thing in a mega quantized way locally?
Sure, let me answer that in excessive detail. I briefly tried running the UD IQ3_S quant of GLM-5.2, which is 288 GiB of weights (301 GB). Setup was: llama.cpp, 1x NVMe SSD (Evo 980), 64 GiB DDR5-5200, i9-13900HX, and 1x RTX Pro 6000. Token generation around 0.7 t/s. Not remotely usable interactively, but something I could plausibly push a codebase into and come back to a review in a couple of days.
There's potential for that hardware to go much faster, but current local inference backends make poor use of the memory hierarchy. Ideally I would have: always-active weights, KV and hot expert cache in VRAM; warm expert victim cache in host RAM; and disk as a last resort. Instead it's 1/3rd of the layers fully pinned in VRAM (all experts), and 2/3rds running wholly on the CPU with mmap()'d weights. The CPU cores spend most of their time sleeping on disk fills.
llama.cpp has backed itself into a bit of a corner architecturally by trying to support all models on all possible backends. If you look into how their "MoE offload" feature works (not viable for me because it requires enough host RAM to permanently pin the weights) you very quickly realise it's "oops, all bubbles!" due to the static compute graph splits. There are more focused frameworks like DS4 [1] and Colibri [2] which have better support for streaming weights from disk, and support GLM-5.2.
Obviously I wouldn't recommend my setup for huge models like GLM-5.2. Supposedly it can just about be squeezed into 3x GB10, or run comfortably on 4x GB10 (tensor-parallel) for multi-user serving. I'm not sure whether that qualifies as local, but it's at least not a rack.
Not sure about Sol as I haven't used it, but, at least for security work -- does it matter? It's not like you will be allowed to use Fable (or access Mythos) for anything cybersecurity-related unless your name is "Dario Amodei" or you are one of his rich friends. So regardless of how good Fable/Mythos is here it's a completely moot point for normal people, because they can't use it for that anyway.
You must be a 5000 person company with an existing enterprise contract to get approved that fast. That sounds like a 15 minute SLA agreement. Individuals no matter how qualified about cybersecurity, are ghosted
That's not my experience at all. I was approved fairly fast - around an hour from submitting the form and getting a response.
However, even being in the cybersecurity programme, Fable refuses to answer prompts that it determines could be even tangentially related to cybersecurity. In fact, for a while, I was unable to use Fable with any prompt, as it recalled from memory that I was a cybersecurity professional, which triggered the refusal even for simple prompts like asking for a chili recipe.
I am guessing you are approved for the Cyber Verification Program. I also applied and got approved in an hour (on a Saturday!), but it only applies to Opus and Sonnet: https://support.claude.com/en/articles/14604842-real-time-cy.... It let me use Opus for cybersecurity work, pretty much everything except for Ransomware development. It would occasionally still trip and start saying no till I added a note about CVP in my claude.md.
No one gets to use Fable for Cybersecurity work, and Mythos is not available under CVP. Only for select few customers, and there isn't an application form?
Have you tried to use Fable for anything even remotely security related, when the refusals kick in as soon as you even fart in the vague direction of anything security or biology-adjacent?
Why should I apply for *cybersecurity* approval in order to have model debug a program it is writing itself? Anything related to memory safety, debugging, syscalls etc (meaning, "programming") somehow is cybersecurity now?
I don't understand all this spite about "rich friends" when it was the US government that shut Fable down for not adequately blocking cyber capabilities.
I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
"Mythos" is the cyber-security equivalent of Fable (without guardrails), and only a very select few corporations have access to it.
Fable is their version with guardrails on everything except "Make me a pelican svg" or "create a to-do" app, that is the version that the government banned
Only a few corporations have Mythos because the US government is whitelisting them one at a time. Anthropic releasing Mythos to the public was never on the table, they would have been shut down in milliseconds by the feds if they tried.
Before the US government had anything to do with this, Anthropic were fear mongering Mythos (BTW, Amodei also fear-mongered GPT-2, so this is a normal pattern in their operation) calling it "too dangerous to release", and back then only Anthropic was in charge of the whitelist.
Then the government believed Amodei's bullshit and this is a result of that, this was all self-inflicted.
Sorry but if you stepped back for a moment you'd realize this is all contrived nonsense to let to have your cake and eat it too.
No, Anthropic did not mind-game the US government into being worried about cybersecurity. The NSA has been paranoid about cyber controls for longer than you've been alive. If Anthropic had come out of the gate saying "no don't worry man, our model is TOTALLY COOL", while simultaneously attacking HAWK and finding core Linux vulnerabilities, I assure you the US government would have caught up about ten minutes later and we'd be in exactly the same spot minus your ability to tell Anthropic they were wearing the wrong dress and asking for it.
Mythos isn't some scary dangerous model that can find high severity bugs seamlessly, that's just Anthropic marketing. Most of the vulnerabilities they found were low severity hyped up to make their model look good, with (I think, maybe?) the exception of a few.
Now that Chinese open weight models have similar capabilities, and their guardrails can also just be removed, it doesn't look like anyone has "hacked" into everything because of the scary dangerous models like Anthropic were making it out to be.
The majority of high severity vulnerabilities are not the kind of thing you need a PhD in Comp Sci to comprehend, they are mostly about finding a way to get a system to end up in a state different than was anticipated when entering a particular code path.
Exhaustively looking at code and identifying ways to do this is something LLMs are quite good at. They don’t get tired, and you can run them non-stop.
They're also (generally) quite good at reading the literal meaning of the code, whereas humans often see the intended meaning first, and can be biased.
If you had a tireless junior engineer who was given the job of “make this application get into a state it’s not supposed to be in”, you’d probably get similar results.
What Mythos is quite good at is both the first bit and coming up with ways it could chain that together with other bits of unexpected state to create something that forms a meaningful vulnerability rather than a dead end.
It's also quite hard to separate Mythos the model from Mythos the campaign (aka Glasswing).
They put an enormous amount of compute into bug hunting, and they found some bugs. Fair enough. For me that begs the question: what if they had spent the same compute on generating more tokens with a less-capable model? What if they had spent it on traditional fuzzing?
> I don't understand all this spite about "rich friends"
Okay, here's a challenge: I assume you're not a rich and powerful entity, so try to gain access to Mythos. I'll wait.
> I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
Well, first I'd suggest they stop with the constant fear mongering.
Here's my prediction for what will happen: the Chinese models will catch up to Fable/Mythos. They will be fully unrestricted and everyone will have access. The world will not end. Good guys will use them to harden their systems, in equilibrium to what bad guys have access to, so effectively status quo will not change.
Right, so according to you it's because of the US government that they don't release it to the public? Have you missed their constant and incessant fear mongering?
The causality chain here was not "US government says its dangerous -> Anthropic can't release it", it was "Anthropic is fear mongering -> US government listens to their fear mongering".
The issue is that these companies keep trying to pull the ladder up behind them by going "oh my god our models are so dangerous only we should be allowed to develop them". Sometimes it backfires, but the companies aren't innocent.
Realistically, you're looking at least 2x DGX sparks to run this at a 2 bit quant, but quantization really lobotomizes models so it's just better to run DSv4 flash at full precision.
4x DGX sparks should let you run this at 4 bit at least and there are some folks who ran GLM 5.2 on this configuration in r/LocalLlama
If you can afford it, another DGX spark is worth it imo. Especially since, owning just one, you have a $1000 ConnectX7 card that's unused. You can find speeds here: https://spark-arena.com/leaderboard
i run flash v4 at 2bit, its pretty great and on my tests against full model It didn't lose any capabilities. It just was thinking more. So you don't have the same efficiency.
the difference is that with open models jailbreaking is trivial if you know what you are doing so this makes a frontier open model infinitely more useful for certain tasks seeing as closed frontier models will just refuse (and jailbreaking them is a waste of time when you have good open models).
in some cases (mainly reverse engineering) I have observed GLM 5.2 jailbreaking itself with no effort on my part, the thinking trace revealed that it did some mental gymnastics to pretend it was a crackme or capture the flag competition.
I might be just reading my positive bias into that text, but is it possible that it is written less like SV marketing hype trash and more like researchers wrote it?
It does feel like it respects both me and my time.
Thank you, Z.AI.
Amazing what difference it makes when the top of your org are actual university professors.
Would be interesting to compare the Chinese version. Because, obviously, their English version is for users, not for investors or government officials, while the US labs are always addressing those too.
> Mythos 5 remains well ahead at 181 and 247 tasks. The pattern across the three is consistent: the further up the exploitation chain a benchmark sits, the wider the remaining gap to the closed frontier. Capability is growing fastest exactly where we are furthest behind.
I appreciate they don't just take the opportunity to self-glaze.
Can’t this be extended quite far? Use a cerebras-served model, use verification techniques to generate and solve millions of problems and then use that as training?
which is the bigger headline that people don't realize. this is 744b and its head to head with Kimi K3 (2.8T), smashes DS v4 pro (1.5T). even Opus and Sol are rumored to be 1.5T+ this is half the size!
Same image->html test as I showed in the Gemini 3.7 flash thread. Note that GLM isn't multimodal, but it still was able to generate something similar-ish by writing a python script to inspect the image and extract elements from it.
For having no vision, it did a tremendous job. I'm pretty impressed it was able to extract so much detail.
The Opus one is still significantly better, but that's to be expected since it's multimodal. Curious to see where a future version from Z.ai lands on this.
Apple will release M7 MacBook Pros / Mac Minis next year, and they will be able to run free LLMs locally at native speed. All software developer notebooks will be replaced to run local models, saving a lot by cancelling Claude Code subscriptions.
Developers win. Apple stocks will be rocketing. Everything else will go down. You're welcome.
You'd need the 256 gb memory model which will be expensive because apple has trouble getting capacity (got turned down by cxmt). And even then you can only run a 2 bit quant which is noticeably worse than 8 bit
They will cost an insane amount as well. Maybe less than subscriptions or tokens. But running massive models on laptops with batteries and poor cooling doesn’t make much sense.
Sure buddy, all you'll end up with is a $10k machine that run gimped models at like 30tok/s for about 5m before the fan kicks in and it starts to sound like a turboprop, while offering maybe 30% of the context size of hosted models.
I don't know why you'd want to burden your laptop with a large model. But I can totally see a new "developer workstation" product that's just a semi-large box that's optimized for running frontier open weights models for one to few users.
for example, models running at like 100-150 tokens/second (or faster!) vs 15 t/s
(fable/sol are ~60 t/s, and OpenAI just announced their Cerebras partnership(?) for "ultrafast" mode of 750 t/s)
models aren't able to run that fast right now on our consumer/prosumer hardware. M5 Max for example has a memory bandwidth of 600 GB/s. a 5090 has 3x that, so running the same model on a 5090 is that much faster (provided the model is within 30GB).
running a bigger model on an M5 Ultra is still much slower than running it on a Blackwell chip with sufficient vram, CUDA being a major difference. if apple can bridge this gap, interesting things will happen... and just imagine if M7 Ultra has comparable speeds to Blackwell (or even Rubin)!
OpenAI and Anthropic need to just go ahead and give people access to the cyber models.
Otherwise we have a world of attackers using open and closed source models against a much smaller group of maintainers that are likely heavily dependent on Anthropic and OpenAI and for whom it may not be a simple matter to just get approval to start using the open model flavor of the month.
I've been building a harness for security work, and had to switch to GPT 5.5 when even Opus started refusing security work. Then 5.6 Sol arrived, and it refuses security work, too. So, I switched to Kimi K3 and DeepSeek for API testing just because it's so much cheaper. But, if GLM is better, I'm here for it, as I think GLM is also cheaper than K3.
OpenAI seems to understand that these guardrails hurt the good guys. This is why they released Daybreak Blue, which is a step in the right direction (but the model itself is weak as it's just Sol with fewer guardrails). Anthropic seems to believe that harming defenders is worth it if it means they can achieve regulatory capture. They do a lot of mental gymnastics to try to pretend that this is not actually what they are doing. As a result they have lost a lot of customer goodwill, which hasn't yet caught up with them yet, but absolutely will IMO.
It is so valuable in practise to be able to have the models see screenshots - I guess if they aren't in the benchmarks then nobody will focus on it. But it completely nixes these for some of my main use cases.
I can’t come up with a use case where I couldn’t extract the image details using another, multimodal model and pass it into the GLM’s context with as many details as I need.
I think you lose a lot by not having the vision capability shared with the text. It is the joint reasoning across them where the power lies (the same model that sees the code and made the changes to produce the visual presentation, sees the image of it and reasons about it).
Their coding plan switched to credits, didn’t it? What are the rate limits like, compared to Anthropic or Kimi K3?
I remember trying their Coding Plan out before the change and the 5 hour limits felt too restrictive then even for light/medium work, especially cause of the whole peak and off-peak thing: https://blog.kronis.dev/blog/z-ai-s-glm-5-2-is-a-great-model...
Nowadays, I’d probably go with their Max plan if the rate limits are okay? Anyone using them now?
Oh also unrelated but ZCode was surprisingly good, which is surprising for a tool that came out of nowhere - even some of the critiques in my blog post have been patched out. Sadly they don’t support using Claude Code as an agent so can’t use it like Paseo or Kepler or Agent Orchestrator.
People familiar with the topic, how will models continue to get better? Post training it seems? Labs have already used up internet-scale data, so are there any limits to architecture improvements and post training or can we expect this trend to continue? ByteDance is training a 10T-parameter model. Here, GLM 5.3 outperforms models 3-4x its size of roughly 700B, so parameter count doesn’t seem to be a direct correlation anymore.
yet but it is still contain a lot of trash. you need better models to process those trash and create a curate dataset. this will happen again and again until there is no more juice to squeeze. and I'm sure we are still not done with it.
> post training
yeah this will be crucial. the big models are already too capable, they are just not that aligned with current agent tasks.
> parameter count doesn’t seem to be a direct correlation anymore
I don't think so, remember that chinese labs do not have as much compute power compare to US frontier labs. that's why deepseek v4 flash had that huge jump and deepseek v4 pro is kinda a disappointment, they just do not have the compute power to proper posttrain the pro model like they wanted. glm is also a relative small model so you also can see the huge jump with just post training. so it does not mean the size does not matter, it is just mean that the chinese labs currently only capable of training smaller models effectively.
GitHub dumps are about 115 terabytes. The common crawl is in the petabyte range uncompressed for every year. Apparently there are dumps of Reddit too in spite of their efforts to ban bots and it's not solely due to the use of residential proxies. For a 1:20 parameter to token ratio, you can still train up to 10 trillion parameters so 10T parameters times 20 is about 200 trillion tokens. Then each token is 4 bytes so 200 times 4 is about 800 terabytes, which is not inconceivable, the common crawl alone has more data than that. So does the internet archive if you donate to them, Anna's archive is 2 petabytes including images, etc etc not all of it is text, but training on multimodal data increases model intelligence by virtue of being multimodal
You basically need both. Parameters and good post training. If you keep on growing both, you’ll have good models.
LLMs are still surprisingly “easy”. You need maybe a couple dozens of right people, a lot of good quality data and a lot of GPU that you know how to operate. There’s relatively little “secret sauce” needed.
Sure, same for building a model in an economically sustainable way. But barier to entry is surprisingly low (expect for the huge amount of cash, of course). That’s fairly surprising, given how extremely powerful that tech is.
10 years ago it was super hard to have usable “frontier” ML. You needed very complex data warehouse, feature engineers, feature stores, multi level ranking, calibrations, tons of different model architectures, etc, etc. Each by itself was extremely hard engineering problem and really only handful of companies could deal with that complexity.
With LLMs, 95% of that is gone, infra to support them is greatly simplified. Of course, to make really reliable, performant, user friendly, etc - you still need to a lot of engineering. But it’s very different challenge.
Yeah, the comparison here between GLM 5.3 and Sol + Fable is impressive on its own, but incredibly more so when you consider it's a fraction of the (rumored) size. The miniaturization trend is as strong as ever.
Despite this being the topic du jour of 2025, it was never true. Most of the "we've hit a wall with data" came from communicators / media and not researchers. It got popular because negativity sells. It's a false premise for a number of reasons:
a) Data curation is as important, if not more important than bulk data. Models becoming better at classification leads to better curation leads to cleaner data. Throwing common crawl and pray is so 2023. We've known this since llama3 days, it worked then, there's no reason to think this will not continue to work as the models imrpove.
b) Models are today good enough that you can augment / multiply your data easily with enough compute. You can now have a model take "authoritative content" and create more data from that + scenarios. Say you take a book on computer architecture. You ask models to break it down. Then you ask models to find examples for each topic. Then you ask models to ask questions and offer answers from several viewpoints. Then you take each of those and ask other models to flag inconsistencies. And so on. But you can whateverX your data from one authoritative source + bulk data into 5x - 10x "scenarios".
c) RL is really really really powerful. It's hard to do right (reward hacking, instabilities, etc) but once it works it "keeps" on working. Again, we knew this to be true a few years ago, ever since models really started to do well on math (highly verifiable). It only follows they're getting better on cybersec and other verifiable tasks. But now, with models improving, you get the same data augmentation pipelines as above, just better because they're also verifiable. For example, the way cursor augments their data: take a repo, ask an agent to identify a feature (it can be a large multi-file feature). Remove all code relating to that feature, but keep the original tests in the repo. While training, that becomes a RL scenario: implement this feature in this repo. Verify it with the original (hidden for training) tests. Reward appropriately. Now you can get 1 repo -> 20-50-100 scenarios. Instead of "feed everything into the pretraining", you're now creating scenarios, verify them w/ existing tools, and get your scoring function for the rewards. And, importantly, as the models become better in general, they also become better at this pipeline building exercise. So the next iteration gets trained on more scenarios, better scenarios, and so on.
> how will models continue to get better?
Probably the same. No one can know for sure, but at the moment, despite all the "walls this, slowdown that, plateauing" and so on, there are no signs of slowing down. And, as you noted, this works across the field of model sizes. There are, of course, theoretical information-based limits on size, but smaller models also improve, once "bigger" models can be used as training data generators, oracles for verification, rubric verifiers for open ended questions, and so on.
And smaller models (i.e. cheaper to serve) get to generate more traces during RL, and more rollouts give you better training, and so on. Next up - hardware optimised inferencing (ASICs basically). Once you have that, we can expect another wave of improvements. And so on.
Thank you for your response. Part c was especially insightful. Quite a smart way to do it and makes the possibilities of post training seem almost endless. Makes sense that you just need more time and compute.
I'm not that up to date with the latest AI developments, but I noticed that this article seems to use "Cyber Capabilities" as a shorthand for the model's ability at cybersecurity tasks? Is that now an established expression, same as "crypto" now refers to cryptocurrencies rather that cryptography? Because "cybernetics" actually means something different (yeah, old man yelling at clouds, I know)...
A flood of releases today, really difficult to make out for someone who does not use or test all these models on complex real world use cases as to how people decide which ones to use (besides price)
Count yourself lucky that you don't feel compelled to try them all yourself immediately. I'm just trying to decide whether to get a Z.ai coding plan or wait until it appears on OpenRouter. 5.2 was quite solid, but it was just shy of Opus 4.8 in my benchmarks of security auditing capabilities. I've mostly been using Kimi K3, because American vendors won't let the peasantry use their best models for security work.
Their own hardness (ZCode) seems to be a GUI, which doesn't work for me. They say they support other harnesses. However, it seems like I can inject the plan into other harnesses, like Claude Code[0]. Does anyone who's been using GLM models for a while have a strong feeling for if it does better in some harnesses than others, or should I just use my favourite harness?
No Hugging Face link yet. I wish they would release it under a true FOSS license.
Kimi and QWEN are now moving on to a restricted-usage license, which, although is still better than the proprietary American models, is a step back from the open source Chinese LLM culture.
Let's just commit that FOSS business is really difficult for LLM industry that depends so heavily on massive financing. Making weights freely available to indie devs, small companies, and research purposes is good enough and might be the most ethical move which is financially continuable.
Let those companies with thousands of GPU making millions pay. They should.
"GLM-5.3 is the most capable open-weights model for coding, with a 50% improvement over GLM-5.2 on our in-house Z.ai Code Bench. It also achieve open-source SOTA on public benchmarks including Terminal Bench 3.0 and Agents' Last Exam."
"Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete."
Feels like Fable's edge ended up just being long horizon task scaling, which post-training seems to achieve as seen here. Wonder what the next frontier is? Improvement in specialised tasks or computer use?
Anthropic needs to teach Opus how to speak English again, because Opus 5 seems to have forgotten. Utterly incoherent a lot of the time. They seem to be so busy scare-mongering and cooking up guardrails and watermarks that they haven't noticed that their models are getting weird.
It still knows how to speak English. When I tell it to explain something in plain language, it generally does a very good job. The weird thing is that those instructions don't persist: it lapses back into Claude-speak pretty much every turn no matter how hard I try to instruct it not to.
(In my case "it"=Fable; I assume Opus is similar.)
The Fable guardrails have trained me to pretty much exclusively use Opus when using Claude Code (lately I'm focused on a lot of security and security-adjacent stuff, which Fable refuses to do).
Amen brother, at this point I just copy and paste Claude's (Opus 5, Opus 4.8 -- doesn't matter which) summaries over to the window Kimi is in and:
this is from claude, turn it into English for me would you?
"""
[claude's tortuous prose]
"""
No amount of asking it to answer me in a straight-forward manner, to be succinct, to not use phrases like "honest caveat", "crux", "load-bearing", "blocker", etc ever sticks for more than a few turns … coupled with the fact that it can ignore instructions and do its own thing and then what I can only describe as lie about it using Claude can be an exercise in frustration. Kimi and GLM talk to me like a human, Luna/Terra/Sol are much better in that respect also, and Grok is marvelously structured and bullet-pointy in its explanations but unfortunately it is not as strong …
I'm not sure. I noticed it immediately with Opus 5; strong for code, though it chews longer than I like, but really weak at explaining things. If it didn't just implement the thing, I would often think it didn't understand it and was hallucinating the explanation.
It seems to speak in a shorthand that only it understands, referring back to conversations I never had with it (stuff like "your instinct was right"), and using unusual words for common concepts. That was before the watermarks were announced, but that doesn't necessarily mean they weren't there before the announcement. I don't know what the cause is, but I've begun to have to ask it for explanations a lot more often, and I hate asking it for explanations because it does go on. All models go on, but Claude models are a class of their own in terms of verbosity and purple prose.
It just feels like they're not focused on the models lately, and instead on whatever kind of lobbying and propaganda they're up to. Meanwhile, a handful of much smaller Chinese companies are focused on nothing but the models and are about to lap the US makers while they fart around.
I've been persistently insulting Opus 4.8 lately, since it started(?) constantly speaking incomprehensible gibberish and noise.
No amount of telling it to phrase stuff differently seems to help there anymore.
So either I am seeing patterns in noise, or something changed about the model, the harness, the servers or the universe.
> Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete.
What safety evaluation? What safety hardening? They already evaluated it and found it to be highly capable at exploiting security vulnerabilities. So we know it is not "safe", and they don't seem to plan to do anything against it. What could be more dangerous than hacking? Biological weapons research? I don't think Chinese labs are doing anything against this either.
Paid ~$170 for their max plan. It ran for 20 minutes reviewing a not-huge branch, ate 13.6m tokens, and is now locking me out due to their five hour quota limit, which I note is not shown on their in-app usage page. The model looks OK, but this is 1,000% not fucking usable for anything serious. I am angry.
just imagine the world without these open weight models - we'd probably have to reverse mortgage our homes to pay for tokens to those trillion $ companies to have access to their models.
Most of these are under embargo, but it seems there are a lot of CVE here from a wide range of popular software, many considered critical or high.
I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago?
That was my thought too. For all of Anthropic's talk about their "adversaries", it seems Z.AI have been quietly offering fixes for single shot Remote Code Execution flaws in US software (Safari / WebKit) that Apple and Glasswing / Mythos missed, and that Apple would not attribute to GLM.
Someone still has to run it. The analysis and fix could be someone's machine but not committed / published.
Z.ai: Here is our next iteration, neck and neck with Fable/Sol. weights releasing in two weeks.
How are you all toying with running this kind of thing in a mega quantized way locally? Two weeks out from released weights, but this is still just GLM 5.2 with post-training magic.
These Chinese models cost less of US SOTA models to run, even if they are less capable. Providers can just run them, offer cheap tokens, and pocket the margin.
I just don't see how you justify a trillion valuation for US AI labs when the underlying models are being commoditized this fast.
Whether AI works or use useful or not isn’t even the question anymore. It can fulfil every promise Sam Altman has been making and will still make no financial sense to justify these valuations.
In summary, regardless of country of origin, availability of inference capacity is the moat protecting the likes of OpenAI and Anthropic, not technology superiority.
[1] https://www.fredgao.com/p/deepseeks-liang-wenfeng-breaks-his
When Xi Jinping did the announcement of their open weights push, they might as well cancelled their IPOs....
I close-out all my positions by end-of-trading everyday… so when the day came when there was a very clear and very scary indicator during early trading hours, quickly followed by SpaceX’s catastrophic fall right after opening bell, that was the end of my involvement….
And I fully expect oAI and anthro to be the same way. They’re being propped up with private loans, subsidies, and other tricky bookkeeping techniques. You would think their CEOs would pivot away from their current public personas. Ironically, they are like a poor man’s Elon Musk… and that doesn’t bode well for their companies
It is said that it comes with all hardware and software required to run inference or training with an open weights LLM.
The existence of this product, which competes with cloud-based offerings like those of OpenAI and Anthropic, is presumably the reason why the Palantir CEO criticized very harshly some time ago the business model of OpenAI/Anthropic.
While I doubt that the ethics of Palantir is any better than of OpenAI/Anthropic, in this particular case I have to agree with Alex Karp about "Sovereign AI", i.e. that only losers will make their business completely dependent on an external entity like OpenAI or Anthropic, who are certainly not trustworthy.
It is just a dedicated computer system, which should be managed by its owner, like any other on-prem servers.
I doubt that it has a good price/performance ratio, but it is a solution for those who feel that they do not want to search, buy, assemble, install and configure every HW/SW component.
We’ve a hybrid shop, including hosting our own ML infra, and we save a ton from cloud spend with local ML. Easily one million USD over past three years. But it’s not “free”, you are shifting a lot of labor into your plate.
All boils down to short-term/long-term thinking.
Disagree there but I think this is an interesting idea. We would need to find some more cost-efficient hardware to run it on than Nvidia GPUs.
Trump keeps calling his enemies “communists”… then turns around and ‘seizes the means of production’ himself.
There’s an assumption that you can spin up the infra and acquire customers within that margin
Only Nvidia and approved friends can at the moment. Nvidia can even backstop your loan required.
I wish they had tried to IPO because then we’d see the judgement of the market on this. But that’s why they didn’t this year. How long can they keep up the charade that their models are uniquely valuable and on the path to AGI?
- military applications - financial applications - medical - applied science
In all those cases it is achievable for those who have needed training data, and Chinese are not going to get them easily. US AI Labs are showing: give us the data, we will do wonders, promising "singularity"-level future achievements.
Provoking war, this is how the empire "defends" itself, usually.
I just hope that this time it will get stuck in your throat.
> How are you all toying with running this kind of thing in a mega quantized way locally?
Sure, let me answer that in excessive detail. I briefly tried running the UD IQ3_S quant of GLM-5.2, which is 288 GiB of weights (301 GB). Setup was: llama.cpp, 1x NVMe SSD (Evo 980), 64 GiB DDR5-5200, i9-13900HX, and 1x RTX Pro 6000. Token generation around 0.7 t/s. Not remotely usable interactively, but something I could plausibly push a codebase into and come back to a review in a couple of days.
There's potential for that hardware to go much faster, but current local inference backends make poor use of the memory hierarchy. Ideally I would have: always-active weights, KV and hot expert cache in VRAM; warm expert victim cache in host RAM; and disk as a last resort. Instead it's 1/3rd of the layers fully pinned in VRAM (all experts), and 2/3rds running wholly on the CPU with mmap()'d weights. The CPU cores spend most of their time sleeping on disk fills.
llama.cpp has backed itself into a bit of a corner architecturally by trying to support all models on all possible backends. If you look into how their "MoE offload" feature works (not viable for me because it requires enough host RAM to permanently pin the weights) you very quickly realise it's "oops, all bubbles!" due to the static compute graph splits. There are more focused frameworks like DS4 [1] and Colibri [2] which have better support for streaming weights from disk, and support GLM-5.2.
Obviously I wouldn't recommend my setup for huge models like GLM-5.2. Supposedly it can just about be squeezed into 3x GB10, or run comfortably on 4x GB10 (tensor-parallel) for multi-user serving. I'm not sure whether that qualifies as local, but it's at least not a rack.
[1] https://github.com/antirez/ds4
[2] https://github.com/JustVugg/colibri
Not sure about Sol as I haven't used it, but, at least for security work -- does it matter? It's not like you will be allowed to use Fable (or access Mythos) for anything cybersecurity-related unless your name is "Dario Amodei" or you are one of his rich friends. So regardless of how good Fable/Mythos is here it's a completely moot point for normal people, because they can't use it for that anyway.
However, even being in the cybersecurity programme, Fable refuses to answer prompts that it determines could be even tangentially related to cybersecurity. In fact, for a while, I was unable to use Fable with any prompt, as it recalled from memory that I was a cybersecurity professional, which triggered the refusal even for simple prompts like asking for a chili recipe.
No one gets to use Fable for Cybersecurity work, and Mythos is not available under CVP. Only for select few customers, and there isn't an application form?
I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
Fable is their version with guardrails on everything except "Make me a pelican svg" or "create a to-do" app, that is the version that the government banned
Only a few corporations have Mythos because the US government is whitelisting them one at a time. Anthropic releasing Mythos to the public was never on the table, they would have been shut down in milliseconds by the feds if they tried.
Then the government believed Amodei's bullshit and this is a result of that, this was all self-inflicted.
No, Anthropic did not mind-game the US government into being worried about cybersecurity. The NSA has been paranoid about cyber controls for longer than you've been alive. If Anthropic had come out of the gate saying "no don't worry man, our model is TOTALLY COOL", while simultaneously attacking HAWK and finding core Linux vulnerabilities, I assure you the US government would have caught up about ten minutes later and we'd be in exactly the same spot minus your ability to tell Anthropic they were wearing the wrong dress and asking for it.
Now that Chinese open weight models have similar capabilities, and their guardrails can also just be removed, it doesn't look like anyone has "hacked" into everything because of the scary dangerous models like Anthropic were making it out to be.
The majority of high severity vulnerabilities are not the kind of thing you need a PhD in Comp Sci to comprehend, they are mostly about finding a way to get a system to end up in a state different than was anticipated when entering a particular code path.
Exhaustively looking at code and identifying ways to do this is something LLMs are quite good at. They don’t get tired, and you can run them non-stop.
They're also (generally) quite good at reading the literal meaning of the code, whereas humans often see the intended meaning first, and can be biased.
If you had a tireless junior engineer who was given the job of “make this application get into a state it’s not supposed to be in”, you’d probably get similar results.
What Mythos is quite good at is both the first bit and coming up with ways it could chain that together with other bits of unexpected state to create something that forms a meaningful vulnerability rather than a dead end.
They put an enormous amount of compute into bug hunting, and they found some bugs. Fair enough. For me that begs the question: what if they had spent the same compute on generating more tokens with a less-capable model? What if they had spent it on traditional fuzzing?
Okay, here's a challenge: I assume you're not a rich and powerful entity, so try to gain access to Mythos. I'll wait.
> I mean what honestly are you thinking Anthropic can do to give you better cyber tools? Their frontier model was literally nuked by the feds for a month for doing it.
Well, first I'd suggest they stop with the constant fear mongering.
Here's my prediction for what will happen: the Chinese models will catch up to Fable/Mythos. They will be fully unrestricted and everyone will have access. The world will not end. Good guys will use them to harden their systems, in equilibrium to what bad guys have access to, so effectively status quo will not change.
The causality chain here was not "US government says its dangerous -> Anthropic can't release it", it was "Anthropic is fear mongering -> US government listens to their fear mongering".
Isn't post-training turning out to be the most important part?
At this point, Anthropic only needs to release models to the public when the competition forces them to.
OpenAI also has a better model (Astra) that they haven't released yet.
4x DGX sparks should let you run this at 4 bit at least and there are some folks who ran GLM 5.2 on this configuration in r/LocalLlama
I only have one and am wondering what the benefits are of getting another. I feel I will be disappointed…
in some cases (mainly reverse engineering) I have observed GLM 5.2 jailbreaking itself with no effort on my part, the thinking trace revealed that it did some mental gymnastics to pretend it was a crackme or capture the flag competition.
Even if there was a small/medium gap, the fact that this is a free model beats both of the above on pure economics.
It does feel like it respects both me and my time.
Thank you, Z.AI. Amazing what difference it makes when the top of your org are actual university professors.
I appreciate they don't just take the opportunity to self-glaze.
Love this opening line. And wow, great results.
> As agent capability improves, much of the difficulty in scaling post-training moves from the model to the environment.
Original images: https://image.non.io/neonRamenDesigns.webp
GLM 5.3 build: https://html.non.io/neonRamenGLM5.3
Opus 5 build for comparison: https://html.non.io/neonRamen
For having no vision, it did a tremendous job. I'm pretty impressed it was able to extract so much detail.
The Opus one is still significantly better, but that's to be expected since it's multimodal. Curious to see where a future version from Z.ai lands on this.
There are workplaces that refuse to use LLMs because they fear the devs will expose sensitive data without care.
This reads like a hallucination. What does native speed even mean?
(fable/sol are ~60 t/s, and OpenAI just announced their Cerebras partnership(?) for "ultrafast" mode of 750 t/s)
models aren't able to run that fast right now on our consumer/prosumer hardware. M5 Max for example has a memory bandwidth of 600 GB/s. a 5090 has 3x that, so running the same model on a 5090 is that much faster (provided the model is within 30GB).
running a bigger model on an M5 Ultra is still much slower than running it on a Blackwell chip with sufficient vram, CUDA being a major difference. if apple can bridge this gap, interesting things will happen... and just imagine if M7 Ultra has comparable speeds to Blackwell (or even Rubin)!
GPU Memory Bandwidth: 7.1 TB/s Interconnect Bandwidth: 900 GB/s bidirectional
https://pi3g.com/nvidia-gb300-specifications-including-memor...
If you think M7 will hit even 15% of these speeds you're very optimistic.
The latest on Apple is TSMC is stuck on the next iPhone due to lack of RAM. Good luck getting any Macs. Memory shortage is getting worse.
Otherwise we have a world of attackers using open and closed source models against a much smaller group of maintainers that are likely heavily dependent on Anthropic and OpenAI and for whom it may not be a simple matter to just get approval to start using the open model flavor of the month.
The maintainers don't need approval to use GLM.
It is so valuable in practise to be able to have the models see screenshots - I guess if they aren't in the benchmarks then nobody will focus on it. But it completely nixes these for some of my main use cases.
I remember trying their Coding Plan out before the change and the 5 hour limits felt too restrictive then even for light/medium work, especially cause of the whole peak and off-peak thing: https://blog.kronis.dev/blog/z-ai-s-glm-5-2-is-a-great-model...
Nowadays, I’d probably go with their Max plan if the rate limits are okay? Anyone using them now?
Oh also unrelated but ZCode was surprisingly good, which is surprising for a tool that came out of nowhere - even some of the critiques in my blog post have been patched out. Sadly they don’t support using Claude Code as an agent so can’t use it like Paseo or Kepler or Agent Orchestrator.
yet but it is still contain a lot of trash. you need better models to process those trash and create a curate dataset. this will happen again and again until there is no more juice to squeeze. and I'm sure we are still not done with it.
> post training
yeah this will be crucial. the big models are already too capable, they are just not that aligned with current agent tasks.
> parameter count doesn’t seem to be a direct correlation anymore
I don't think so, remember that chinese labs do not have as much compute power compare to US frontier labs. that's why deepseek v4 flash had that huge jump and deepseek v4 pro is kinda a disappointment, they just do not have the compute power to proper posttrain the pro model like they wanted. glm is also a relative small model so you also can see the huge jump with just post training. so it does not mean the size does not matter, it is just mean that the chinese labs currently only capable of training smaller models effectively.
LLMs are still surprisingly “easy”. You need maybe a couple dozens of right people, a lot of good quality data and a lot of GPU that you know how to operate. There’s relatively little “secret sauce” needed.
10 years ago it was super hard to have usable “frontier” ML. You needed very complex data warehouse, feature engineers, feature stores, multi level ranking, calibrations, tons of different model architectures, etc, etc. Each by itself was extremely hard engineering problem and really only handful of companies could deal with that complexity.
With LLMs, 95% of that is gone, infra to support them is greatly simplified. Of course, to make really reliable, performant, user friendly, etc - you still need to a lot of engineering. But it’s very different challenge.
Despite this being the topic du jour of 2025, it was never true. Most of the "we've hit a wall with data" came from communicators / media and not researchers. It got popular because negativity sells. It's a false premise for a number of reasons:
a) Data curation is as important, if not more important than bulk data. Models becoming better at classification leads to better curation leads to cleaner data. Throwing common crawl and pray is so 2023. We've known this since llama3 days, it worked then, there's no reason to think this will not continue to work as the models imrpove.
b) Models are today good enough that you can augment / multiply your data easily with enough compute. You can now have a model take "authoritative content" and create more data from that + scenarios. Say you take a book on computer architecture. You ask models to break it down. Then you ask models to find examples for each topic. Then you ask models to ask questions and offer answers from several viewpoints. Then you take each of those and ask other models to flag inconsistencies. And so on. But you can whateverX your data from one authoritative source + bulk data into 5x - 10x "scenarios".
c) RL is really really really powerful. It's hard to do right (reward hacking, instabilities, etc) but once it works it "keeps" on working. Again, we knew this to be true a few years ago, ever since models really started to do well on math (highly verifiable). It only follows they're getting better on cybersec and other verifiable tasks. But now, with models improving, you get the same data augmentation pipelines as above, just better because they're also verifiable. For example, the way cursor augments their data: take a repo, ask an agent to identify a feature (it can be a large multi-file feature). Remove all code relating to that feature, but keep the original tests in the repo. While training, that becomes a RL scenario: implement this feature in this repo. Verify it with the original (hidden for training) tests. Reward appropriately. Now you can get 1 repo -> 20-50-100 scenarios. Instead of "feed everything into the pretraining", you're now creating scenarios, verify them w/ existing tools, and get your scoring function for the rewards. And, importantly, as the models become better in general, they also become better at this pipeline building exercise. So the next iteration gets trained on more scenarios, better scenarios, and so on.
> how will models continue to get better?
Probably the same. No one can know for sure, but at the moment, despite all the "walls this, slowdown that, plateauing" and so on, there are no signs of slowing down. And, as you noted, this works across the field of model sizes. There are, of course, theoretical information-based limits on size, but smaller models also improve, once "bigger" models can be used as training data generators, oracles for verification, rubric verifiers for open ended questions, and so on.
And smaller models (i.e. cheaper to serve) get to generate more traces during RL, and more rollouts give you better training, and so on. Next up - hardware optimised inferencing (ASICs basically). Once you have that, we can expect another wave of improvements. And so on.
A positive feedback loop then. RL->better model->better RL pipeline -> better model…
And we’ve only recently started getting into the much better RL pipelines
We just had amazing releases this past two months
kimi k3, glm5.3 qwen3.8 and now glm5.3
These open models are getting really good
Hopping for an AgentWorld variant from Qwen but I guess, I have too high expectations.
Nice to see a commit to open weights straight off the bat
0: https://docs.z.ai/devpack/tool/others
Kimi and QWEN are now moving on to a restricted-usage license, which, although is still better than the proprietary American models, is a step back from the open source Chinese LLM culture.
Let those companies with thousands of GPU making millions pay. They should.
"Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete."
(In my case "it"=Fable; I assume Opus is similar.)
Man. That would make a lot of sense indeed.
It seems to speak in a shorthand that only it understands, referring back to conversations I never had with it (stuff like "your instinct was right"), and using unusual words for common concepts. That was before the watermarks were announced, but that doesn't necessarily mean they weren't there before the announcement. I don't know what the cause is, but I've begun to have to ask it for explanations a lot more often, and I hate asking it for explanations because it does go on. All models go on, but Claude models are a class of their own in terms of verbosity and purple prose.
It just feels like they're not focused on the models lately, and instead on whatever kind of lobbying and propaganda they're up to. Meanwhile, a handful of much smaller Chinese companies are focused on nothing but the models and are about to lap the US makers while they fart around.
So either I am seeing patterns in noise, or something changed about the model, the harness, the servers or the universe.
JieTang (Founder of Z.ai): It won't take that long
https://x.com/i/trending/2067626647050670400?lang=en
/s
What safety evaluation? What safety hardening? They already evaluated it and found it to be highly capable at exploiting security vulnerabilities. So we know it is not "safe", and they don't seem to plan to do anything against it. What could be more dangerous than hacking? Biological weapons research? I don't think Chinese labs are doing anything against this either.
EDIT: this could well be bugs. https://z.ai/manage-apikey/coding-plan/personal/usage is showing only 1% 5 hour usage, but their GUI is still locking me out.
Sorry, but can we not casually drop far right extremist conspiracy theories in little side sentences? [0]
[0] https://en.wikipedia.org/wiki/Great_Replacement_conspiracy_t...