> Runs on a compatible personal laptop, such as a MacBook Pro
I'm curious, do normal people (so not HN professional developers or entrepreneurs) have that, as actually personal device (not from their tech company)?
I imagine some might have but I feel, and that's me so I'm asking the broader community here, that the average PC user has at best a gaming rig, but that's already a minority, and a lot more often a low end laptop with 16GB of normal RAM, not unified or VRAM. Am I out of touch?
Their GitHub repo and HuggingFace pages directly state that it's based on Qwen3.8 Flash Next. Their inference engine is called edge0-strata because it's a strata fork.
They fine tuned it and tweaked the architecture.
The real question is if the model performs better on general purpose tasks, or if it was just fine tuned to increase the benchmark scores.
Fair enough. I was just responding to the "just a quantized version of 3.8 next flash" claim. Benchmaxxing via fine-tuning is a very plausible possibility.
> Sharing matrices substantially reduces the weights that must be stored and moved while keeping the expert count unchanged. Each group of 64 experts shares two large matrices, while each expert retains a small set of parameters that captures its differences.
145B parameters with only peak 29GB memory usage - really impressive, truly something you can actually run on a decent spec MBP
I am not in the material, so perhaps someone else can comment on the quality of the outcome and technique? After skimming a page I am not really sure that the whole model fits inside 29 GB RAM. They seem to use some interesting techniques.
You have roughly a factor 8 in memory usage from 32 bit down to 4 bit precision. That is quite some room already, but not new.
It seems that they radically reduce the model information required per token via the sparse expert approach - 145B->10B parameters, probably at lower precision - and then use some other approaches I do not understand at that point to make effective use of loaded information.
In the "Under the Hood of Edge0" section it mentions that it needs another 40 GB of system memory for experts plus 48 GB of SSD storage for vocabulary (engrams?). The 29 GB headline seems to just be a flat lie (/the LLM writing this article getting confused about what memory usage it's reading).
Oh that’s annoying, from the hype I was hoping my 48GB MBP would be able to run this happily, but if we’re talking about -70GB combined RAM footprint then that’s not going to cut it.
> For Edge0-145B, day-one support covers macOS, Windows, and Linux, including Apple M-series chips and NVIDIA platforms such as RTX Spark, RTX 4090, and RTX 5090. The release includes one-click deployment tools and hardware guides.
But... there's no guidance on how to run on a mac at all. Weird.
I'm really impressed with the Blender examples. It's amazing how far "tool use" has come, when it now means using a tool as complicated as Blender to perform work that would take a human days to complete. I'm a little skeptical the actual models are workable, in the sense that a human could come in after and manually refine and apply better textures, etc. Historically, agent-made designs are messy as hell, and by the time you clean them up, you haven't saved any time. But, that's begun to get better; the best models are now very good at making clean shape-based SVGs, for example, that are no messier than what a human would make in Inkscape or Illustrator. But, I'll definitely give it a try.
I just gave Fable 5.2 a shot at placing and routing a couple of circuit boards. I'm actually sure it would work fine for simpler/sparser designs but when the component density gets high enough, Fable spends too long struggling in the layer of the layout it's in rather than doing what a human would do, and intuiting that placement, or a component footprint, or different part choice altogether would enable the work to continue.
At first the work looked awesome, and once I started checking it out, I realized I would have to significantly alter the placements and manually draw most of the schematic to get it up to my standards (and hit the performance criteria of the board).
Yeah, I still expect to do a lot of manual work with this kind of thing. By "this kind of thing", I mean any kind of complex visual thing in SVG. I haven't tried any kind of 3D or circuit layout, yet, as it's just not something I do often, but if models could do it, I probably would do more of it...I have a 3D printer, but too slow at 3D modeling to ever want to do anything custom, and I do electronics projects occasionally, but have never had my own boards printed for the same reason. But, I would expect the capability of the models for 3D and circuits to trail SVG, simply because there's certainly less training data and 3D can be wildly more complicated than the average SVG.
Do they say how they interact with Blender? Launched within a python interpreter everything in Blender is accessible as a method call. I hope they didn’t do anything more complex than that.
Is this a derived from flash next? That's the impression I get from reading over everything. Using Strata + Qwen 3.8 Flash Next now and I am beyond impressed. It's basically as capable as 5.6 Luna in my hands so far. If this is an upgrade I can't wait to try it out.
I assume so - Qwen/Qwen3.8-Flash-Next is marked as the "Base Model" on the huggingface page and states it may be subject to "Qwen's separate authorization or attribution requirements"
And I assume the metrics comparing qwen are using a quantized version - from the blog they compare memory use against qwen's "standard 4-bit deployment". Which might explain the improved performance over that base model they show - it might lose less from quantization.
It's interesting. It does seem to be based on QFN, but they've modified the architecture. 10B active versus QFN's 6B, plus also QFN is 180B parameters, not 145B. But it has the same 51B n-gram table.
Also, why ship both a 145B and a 120B size? That's basically the same size class.
> Using Strata + Qwen 3.8 Flash Next now and I am beyond impressed.
The inference engine they use is called edge0-strata because it's a strata fork.
This project looks like someone fine-tuned Qwen3.8 Flash Next, made some tweaks to the model, forked strata to run it, and then put a lot of marketing and branding on top of it
3.8F/N is amazing. Truly useful, runnable local AI on machines that are not ridiculously expensive. (I’m running it on 2x 3090 and 128gb of ram, on the quantum roughly equivalent to 5-K-M. fast, but you can run it on much less
This looks like it would actually work on my 64mb M1 MBP. That would be fantastic.
Thy have shipped this as NVFP4 so its Nvidia blackwell only, presumably their inference engine doesn't support anything outside CUDA 13 specified either. Will be interesting to see if this gains some other quants and other inference engines pick it up and we get some standard GGUF files, its a bit of weird release at the moment rather than GGUF file downloads the variants link on HF is leading to a directory of stuff which contains the model spread across a variety of files.
It is strange that they also benchmarked an M2 Ultra, but they are specifically not releasing a Mac runtime today. Maybe that is still under development.
It's amazing the level of optimizations we are discovering. At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications
Over the past few weeks, I've been putting together a small, 6.5M param model. It is conversational, does tool calling, RAG, mathematical modeling/calculations, basic physical simulations, and multi-step chained reasoning. It's still very much _not_ a frontier model. But it already does more than I could get a qwen 2.5 0.5b param model to do with finetuning just 1 year ago
Most of the improvements come from being able to direct claude to apply different ideas from recent research that keep coming out. AI labs seem to be focused on 1B+ models, so not much comes out about small models, but many of the things being discovered in research with billion+ model, are adaptable to small ones as well
> At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications
Been reading this for years now. Seems both, bigger models optimized for scale AND smaller models optimized for single machines, keep on being released. I don't think it's changing, different markets.
Has anyone figured out the macOS situation with this? The blog post mentions Mac repeatedly, saying things like "day-one support covers macOS". But the released stuff so far seems to be CUDA only. I see no info on when and if the actual Mac support is coming.
This weird approach is frustrating, because the actual promise sounds very intriguing, so I'd very much like to test this out.
Harness improvements like context pruning, malformed tool call healing also very helpful to boost smaller local models. Doing alot of work in this area lately.
Qwen 27B in Terminal Bench 2.1 73->85 for instance.
TerminalBench scores often correspond to real world performance for coding agents that rely heavily on bash.
This score is roughly on par with 5.6 Terra max, which in my direct experience is perfectly capable of driving a cloud agent harness that can do pretty complex knowledge work.
For those of us with slow SSDs but a lot of RAM (relative to this model's file size) would this benefit from the extra memory, loading all in, or would I need to set up an old school RAM disk to prevent the swapping?
I’m admittedly not an expert, I’ve not had the time or energy to try out many of these new American open weight models.
I’ll just say this, we obviously all know they (American open weight models) are playing catch-up to the competition. So if your reaction is “why would I use this over Qwen this, or Deepseek that” you’re really missing the mark.
Yeah, they are all going to be behind on benchmarks for now. Nobody realistically expects some American open weight model to come out of nowhere and beat Qwen/ Deepseek/ et cetera. But there has been a flurry of new releases from this segment, and it’s interesting to me to watch how it develops.
Based on Qwen3.8-Flash-Next and Strata. They made some interesting tweaks and did some fine tuning on the model, but it's interesting how much marketing speak was layered on their website to hide this fact.
It's not at all unusual for people to fine tune models, tweak them, and fork inference engines. It's highly encouraged and appreciated in the community. What is unusual is seeing someone wrap this all up in extensive branding and publish it online as a company.
As with all fine-tunes, I'd be a little skeptical of the improved benchmark numbers until people actually use it more. It's really easy to take off the shelf models and fine tune them to get slightly higher benchmark scores without really improving them for normal use. HuggingFace is full of fine tunes that advertise amazing results on specific benchmarks but are worse than the base model when you use them for real work.
I am once again impressed by the ability of language models to generate huge amounts of relevant-looking text (and diagrams now!) while actually communicating as little as possible.
Interesting, if 3rd party testing and benchmarking hold the published values this is very exciting. I am a bit skeptical given the model size, but I hope it turns out to be true!
So did Strata Vs Llama.cpp, yet Strata pulls 2.5 to 3x the performance of the Llama.cpp codebase it was based on.
If they've improved the architecture further by sharing experts, then this might be plausible.
I'm curious, do normal people (so not HN professional developers or entrepreneurs) have that, as actually personal device (not from their tech company)?
I imagine some might have but I feel, and that's me so I'm asking the broader community here, that the average PC user has at best a gaming rig, but that's already a minority, and a lot more often a low end laptop with 16GB of normal RAM, not unified or VRAM. Am I out of touch?
Their GitHub repo and HuggingFace pages directly state that it's based on Qwen3.8 Flash Next. Their inference engine is called edge0-strata because it's a strata fork.
They fine tuned it and tweaked the architecture.
The real question is if the model performs better on general purpose tasks, or if it was just fine tuned to increase the benchmark scores.
145B parameters with only peak 29GB memory usage - really impressive, truly something you can actually run on a decent spec MBP
ModelTokenMemoryUsage [Bytes] = Parameters * (BitsPerParameter/8)
For a model with 145B parameters, this means:
You have roughly a factor 8 in memory usage from 32 bit down to 4 bit precision. That is quite some room already, but not new.It seems that they radically reduce the model information required per token via the sparse expert approach - 145B->10B parameters, probably at lower precision - and then use some other approaches I do not understand at that point to make effective use of loaded information.
Did anyone try the model?
Still very interesting though.
> Edge0-145B scores above comparable models with less than half the memory (29 GB vs ~70 GB) and runs on a 48GB laptop
So either they’re lying or confused or I’m confused.
They're not hiding that fact once you get to the repo. I mean it's called edge0-strata and the README says it's a fork of strata.
It's weird to see this marketing web page imply that it was all new and novel work that they did, though.
> For Edge0-145B, day-one support covers macOS, Windows, and Linux, including Apple M-series chips and NVIDIA platforms such as RTX Spark, RTX 4090, and RTX 5090. The release includes one-click deployment tools and hardware guides.
But... there's no guidance on how to run on a mac at all. Weird.
https://github.com/Edge0-AI/Edge0/tree/main#platforms
At first the work looked awesome, and once I started checking it out, I realized I would have to significantly alter the placements and manually draw most of the schematic to get it up to my standards (and hit the performance criteria of the board).
And I assume the metrics comparing qwen are using a quantized version - from the blog they compare memory use against qwen's "standard 4-bit deployment". Which might explain the improved performance over that base model they show - it might lose less from quantization.
https://huggingface.co/Edge0/Edge0-145B-NVFP4
Also, why ship both a 145B and a 120B size? That's basically the same size class.
There is also a reference to a 8B
https://github.com/Edge0-AI/Edge0/tree/main/docs/models
https://github.com/Edge0-AI/Edge0/blob/main/licenses/edge0-m...
The inference engine they use is called edge0-strata because it's a strata fork.
This project looks like someone fine-tuned Qwen3.8 Flash Next, made some tweaks to the model, forked strata to run it, and then put a lot of marketing and branding on top of it
This looks like it would actually work on my 64mb M1 MBP. That would be fantastic.
It is strange that they also benchmarked an M2 Ultra, but they are specifically not releasing a Mac runtime today. Maybe that is still under development.
Over the past few weeks, I've been putting together a small, 6.5M param model. It is conversational, does tool calling, RAG, mathematical modeling/calculations, basic physical simulations, and multi-step chained reasoning. It's still very much _not_ a frontier model. But it already does more than I could get a qwen 2.5 0.5b param model to do with finetuning just 1 year ago
Most of the improvements come from being able to direct claude to apply different ideas from recent research that keep coming out. AI labs seem to be focused on 1B+ models, so not much comes out about small models, but many of the things being discovered in research with billion+ model, are adaptable to small ones as well
Been reading this for years now. Seems both, bigger models optimized for scale AND smaller models optimized for single machines, keep on being released. I don't think it's changing, different markets.
This weird approach is frustrating, because the actual promise sounds very intriguing, so I'd very much like to test this out.
This score is roughly on par with 5.6 Terra max, which in my direct experience is perfectly capable of driving a cloud agent harness that can do pretty complex knowledge work.
Definitely worth an evaluation!
I’ll just say this, we obviously all know they (American open weight models) are playing catch-up to the competition. So if your reaction is “why would I use this over Qwen this, or Deepseek that” you’re really missing the mark.
Yeah, they are all going to be behind on benchmarks for now. Nobody realistically expects some American open weight model to come out of nowhere and beat Qwen/ Deepseek/ et cetera. But there has been a flurry of new releases from this segment, and it’s interesting to me to watch how it develops.
Based on Qwen3.8-Flash-Next and Strata. They made some interesting tweaks and did some fine tuning on the model, but it's interesting how much marketing speak was layered on their website to hide this fact.
It's not at all unusual for people to fine tune models, tweak them, and fork inference engines. It's highly encouraged and appreciated in the community. What is unusual is seeing someone wrap this all up in extensive branding and publish it online as a company.
As with all fine-tunes, I'd be a little skeptical of the improved benchmark numbers until people actually use it more. It's really easy to take off the shelf models and fine tune them to get slightly higher benchmark scores without really improving them for normal use. HuggingFace is full of fine tunes that advertise amazing results on specific benchmarks but are worse than the base model when you use them for real work.
Damn, guess I can't run this on my desktop. /s