Muse Spark Open is an 816B total/42B active parameter open weights model with a context window of 1M tokens. Built for coding and agentic work, Muse Spark Open is natively multimodal.
Don’t bother with their download restrictions, just check the quant providers https://huggingface.co/models?other=base_model%3Aquantized%3Ameta-models%2FMuse-Spark-Open-816B-A42B
I wonder if they’ve hidden a call home functionality in the model to steal all our data. Knowing Meta, it’s likely.
Hey they actually released them this time!
Fantastic! Still can’t do shit with 816B parameters
Continued scaling is quietly hilarious when what people would obviously love is a narrower focus on coding and/or reasoning. The minimum viable product speaks C but not hieratic Egyptian. An agentic model probably should not be graded on its storytelling abilities. It’s just bolting on a lot of cruft instead of improving the capabilities of what mere mortals can run.
On how many gb does such a more run?
Unsloth has their Q4 quant at 506 GB 💀
Active parameters aren’t that high, you’ll need KV cache in VRAM too, so maybe it would run in 64GB VRAM
