Muse Spark Open is an 816B total/42B active parameter open weights model with a context window of 1M tokens. Built for coding and agentic work, Muse Spark Open is natively multimodal.
Don’t bother with their download restrictions, just check the quant providers https://huggingface.co/models?other=base_model%3Aquantized%3Ameta-models%2FMuse-Spark-Open-816B-A42B
Hey they actually released them this time!
Fantastic! Still can’t do shit with 816B parameters
Continued scaling is quietly hilarious when what people would obviously love is a narrower focus on coding and/or reasoning. The minimum viable product speaks C but not hieratic Egyptian. An agentic model probably should not be graded on its storytelling abilities. It’s just bolting on a lot of cruft instead of improving the capabilities of what mere mortals can run.
On how many gb does such a more run?
Unsloth has their Q4 quant at 506 GB 💀
Active parameters aren’t that high, you’ll need KV cache in VRAM too, so maybe it would run in 64GB VRAM
