suoko@feddit.it to LocalLLaMA@sh.itjust.worksEnglish · edit-28 days agoI dropped Qwen and Gemma for a 27B model that runs smoothly on my 8GB graphics card - Bonsai 27Bwww.xda-developers.comexternal-linkmessage-square20linkfedilinkarrow-up147arrow-down18
arrow-up139arrow-down1external-linkI dropped Qwen and Gemma for a 27B model that runs smoothly on my 8GB graphics card - Bonsai 27Bwww.xda-developers.comsuoko@feddit.it to LocalLLaMA@sh.itjust.worksEnglish · edit-28 days agomessage-square20linkfedilink
minus-squareFtumch@lemmy.todaylinkfedilinkEnglisharrow-up1·9 days agoI’ll have to try this out. I couldn’t get Microsoft’s ternary bitnets running on llama.cpp or ollama.
I’ll have to try this out. I couldn’t get Microsoft’s ternary bitnets running on llama.cpp or ollama.