suoko@feddit.it to LocalLLaMA@sh.itjust.worksEnglish · edit-226 days agoI dropped Qwen and Gemma for a 27B model that runs smoothly on my 8GB graphics card - Bonsai 27Bwww.xda-developers.comexternal-linkmessage-square20linkfedilinkarrow-up149arrow-down18
arrow-up141arrow-down1external-linkI dropped Qwen and Gemma for a 27B model that runs smoothly on my 8GB graphics card - Bonsai 27Bwww.xda-developers.comsuoko@feddit.it to LocalLLaMA@sh.itjust.worksEnglish · edit-226 days agomessage-square20linkfedilink
minus-squareFtumch@lemmy.todaylinkfedilinkEnglisharrow-up1·27 days agoI’ll have to try this out. I couldn’t get Microsoft’s ternary bitnets running on llama.cpp or ollama.
I’ll have to try this out. I couldn’t get Microsoft’s ternary bitnets running on llama.cpp or ollama.