SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 1 month agoDo you host your own AI?message-squaremessage-square203linkfedilinkarrow-up1184arrow-down143file-text
arrow-up1141arrow-down1message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 1 month agomessage-square203linkfedilinkfile-text
minus-squareFranconian_Nomad@feddit.orglinkfedilinkEnglisharrow-up2·1 month agoYeah, a higher quant would be nice, I actually try not to go below Q5, but you can domino’s so much with 16GB of VRAM and the ddr4 system RAM. But I must say I‘m pretty impressed by Qwen3.6-35b, not only from its capabilities but also from hardware requirements. MoE for the win I guess. RWKV sounds interesting, have to look into it, thanks!
Yeah, a higher quant would be nice, I actually try not to go below Q5, but you can domino’s so much with 16GB of VRAM and the ddr4 system RAM.
But I must say I‘m pretty impressed by Qwen3.6-35b, not only from its capabilities but also from hardware requirements. MoE for the win I guess.
RWKV sounds interesting, have to look into it, thanks!