🍹Early to RISA 🧉@sh.itjust.worksM to Greentext@sh.itjust.works · 8 months agoAnon helps people on StackOverflowsh.itjust.worksimagemessage-square25fedilinkarrow-up1361arrow-down114
arrow-up1347arrow-down1imageAnon helps people on StackOverflowsh.itjust.works🍹Early to RISA 🧉@sh.itjust.worksM to Greentext@sh.itjust.works · 8 months agomessage-square25fedilink
minus-squareWaxedWookie@lemmy.worldlinkfedilinkarrow-up3·8 months agoLook into ollama and mixtral variants… You’ll be limited by things like GPU memory though.
minus-squareMiss Brainfarts@lemmy.blahaj.zonelinkfedilinkarrow-up2·8 months agoMixtral 8x7B seems fairly capable to me so far. It’s just that I need to wait a few minutes for it to reply, given I’m running it on a 1st gen Ryzen… Any models you’d reccommend that fit into 4GB of VRAM? I’ve tried Deepseek Coder, and it certainly works well for quickly churning out bash scripts for whatever purpose I can possibly think of
minus-squareWaxedWookie@lemmy.worldlinkfedilinkarrow-up2·8 months agoSimilar story here with an old 10xx GPU. I’ve just started tinkering with dolphin-mixtral-8x7b, but it’s early days.
minus-squareMiss Brainfarts@lemmy.blahaj.zonelinkfedilinkarrow-up2·8 months agoDolphin is just peak amusement to me. I’ve asked so many weird things of it that it’s basically just cowering in a corner now, afraid of answering any question for fear of getting another kitten killed.
Look into ollama and mixtral variants… You’ll be limited by things like GPU memory though.
Mixtral 8x7B seems fairly capable to me so far. It’s just that I need to wait a few minutes for it to reply, given I’m running it on a 1st gen Ryzen…
Any models you’d reccommend that fit into 4GB of VRAM?
I’ve tried Deepseek Coder, and it certainly works well for quickly churning out bash scripts for whatever purpose I can possibly think of
Similar story here with an old 10xx GPU. I’ve just started tinkering with dolphin-mixtral-8x7b, but it’s early days.
Dolphin is just peak amusement to me. I’ve asked so many weird things of it that it’s basically just cowering in a corner now, afraid of answering any question for fear of getting another kitten killed.