One use of LLMs that I haven’t seen mentioned before is to use them as a sounding board for your own ideas. By discussing your concept with an LLM, you can gain fresh perspectives through its generated responses.
In this context, the LLM’s actual comprehension is irrelevant. The purpose lies in its ability to spark new thought processes by prompting you with unexpected framings or questions.
Definitely recommend trying this trick next time you’re writing something.
I read about this on the cursed orange site. Some guy talked about going on a walk with his wireless warplugs on, talking to ChatGPT’s audio interface discussing some world building he was doing.
Are there any LLM services that can be reasonably used without paying? I tried some llamafiles but seems like my laptop cannot handle them well.
seconding gpt4all, makes it quick and easy to run and if youre fancy you can stream the output from your computer to your phone. i run a capybara-hermes-mistral mix but i would suggest starting with mistral instruct until claude3 comes out
As long as you don’t care about your inputs being harvested, gemini is free currently. I’ve been using GPT4All to run stuff locally, but if your laptop is having trouble with llamafiles, then it’s probably gonna have trouble with that too.
What model do you run?
I find I like Wizard 1.2 and Hermes the best
On the topic of GPT4ALL, I’m curious is there an equivalent of that that but for txt2img/img2img models? All the FOSS txt2img stuff I’ve tried so far is either buggy (some of the projects I tried don’t even compile), require a stupid amount of third party dependencies, are made with NVidia hardware in mind while everyone else is second class or require unspeakable amounts of VRAM.
automatic1111 webui launcher, its stable diffusion. fun fact its icon is a pic of ho chi minh
if you wait, stable diffusion 3 is coming out soon. nvidia will run faster because its tensors are better unfortunately. SD is more ethical than others, you can load up models that are trained only on public art and pics
I’m pretty sure I tried that one but it kept running out of VRAM. Also it utilizes proprietary AMD/NVidia software stacks which are a pain to set up. GPT4ALL is a lot better in that regard, they just use Vulkan compute shaders to run the models.
There’s also ComfyUI, but the learning curve is a bit steeper https://github.com/comfyanonymous/ComfyUI
although there’s CushyStudio frontend for it that’s more user friendly https://github.com/rvion/CushyStudio
ComfyUI seems like the most promising but it also uses ROCm/CUDA which don’t officially support any of my current GPUs (models load successfully but midway through computing it fails). Why can’t everyone just use compute shaders lol.
Oh yeah that whole thing is just such a mess, another L for proprietary tech.