Is there any computer program with AI capabilities (the generative ones seen in ChatGPT; onlineText-to-Picture generators, etc.) that is actually standalone? i.e. able to run in a fully offline environment.
As far as I understand, the most popular AI technology rn consists of a bunch of matrix algebra, convolutions and parallel processing of many low-precision floating-point numbers, which works because statistics and absurdly huge datasets. So if any such program existed, how would it even have a reasonable storage size if it needs the dataset?
The breaking down I mentioned is the quantization that forms a smaller model from the larger one. I didn’t want to get technical because I don’t understand the math details myself past how to use them. :)
Ah, sure. I think a good way to phrase it is to say they lower the precision. That’s basically what they do, convert the high precision numbers to lower precision fomats. That makes the computations easier/faster and the files smaller.
And it also doesn’t apply to text, audio and images. As far as I know quantization is mainly used with LLMs. It’s also possible with images and audio, but generally they don’t do that. As far as I remember it leads to degradation and distortions pretty fast. There are other methods like pruning used with generative image models. That brings down their size substantially.