I’m not calling it a search algorithm. I’m saying they all do the same math, and doing the math with more parallelism and variables doesn’t make what it is a mystery.
Search algorithms searching for functions isn’t new. Not knowing what each parameter corresponds to because you made your model huge doesn’t make LLMs a mystery. It’s still functionally one part. The hormone system is as complex as LLMs. Regulation of neurotransmitters is as complex as LLMs. Ignoring those external factors that are critical to how it works, individual portions of the brain are more complex than LLMs, then are all interconnected on top of that.
I fully believe we’ll get to AGI eventually (probably not before we understand the brain a lot better), but the idea that one pretty simple algorithm is going to get us there is crazy. Human intelligence is a system of disparate systems of disparate systems at minimum.
I’m not calling it a search algorithm. I’m saying they all do the same math, and doing the math with more parallelism and variables doesn’t make what it is a mystery.
Search algorithms searching for functions isn’t new. Not knowing what each parameter corresponds to because you made your model huge doesn’t make LLMs a mystery. It’s still functionally one part. The hormone system is as complex as LLMs. Regulation of neurotransmitters is as complex as LLMs. Ignoring those external factors that are critical to how it works, individual portions of the brain are more complex than LLMs, then are all interconnected on top of that.
I fully believe we’ll get to AGI eventually (probably not before we understand the brain a lot better), but the idea that one pretty simple algorithm is going to get us there is crazy. Human intelligence is a system of disparate systems of disparate systems at minimum.
So does having more parts make something a mystery, like the second paragraph, or not a mystery like the first?
I was a skeptic back in the day too, but they’ve already far exceeded what an algorithm I could write from memory seems like it should be able to do.
A combination of unique, varied parts is a complex algorithm.
A bunch of the same part repeated is a complex model.
Model complexity is not in any way similar to algorithmic complexity. They’re only described using the same word because language is abstract.