Wasn’t the Gadsden flag anti-colonialist originally? Or is there a deeper history here I’m unaware of, like that it was only championed by wealthy slave-owners?

I assume you’re not referring to the fact that systemic oppression coexisted (which included slavery and genocide) since by that measure no liberationist symbol in history would survive.