They were colonial landowners, only thing differing them from the old decrepit British aristocracy was the lack of old decrepit titles and being snubbed on salons by them. And considering that their slogan was “no taxation without representation” (for them ofc, poors and slaves should just not be seen), seems it was also what bothered them greatly.