All three are web based frontends for git repositories; you use git to send and receive code to/from them for storage and sharing. They all also provide other things useful to developers such as issue tracking, wikis and such. They are different products that fulfill the same role.

what software does github.com use?

It’s all proprietary software (presumably) written in-house. We don’t have access to it.

whats the difference between them (pros/cons)?

Github:
Pro: Wider reach, everyone knows about Github.
Con: Proprietary; your code is hosted based on the whims of Microsoft.

Forgejo:
Pro: Open source, selfhostable. There’s a big instance on https://codeberg.org/ which a lot of open source projects are starting to move to.
Con: It’s smaller and not as well known as Github. In theory it may also lack features, but I’ve not seen any that have gotten in my way.

Gitlab:
Pro: It’s… I guess in second place in terms of popularity? It’s also selfhostable.
Con: It’s one of those open source projects with paid closed source features, so not really appealing to either group. It’s also had questionable management decisions recently.

what about self-hosting? Possibilities/Preferences?

If you want to selfhost a git server, I’d recommend Forgejo; it seems to be the most friendly towards the open source and selfhosting communities.