Thank you! I did some reading and that's also how I understand it: at least one peer has to have port forwarding enabled / listen on a port for two peers to connect.
Also I found out about "Hole punching" or "NAT punching" where a middleman server is used to open up ports on two peers that do not have ports forwarded yet to allow them to talk to each other directly. This is also used in BitTorrent. And also explains why it works without explicit port forwarding enabled.
Why is port forwarding important? I have my torrent server running, downloading and uploading perfectly fine. Is port forwarding needed for like something else besides general down/uploading?