Im in simmilar position, what would you suggest for someone who use it on local network or vpn only? Is it possible to not expose port and still use domain/ssl?
Yes it is possible. All you need is to use a DNS challenge with let's encrypt and you will get an SSL certificate. You can then access everything using wireguard or whatever VPN you use