My main system is a Dell Wyse 5070 Extended running Pop-OS, but its main purpose is to be small on my desktop while being just enough computer to run ThinLinc (which is like RDP for Linux). ThinLinc connects to a much more powerful VM in my Proxmox cluster that runs Debian 11, which is my "real" desktop where I spend all of my time.
However, I'm currently working on setting up a new server with a better CPU, more RAM and a Tesla P4, after which my "real" desktop will be a Nobara 38 VM (based on Fedora) with the Tesla card passed through.
Do you access your server in different places? Curious what the benefit is in remoting into your daily computer. Wouldn't there be input lag? Can you watch videos through it?
The rack with my cluster is in a closet about 5 feet away from my desk. Performance with Thinlinc is great, video is no problem at all, no lag, even with my slow gigabit network.
Using a thin client at my desk to access multiple VMs depending on what I want to do is pretty ideal for me.