I probably would have gone with Xfce since this is mostly acting as a server but this was also my first real experiment to see if I could replace a Windows desktop with Linux so I wanted some more functionality in that regard.
Sure, expending hours learning shortcuts is way more minimalist than using plasma. Plasma used to be bloated, but now is very similar to XFCE in terms of resources, and if you don't use akonady or baloo even more minimal.