Are you actually comfortable with running kubernetes?

This is an automated archive made by the Lemmit Bot.

The original was posted on /r/selfhosted by /u/Nils-22 on 2023-08-06 19:40:41.

First post here, because this just drives me nuts and has kept me busy for months.

Quick introduction to me:

• I would describe me as pretty tech savvy
• I work in IT (Primary selling and deploying Microsoft SaaS as Consultant but also did administration before)
• I have a not small homelab
  • 6 Servers
    • 1x Storage Server (about 60TB of spinning rust usable)
    • 1x dedicated backup server for Proxmox and docker running PBS and Minio
    • 1x dedicated management server (Git, Portainer, Jenkins, etc.)
    • 3 node Proxmox cluster (1x beefy server and 2x small node using an N100 in a 1u case)
  • 10g backhaul

My "problem":

I currently mainly use VMs with Docker for deploying all my services but, as we all are somewhat the same we want to try other things and the homelab never has a state where we are 100% satisfied, so I wanted to try kubernetes. Mainly for HA because some of my services are used by other people and the services should stay available if one server goes down.

I successfully deployed now a k3s 3 node cluster with Longhorn as storage and everything is working fine. Actually so well it ran since I began using it without problems non stop which i can't say about my Docker environment (but thats mainly caused by my backups which is hopefully fixed now with the dedicated backup server).

I have been doing and learning quite a bit with kubernetes the last few months but just can't seem to get comfortable with it. My problem now is that I don't really trust my self operating this kubernetes cluster. What happens when the cluster goes boom and I have to redeploy all services. Its not just restoring the files and one command and everything runs. I need to create a new cluster or repair it, restore the data using Longhorn and then start the service with the restored PVCs.

With my docker services the backup currently contains everything I need to redeploy it (all application data and the the docker-compose file) so I just restore the files, run docker compose up -d and it runs again.

Do those of you using kubernetes (and not using it at work) are actually comfortable using it and have a plan to restore?