Interesting. Is Kubernetes ever used as the orchestrator in HPC workloads, or rather are even containers used there? Also what are some good resources to get better at kubernetes? Currently I am mostly playing with managed k8s like digital ocean's and thinking about transitioning to k3s based bare-metal solution. So, I would really like to learn what's the right way of doing this.
There are some HPC environments that use Kubernetes but they likely use custom schedulers optimized for batch workloads (e.g. https://github.com/volcano-sh/volcano).
"containers" are often used but not always docker containers. HPC environments I've seen will often use container primitives (e.g. cgroups, namespaces).