My team ran on-premise with Talos for the last bunch of years. It avoided tons of odd issues I saw when I built k8s clusters on top of Ubuntu that I never quite ironed out.
It gave me a ton of appreciation for Kubernetes and Linux and how k8s is built on linux assumptions: the file system layout’s a little different in Talos to the default k8s assumptions, so there were some small adjustments I’d make every now and then.
Sidero’s professional support is out of the world, too. I had simple questions and got answers back from their devops engineers who are running clusters 100-1000 times the size of anything I had to touch, or from their software engineering team who write Talos.
Definitely worth a look if you’re hosting in your home lab, definitely worth a look if you’re doing self-hosted k8s for your job.
I do, still. It's often a lot cheaper and modern HW is very robust and reliable. But it comes with more responsibility, of course. Not happy with performance on typical cheap VMs.
With Kubernetes it makes a lot of sense to run bare metal. What advantage is there to running K8s nodes on a hypervisor? It's one more thing that requires maintenance and can go wrong.
And that's actually my concern with K8s still. It still has an unnecessarily high maintenance burden for projects that are limited in scope and few scalability requirements. Which can often still be an important project.
Often I find that a load balancer and a couple of docker compose file sis more rugged, because you'll be able to leave it for a year and it'll still work and easily be understood by just glancing at it.
I do :)
alongside cloud of course, hybrid is the way to go in my view.
I run a bunch of openshift clusters for very private container workloads and VMs.
You can't beat a local fiber channel SAN for speed and reliability, especially compared to cloud software defined storage, and even on the networking side of things if you worked out your local connectivity well using real networking vs cloud providers software defined networks is so much better.
CSIs are hit and miss, some vendors are really really good, some are meh (hitachi, even though they are improving a lot with the latest relases), but the moment you get an IBM nvme san and start to tune all layers you get performance that are out of this world, some of our dbms workloads are very io sensitive and they improved 10x with nvme fabric.
I'd love to know some details on why? I don't have any particular complaints about my k3s on Debian, but if there is a better option, I would love to know!
You'd think someone with a "talk to us" enterprise plan and a bunch of well known brands as customers would have at least one human copytexter and a designer on staff. Instead, a landing page reeking of Claudish in text and look.
talosctl debug docker.io/library/alpine:3.24 gives you a shell in the host namespaces, with root mounted at /host. I think it works without kubelet, I suppose it might not help with a failed containerd though.
It gave me a ton of appreciation for Kubernetes and Linux and how k8s is built on linux assumptions: the file system layout’s a little different in Talos to the default k8s assumptions, so there were some small adjustments I’d make every now and then.
Sidero’s professional support is out of the world, too. I had simple questions and got answers back from their devops engineers who are running clusters 100-1000 times the size of anything I had to touch, or from their software engineering team who write Talos.
Definitely worth a look if you’re hosting in your home lab, definitely worth a look if you’re doing self-hosted k8s for your job.
Often I find that a load balancer and a couple of docker compose file sis more rugged, because you'll be able to leave it for a year and it'll still work and easily be understood by just glancing at it.
I run a bunch of openshift clusters for very private container workloads and VMs.
You can't beat a local fiber channel SAN for speed and reliability, especially compared to cloud software defined storage, and even on the networking side of things if you worked out your local connectivity well using real networking vs cloud providers software defined networks is so much better.
CSIs are hit and miss, some vendors are really really good, some are meh (hitachi, even though they are improving a lot with the latest relases), but the moment you get an IBM nvme san and start to tune all layers you get performance that are out of this world, some of our dbms workloads are very io sensitive and they improved 10x with nvme fabric.
Jesus.