Previously, I had been using the Layer2 ip advertisement of metallb for my cluster. This worked out of the box, but over time some cracks appeared. Some devices, epecially IoT and other smart devices, did not play very nicely with this. They would periodically be unable to route to loadbalancer ips. This create random drop outs of connectivity which really killed the experiance. Resolving this would involve dropping arp caches and/or bouncing the metallb speakers. Limped along with this for a while before putting together a path forward with BGP.

While one of the main tennants of this cluster project is to be entirely selfhosted. There are some things that simply cannot be. One of those things is public DNS.

Chose cloudflare as a domain registrar because not only does it have a terraform provider to manage configuration as code, but also has free to use tunnel proxies as a mechanism to ingress public traffic (more on these later) and the community has created the cloudflare-operator for first class integration of those proxies inside of kubernetes. A perfect match.

Set out to host this blog to document and share things. This would be one of the first times to really put all the pieces together of this stack. Gitea was choosen for git source code hosting and it does have a container registry so will go ahead and use it to push artifacts to.

Source Repository Structure

hugo/
docker/
helm/
terraform/

Hugo

Initialized the hugo projects structure with

hugo new site hugo

After creating an empty project its required to add a theme, else you get an empty site. Not the first whose run into this, that Step 3 of the quick start is a doozy.

After installing kubevirt and pouring over documentation it seemed apparent that I was going to need to get the CDI operator installed as well. This operator was created to allow newly created PVCs to be populated with data before usage. I was going to need this to get a live iso of solaris loaded into a PVC so it could be mounted as a virtual cdrom drive of my vm.

Recently have come to the need of having to run a virtual machine as a part of the k8s cluster.

My main storage was a solaris ZFS raidz2 that has served me well for a long time now. The machine I had originally hosted it on was out of commission so used an old optiplex desktop for a while. The problem with this was that all the drives were just strewn about and it was quite haphazard. Picked up an Asus 4x4 box as this seemed to be a good hardware fit with the rest of my cluster and it has ECC ram. When booting solaris onto it from the live cd it refused to start with some tty error. Its not happy about the something in the bios (probably the fact that its uefi). So to use the new hardware and still access the data will need to run a VM on this machine. And what better way to manage that than via kubevirt!