Colima issues and tips

On macs Colima is a way to run containerd and Kubernetes locally, and with the recent addition of native virtualization support (VZ), it works really well.

Port-forwarding

If the same container is started again with the same port-forward option, so there are more than one container listed by nerdctl ps -a, then connection to localhost:8080 fails with connection refused. The solution would be to remove all the containers: nerdctl ps -aq | xargs nerdctl rm, or starting containers with --rm option to automatically remove them on completion.

# run container with port-forward
nerdctl run --rm -it -p 8080:80 nginx

# test it works
curl localhost:8080

Quick start

colima start \
	--memory 8 --cpu 4 --disk 100 \
	--runtime containerd \
	--kubernetes \
	--vm-type vz \
	--mount-type virtiofs \
	--network-address

--network-address is an important option than assigns a host-reachable IP address, usually 192.168.106.2.

Problem

Sometimes the host-reachable IP address is not assigned, and colima reports an error on start:

The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?

Which means that k3s failed to start inside the lima's virtual machine.

Solution

The easiest one is to enable internet sharing option. However, if for some reason, the option is disabled by security, there is still a way:

# install dnsmasq
brew install dnsmasq

# configure dnsmasq
cat <<EOT > /opt/homebrew/etc/dnsmasq.conf
interface=bridge101
dhcp-range=192.168.106.2,192.168.106.150,12h
dhcp-host=colima,192.168.106.2,infinite
EOT

# disable mac dhcp
sudo /bin/launchctl unload -w /System/Library/LaunchDaemons/bootps.plist

# start dnsmasq as service
sudo brew services start dnsmasq

dnsmasq lease file: /opt/homebrew/var/lib/misc/dnsmasq/dnsmasq.leases

Complete script

# cleanup
colima stop
colima delete -f
sudo brew services stop dnsmasq
rm -rf /opt/homebrew/var/lib/misc/dnsmasq/dnsmasq.leases

# start fresh instance to create bridge101 interface
colima start --vm-type vz

# configure dns
cat <<EOT > /opt/homebrew/etc/dnsmasq.conf
interface=bridge101
port=0
dhcp-range=192.168.107.2,192.168.107.150,12h
dhcp-host=colima,192.168.107.2,infinite
EOT
sudo /bin/launchctl unload -w /System/Library/LaunchDaemons/bootps.plist
sudo brew services start dnsmasq

# stop instance
colima stop

# start colima with kubernetes
colima start \
	--memory 8 --cpu 4 --disk 100 \
	--runtime containerd \
	--kubernetes \
	--vm-type vz \
	--mount-type virtiofs \
	--network-address