Quick and Easy Kubernetes with Minikube 🌯

Minikube is ideal tool to setup kubernetes (k8s from now on) locally to test and experiment with your deployments.

In this guide I will try to help you get it up and running on your local machine, drop some tips on where and how particular stuff should be done and also make it helm capable (I assume when you use k8s that at some point you will want to learn about and use Helm, etcd, istio etc).

This is your local k8s environment scaffolding guide.

Minikube installation

Grab a 7 layer and a beer and lets get after it!
Minikube works with virtual machine, and for this it can use various options depending on your preference and operating system. My preference in this case is Oracle’s VirtualBox. (I know right? UGH!!!)

You can use brew to install everything:

$ brew cask install virtualbox minikube

In this case you could get some kind of inconclusive installation error related to virtualbox installation, especially on Mojave and probably everything afterwards.

Whatever it says, it is most probably a new security feature in MacOS X that is in your way.

Go to System Preferences > Security & Privacy and on General screen you will see one (or few) messages about some software needing approval to install. You should carefully review the list if there is more then one and allow installation of software you need — in this case software by Oracle.

After that is done you can re-run the command above and when it is done you should be ready for next steps.

Running and accessing the cluster

Starting it would be as easy as

$ minikube start

There is a newer version of minikube available (v0.35.0).  Download it here:

https://github.com/kubernetes/minikube/releases/tag/v0.35.0




To disable this notification, run the following:

minikube config set WantUpdateNotification false

Starting local Kubernetes v1.12.4 cluster...

Starting VM...

Getting VM IP address...

Moving files into cluster...

Setting up certs...

Connecting to cluster...

Setting up kubeconfig...

Stopping extra container runtimes...

Machine exists, restarting cluster components...

Verifying kubelet health ...

Verifying apiserver health ....Kubectl is now configured to use the cluster.

Loading cached images from config file.



Everything looks great. Please enjoy minikube!

In order to optimally utilize your local machine’s resources I would suggest stopping it when you do not need it any more… With VirtualBox in center of it, it will go through you laptop’s battery pretty quickly. Starting it again later will get you back where you left off:

$ minikube stop

Kubernetes dashboard is also available to you (while minikube is running):

$ minikube dashboard

I will assume you have kubectl installed locally and that you are already using it for some remote clusters so you got multiple contexts. In this case, you need to list contexts and switch to minikube one (in following commands assuming default name that is, ofc, “minikube”)

$ kubectl config get-contexts
$ kubectl config use-context minikube

Now you are in the context of your local k8s cluster that runs on minikube and you can do all the k8s things in it.

Ingress controller

To run your deployments that have ingress (and I assume most of them will), you will need ingress add-on:

minikube addons enable ingress

Make sure that you setup ingress based on your local hosts. It basically means that whatever you set as host in your ingress rules needs to be set up in your /etc/hosts file

[minikube ip] your.host

Where “[minikube ip]” should be replaced with actual minikube ip. It also works with multiple, space separated local hosts after minikube ip.

Here is shortcut to do it in bash:

$ echo "$(minikube ip) local.host" | sudo tee -a /etc/hosts

Container registry — i.e. Docker registry

Reality of the real container registry usage in local environment is a rough one, so I will provide easy, quick and dirty option that makes it quite easy to deploy your local work to your local k8s, but deprives you of really important experience of using proper container registry.

Local container registry

Get your local docker context to point to minikube context:

$ eval $(minikube docker-env)

To revert:$ eval $(docker-machine env -u)

When in minikube context, to start local docker registry:

$ docker run -d -p 5000:5000 --restart=always --name registry registry:2

So, now you have local registry to push stuff to (as long as your docker is in context of minikube).

You can now do:

$ docker build . -t 
$ docker tag  localhost:5000/:

At this point you can use localhost:5000/: as image in your deployment and that is it.

Using remote container repo

To use remote container repo locally you need to provide way to authenticate, which is through k8s secrets.

For local secrets management for ECR, GCR and Docker registry I recommend using minikube addon called registry-creds. I do n ot consider it safe enough to be used anywhere but in local env.

$ minikube addons configure registry-creds
$ minikube addons enable registry-creds

Note on ECR setup: Make sure that, if you are setting it for AWS ECR, and you do not have role arn you want to use (you usually wont have and it is optional), you set it as something random as “changeme” or smt… It requires value, if you just press enter (since it is optional) deployment of creds pod will fail and make your life miserable.

In case of AWS ECR, that will let you pull from your repo directly setting url as container image and adding pull secret named awsecr-cred:

imagePullSecrets:
      - name: awsecr-cred

I have to note here that running this locally worked quite chaotically for me and every session was new experience and new hack to make it work… Not a happy path.

Helm

Helm is package manager for k8s, and is often used for configuration management across deployments. With high popularity of the tool and raising adoption, I want to end this guide with the note about adding helm to your local k8s env.

It is quite easy at this point, just have minikube up and:

$ brew install kubernetes-helm
$ helm init

This should be deprecated information pretty soon, but in current case helm uses backend called Tiller and that is what gets installed/deployed during helm initexecution.

You should check tiller deployment with: $ kubectl describe deploy tiller-deploy — namespace=kube-system

Valuable read: https://docs.helm.sh/using_helm/

Now that Helm is installed lets fire up the cluster!

Now you should be able to deploy containers using Helm
helm install stable/prometheus
then run
kubectl proxy
and
minikube dashboard &
And you should see your prometheus container coming online in the K8s UI like so!

Now you have full k8s local environment able to accept all of your test deployments before you decide to put them in the cloud (or “raw iron” server somewhere).

HAPPY H@x0ring!