Kubernetes: Cross Namespace Service Communication

The following explanations are done for Minikube environment installed on Ubuntu.

I have used the following references:

• https://stackoverflow.com/questions/62162209/ingress-nginx-errors-connection-refused
• https://medium.com/faun/metallb-configuration-in-minikube-to-enable-kubernetes-service-of-type-loadbalancer-9559739787df

The problem that we try to solve

In order for the code running inside of a pod (the name of the pod could be client) to make a sync call over HTTP to a Kubernetes service located in the default namespace (the name of the service could be auth-srv) there are two options that can be uses:

• client can call directly into the auth-srv using an URL that has the following format: http://auth-srv:port/path
• client can call the auth-srv through ingress-nginx and this option allows one to take advantage of all the configured routing rules that route the request to the appropriate service. In this case one will use cross namespace service communication since the ingress-nginx-controller is deployed in another namespace kube-system

To make the second option work one should make sure that the ingress and metallb addons are enabled in Minikube:

$ minikube addons enable ingress

$ minikube addons enable metallb

Than make sure to apply all the configuration files to your k8s cluster:

$ kubectl apply -f .

In my case I am using Skaffold to handle my k8s configuration files, so I will use instead:

$ skaffold dev

Check the objects created in kube-system namespace:

$ kubectl get all -n kube-system

Next create in kube-system a LoadBalancer service that will expose the ingress-nginx-controller:

$ kubectl expose deployment ingress-nginx-controller --target-port=80 --type=LoadBalancer -n kube-system

Check that the service has been created:

$ kubectl get services -n kube-system

Now everything is set up and you can use inside the client pod the following URL to make sync calls through ingress-nginx: http://ingress-nginx-controller.kube-system.svc.cluster.local/path

The URL above respects the following format http://SERVICENAME.NAMESPACE.svc.cluster.local

Kubernetes Secrets

Kubernetes offers a feature that can be used in order to share secret information throughout different pieces of the application that is deployed on top of a Kubernetes cluster.

Let’s assume that our application consists of several pods. A pod runs a container, and a container runs code that might need access to a secret information. That secret information might be a private key that is used to sign a JWT. In order to securely store that private key one can create a new kind of object inside of a Kubernetes cluster. The new kind of Kubernetes object is called ‘Secret’.

Kubernetes Secret object:

• is an object like Pod, Deployment, Service
• inside of a Secret object one can store key-value pairs of secret information
• once a key-value pair is created, we can load inside of our pods, and the key-value pair will going to be exposed as an environment variable to the pod; actually the environment variables will be exposed inside of the container that runs our piece of application

In order to create a secret inside a Kubernetes cluster, one could use the following command:
$ kubectl create secret generic jwt-secret --from-literal=JWT_KEY=asdf

Description:

• generic: designates the kind of secret (there are multiple kinds of secrets inside the Kubernetes cluster)
• jwt-secret: the name of the secret (it is very similar to the name we assign to pods or deployments)
  • mostly used for logging purposes
  • we used this to reference the secret from inside a pod config file
• JWT_KEY=asdf: we can assign many different key-value pairs to a secret; in this case we assign a key-value pair JWT_KEY=asdf

Further explanation:

• the command above is an imperative command
• for everything else (like creating deployments or services) we have used a declarative approach using config files
• the main purpose to use an imperative command is that we do not want to have a config file where we list out secrets
• still we could technically use local Env Variables on the host machine and refer to them from the config files

In order to get all the secrets that have been created in a Kubernetes cluster:
$ kubectl get secrets

In ordere to make the secret available as Env Variable inside of a Kubernetes pod, one should edit the Deployment config file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-depl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: auth
  template:
    metadata:
      labels:
        app: auth
    spec:
      containers:
        - name: auth
          image: luckpp/auth
          imagePullPolicy: Never
          env:
            - name: JWT_KEY
              valueFrom:
                secretKeyRef:
                  key: jwt-secret
                  name: JWT_KEY