2. JTI - K8s - production

Goals

In the last exercice we managed to get a k8s environment up for local development. Now we will look at how to create and deploy this to a self hosted cluster using gitlab. Of course, you are free to choose what ever application you like, and you do not need to deploy to CSCloud, but this guide is heavliy focused on doing just that.

Prerequisites

Make sure to have your application in a Gitlab project. This guide makes use of our example project and the files in this project. Pay extra attention to:

  1. ./k8s/taskit.yaml
  2. gitlab-ci.yml

Exercise

The exercise will go through theese steps:

  1. Create a Kubernetes cluster in CSCloud
  2. Connect the cluster to Gitlab (agent and container registry)
  3. Create a LoadBalancer in CSCloud

Build the infrastructure

We will start by creating a Kubernetes cluster in CSCloud. To make this step as easy as possible, we have prepared a Terrafrom script for doing this.

Please follow the steps in Just Task It - Kubernetes / Infrastructure.

Additionally: Install Helm to be able to install the Gitlab agent.

k8s cscloud

Connect Cluster to Gitlab and deploy

A nice feature in Gitlab is that we can install a Gitlab agent in our cluster to authorize gitlab to work with the cluster.

1. Connect to the cluster

Navigate to Gitlab->Your Application Project->Infrastructure->Kubernetes clusters and click "Connect a cluster".

In the searchbox, write "cscloud-agent" and press enter. Press "Register".

Now you will be presented with a Agent access token and installation sugestion for the cluster.

  1. Copy the instructions,
  2. open a ssh-connection to your cluster-plane-server,
  3. paste the instructions into the server.

Close the installation sugestion and you should now see the agent beeing connected. (reload page)

Agent connected

In the deploy-pipeline we will connect to cluster using this agent. (go have a look)

You will see that to change k8s context we use the environment variable, K8S_CONTEXT. Create it (Settings->CI/CD->Variables) and set it to: <project-path>:cscloud-agent example: 2dv013/student/xx22yy/part2-architecture/jti-application:cscloud-agent

That should be it!

2. Deploy from Container Registry using a deploy token

Container registry

The build script that is in our example pipeline is building a new image of our Taskit application. This image is placed in our Gitlab Container Registry.

If the pipeline have built the image, navigate to Packages & Registries->Container Registry. Click the taskit-url and you should find the tag "latest". Copy the image path and edit ./k8s/taskit.yaml. Replace the path to the image under Deployment->taskit

Since this registry is private, the cluster will not be able to pull images from your registry. To allow this, we need to create a "Deploy Token" and place that as a secret in the cluster.

  1. Visit: Settings->Repository->Deploy tokens
  2. Create a token with the name gitlab-deploy-token and the Scope read_registry. (date and username can be left blank)

This token will automaticly be copied to a cluster secret named regcred every time the gitlab-ci pipeline is executed. Of course, we could copy the credentials in to a kubernetes secret manually instead. (In the cluster you can list secrets by kubectl get secrets, however, it will not show up until you have deployed at least once.)

Deploy!

Now, try to deploy by going to CI/CD->Pipeline and manually start the deploy.

Deploy button

If everyting works out, you should be able to go into your cluster, run kubectl get svc to list all services, incl. taskit. Take note of the portnumber provided by the NodePort. 8888:XXXXX. (It is the right part we want.)

Try to curl the port on one or more of your nodes. Example:

‌‌‌‌‌

If you get the HTML of the JTI-application everything is working! But, we can not connect to the application from outside of CSCloud.

Set up a loadbalancer

The loadbalancer will act as a gateway to our application in CSCloud. It will get a floating ip and it will direct traffic from port 80 into our kubernetes nodes. We will create the loadbalancer manually, but we could use Terraform.

Log into cscloud.lnu.se and navigate to Network->Load Balancers

Create the Load Balancer

Click "Create a new Load Balancer". (If not specified below, leave blank or default)

Load Balancer Details

  • Name: 'Taskit Loadbalancer' or whatever
  • Subnet: The subnet of our cluster

Listener Details

  • Name: 'Taskit Listener' or whatever.
  • Protocol: HTTP
  • Port: 80

Pool Details

  • Name: 'Pool 1' or whatever
  • Method: ROUND_ROBIN

Pool Members

Add your nodes, but not your cluster-plane.

On each node, change port from 80 to the NodePort of the service taskit in the cluster. (See Deploy-step above)

Monitor Details

  • Monitor type: TCP

Now: Create Load Balancer

Open up port 80

  • Click the link right of "Port ID".
  • Click "Edit Port"
  • Click "Security Groups"
  • Add the security group for HTTP, HTTPS
  • Update

Add an floating IP

Go back to "Network->Load Balancers" and choose "Assosiate Floating IP" for your Load Balancer. Add an IP from the list.

Test it!

Now, try to visit that IP in your browser. If everything works you should se Just Task It!

Environment URL (optional)

To get a nice way to reach your application from Gitlab Deployments->Environments. Set the environment variable TASKIT_URL to the IP you assigned to the load balancer.

Create the Session env as secrets (otional)

The session name and secret have so far been defined in the taskt manifest, but it is important that theese secrets remain secret. We can manually create a secret in our cluster and reference that secret from our mainfest file.

Log in to the cluster and run (pleace replace my examples with your own secrets):

‌‌‌‌‌

You can list secrets by:

‌‌‌‌‌

and see details about your secret by:

‌‌‌‌‌

Now, modify taskit.yaml by replacing the env. part of the sessions with:

‌‌‌‌‌

Try it out

Now every commit to Gitlab will rebuild your image and make it available for deployment. Of course this is a bit extreme and you can tweek this a lot.