JTI - Deploy through a pipeline

Deploy trough a pipeline

When everything works, you want to be able to deploy your code when it is pushed to GitLab. There are many different branching strategies to choose from, and you should probably have the main branch protected so that you always merge it through a Merge Request that can be audited and approved before the code is pushed to production. However, you will start by just making sure that new commits to main will be automatically deployed to the staging server, and manually deployed to the production server.

1 The .gitlab-ci.yml file

To create a pipeline on GitLab, simply create a file named .gitlab-ci.yml and place it in the root of your project. (Doing this in the web UI of GitLab is pretty neat.)

Create two stages named "integration-test" (staging) and "deploy" (production). Start with the staging stage first since the production stage is more or less a copy of staging.

Some tips on the way:

• Setting the "environment" to "staging" and providing a URL will make testing even easier.
• You need to make sure that the GitLab runner executing the pipeline has access to your private SSH key. GitLab has good information on how to do this.
• Pay attention to "Verifying the SSH host keys" in the documentation at GitLab. Your runner must trust your newly created servers. (see environment variables)
• If you created the pipeline from a template, you can lower the sleeps that simulate testing and linting.

Example project - .gitlab-ci.yml

Pipeline environment variables

You need to set some environment variables on your GitLab project. (Settings -> Ci/CD -> Variables)

• PRODUCTION_HOST: The IP-address of the production server.
• SSH_PRIVATE_KEY: The private key used to connection to the servers.
• SESSION_SECRET: Used by NODE and gets built into the container.
• DOCKER_PORT: On which port to start Docker. (in our set up server is mapped against 80)

2 Validate your pipeline

Start by validating the pipeline by entering the "CI/CD editor" (CI/CD->Editor) and clicking "validate". Here you can show a visualization of the pipeline and a merged version of your pipeline if you are importing parts of the pipeline from other places.

To run the pipeline you can either:

• Push a new commit to the project.
• Click "Run Pipeline" in CI/CD->Pipelines.

You can always enter a stage in the pipeline and look at the command-line output to troubleshoot.

Make a notable change in the CSS or a view file and wait for the pipeline to finish executing. Click Deployments->Environments to get the link to the application. Visit and confirm.

3 Trigger pipeline

Every time you make a change in your repo the pipeline will try again. It is also possible to manually retrigger the jobs in Gitlab.

Once everything worked you should be able to open PRODUCTION_HOST in the browser and see the JTI application.

You now have a CI/CD pipeline for your application. You can develop the application locally, and new versions are easily deployable.

4 Troubleshooting

If your pipeline fails or your application doesn't deploy correctly, here are some common troubleshooting steps:

• Examine the GitLab Pipeline Output: Click on the failed stage or job in the GitLab UI to see the detailed logs. This output often contains error messages that can pinpoint the issue. Look for specific error codes or messages related to SSH, Docker, or your application.

• SSH Connection Issues:

  • Verify SSH_PRIVATE_KEY: Ensure the SSH_PRIVATE_KEY variable in GitLab is correctly set and matches the private key on your GitLab runner. Double-check for any extra spaces or line breaks.
  • Verify Host Key: Make sure the host key for your production server has been correctly added to the known_hosts file. The .before_script_ssh_setup should handle this, but verify the logs for any errors related to ssh-keyscan. You might need to manually SSH into the server from the runner (if possible for debugging) to add the key.
  • Firewall Rules: Check if there are any firewall rules on your production server that might be blocking incoming SSH connections from the GitLab runner. Ensure port 22 is open.

• Docker Issues:

  • Docker Installation: Confirm that Docker and Docker Compose are correctly installed and running on your production server (as set up by the cloud-init script). You can SSH into the server and run sudo systemctl status docker and docker compose version.
  • Docker Compose Errors: Review the output of the docker compose up --build -d command in the pipeline logs. Look for errors related to building the Docker image, pulling images, or starting containers.
  • Container Logs: If the containers start successfully but your application doesn't work, SSH into the production server and use docker ps to see the running containers. Then, use docker logs <container_id> to view the logs of your application container for any application-level errors.
  • Port Mapping: Ensure that the DOCKER_PORT environment variable is correctly configured and that the port mapping in your docker-compose.yaml file is as expected (e.g., mapping the container port to port 80 on the host).

• Environment Variable Issues:

  • GitLab Variables: Double-check that all the required environment variables (PRODUCTION_HOST, SSH_PRIVATE_KEY, SESSION_SECRET, DOCKER_PORT) are correctly defined in GitLab's CI/CD settings.
  • Application Access: Ensure that your application within the Docker container is correctly accessing these environment variables.

• Server Resource Issues: Although less common for basic deployments, check if your production server has sufficient resources (CPU, memory) to run the Docker containers.

By systematically checking these areas based on the error messages in your pipeline logs, you should be able to diagnose and resolve most deployment issues. Remember to make small changes and re-run the pipeline after each potential fix to isolate the problem.