CI/CD Guide with GitLab: A Complete Workflow

GitLab offers a robust CI/CD platform integrated directly into your Git hosting environment. With GitLab CI/CD, teams can build, test, and deploy applications efficiently with workflows defined inside a single configuration file, .gitlab-ci.yml.

This guide provides a complete overview of setting up a CI/CD pipeline in GitLab, covering the structure and purpose of .gitlab-ci.yml, defining stages like build, test, and deploy, managing environments, and using features like artifacts, caching, and runners.

Table of Contents

1. What is GitLab CI/CD?
2. Understanding .gitlab-ci.yml
   • Structure of .gitlab-ci.yml
   • Key Components
3. Defining Stages in GitLab CI/CD
   • Example Workflow with Stages
4. Managing Environments in GitLab
   • Defining Staging and Production Environments
5. Using Artifacts, Caching, and Runners
   • Artifacts
   • Caching
   • Runners
6. Best Practices for GitLab CI/CD
7. Final Thoughts

What is GitLab CI/CD?

GitLab CI/CD is a continuous integration and delivery platform built into GitLab. It automates the process of building, testing, and deploying code whenever new changes are pushed, ensuring early detection of issues and faster feedback for developers.

Key Benefits of GitLab CI/CD:

• Integrated Workflow: CI/CD pipelines live in the same repository as your code.
• Customization Flexibility: Fully customizable pipelines via .gitlab-ci.yml.
• Scalability: Capable of handling workflows for small teams to large organizations.
• Extensive Features: Includes runners, artifacts, caching, and built-in environment management.

Understanding .gitlab-ci.yml

The .gitlab-ci.yml file defines the CI/CD pipeline in GitLab. It contains instructions for each step, environment, and tool used in the pipeline.

Structure of .gitlab-ci.yml

At its core, .gitlab-ci.yml includes the following components:

1. stages: Defines the sequence of steps (like build, test, deploy).
2. jobs: Executes tasks in each stage (e.g., build the app, run tests).
3. artifacts, caches, and dependencies: Used to manage and reuse data between stages.
4. rules/only: Controls when a job is triggered (e.g., on a branch or tag).

Basic Example:

stages:
  - build
  - test
  - deploy

build_job:
  stage: build
  script:
    - echo "Building the application"
    - mvn clean package

test_job:
  stage: test
  script:
    - echo "Running tests"
    - mvn test

deploy_job:
  stage: deploy
  script:
    - echo "Deploying the application"
    - scp target/app.jar user@server:/path/to/deploy

Key Components

1. Stages

Stages define the overall pipeline flow. They are executed sequentially by default.

2. Jobs

Jobs contain the logic for each pipeline task. A pipeline stage can have multiple jobs running in parallel.

3. Services

Services specify extra tools (like databases) needed during the pipeline.

Example: Add PostgreSQL for testing:

services:
  - name: postgres

4. Variables

Environment variables make pipelines dynamic.

Example:

variables:
  TEST_ENVIRONMENT_URL: "http://staging.example.com"

Defining Stages in GitLab CI/CD

Example Workflow with Stages

A typical GitLab CI/CD workflow might include the following stages:

1. Build Stage

A stage to compile or package the application.

build:
  stage: build
  script:
    - npm install
    - npm run build
  artifacts:
    paths:
      - dist/

2. Test Stage

A stage to run unit and integration tests.

test:
  stage: test
  script:
    - npm test
  dependencies:
    - build

3. Deploy Stage

A stage to deploy the application to an environment.

deploy:
  stage: deploy
  environment:
    name: production
    url: https://example.com
  script:
    - scp dist/* user@server:/var/www/html

Stages execute in the order defined in stages:. If the build fails, subsequent stages will not run.

Managing Environments in GitLab

GitLab CI/CD simplifies environment management, allowing teams to deploy and monitor applications in staging, production, or custom contexts.

Defining Staging and Production Environments

You can define environments in .gitlab-ci.yml to tag deployments for specific contexts like staging or production.

Example for Staging Deployment:

staging_deploy:
  stage: deploy
  environment:
    name: staging
    url: https://staging.example.com
  script:
    - echo "Deploying to staging"
    - scp target/app.jar user@staging:/path/to/deploy

Example for Production Deployment:

production_deploy:
  stage: deploy
  environment:
    name: production
    url: https://example.com
  script:
    - echo "Deploying to production"
    - scp target/app.jar user@production:/path/to/deploy
  only:
    - tags

Auto-Stop Environments

Temporary environments (e.g., for feature branches) can be stopped automatically after merging.

Example:

stop_review:
  stage: cleanup
  environment:
    name: review/$CI_COMMIT_REF_NAME
    action: stop
  script:
    - echo "Stopping review environment"

Using Artifacts, Caching, and Runners

Artifacts

Artifacts are files generated by one job and passed to another. Use them to save build outputs like compiled code or test results.

Example of Artifacts:

build:
  stage: build
  script:
    - mvn clean package
  artifacts:
    paths:
      - target/*.jar

Caching

Caching helps speed up pipelines by reusing dependencies or build outputs.

Example of Caching with Maven:

cache:
  paths:
    - .m2/repository

Runners

GitLab Runners execute pipeline tasks. Runners can be shared across projects or set up as dedicated instances.

Setting up a Runner:

1. Configure a GitLab Runner on a server or local machine.
2. Register the Runner with your GitLab project: gitlab-runner register
3. Use the Runner in your pipeline: tags: - docker

Best Practices for GitLab CI/CD

1. Modular Pipelines: Keep pipeline logic focused on specific tasks to enhance readability and debugging.
2. Run Jobs in Parallel: Use dependencies and parallel stages to optimize pipeline execution time.
3. Use Variables: Store API keys, configuration settings, and sensitive data as environment variables to keep .gitlab-ci.yml secure.
4. Test on Multiple Environments: Define separate jobs for testing different environments (e.g., Node.js, Java) or versions.
5. Clean Up Resources: Use cleanup jobs to remove temporary deployment spaces like review apps.

Final Thoughts

GitLab CI/CD is a powerful tool for automating builds, tests, and deployments. By defining workflows in .gitlab-ci.yml, managing environments, and leveraging features such as caching, artifacts, and runners, you can create efficient pipelines tailored to your development needs.

Start small with a basic pipeline and expand it as your project requires. With GitLab CI/CD, your team can ship high-quality software faster and more reliably.