GitLab CI/CD Pipelines — Unlock the Full Power of .gitlab-ci.yml

GitLab CI's .gitlab-ci.yml is the most feature-rich CI configuration format available, but that power comes with complexity. The interaction between rules, only/except, workflow:rules, and pipeline sources (push, merge request, schedule, API) creates a combinatorial explosion of pipeline behavior. A misconfigured rules block can cause pipelines to run twice, not at all, or with the wrong variables.

GitLab's pipeline architecture supports stages (sequential), DAG mode (dependency-based), parent-child pipelines, and multi-project pipelines. Choosing the right architecture depends on your project's structure and deployment needs. Stages are simple but slow (everything in Stage 2 waits for everything in Stage 1). DAG mode allows jobs to start as soon as their specific dependencies finish, significantly reducing total pipeline time.

GitLab Runners come in multiple executors: Docker, Docker Machine, Kubernetes, Shell. Each has different performance characteristics, caching behavior, and security implications. Docker Machine autoscaling can dramatically reduce costs for burst workloads, but configuring the spot instance pool, cache distribution, and concurrent job limits requires expertise.

We design pipelines using GitLab's include keyword with template files stored in a central CI/CD repository. Service-specific .gitlab-ci.yml files extend these templates with overrides. We use extends and !reference for DRY configuration and YAML anchors for variable reuse. The result is a consistent pipeline structure across all projects.

Pipeline architecture uses DAG mode with needs keywords to express actual dependencies between jobs. A typical pipeline has lint, test, and build jobs that start independently as soon as their dependencies are met, rather than waiting for all jobs in the previous stage. This can reduce pipeline time by 40% compared to stage-based execution.

Review apps are configured for every merge request, spinning up isolated environments with their own database and unique URL. The review app deploys automatically when a merge request is opened and tears down when it is merged or closed. We use GitLab's environment management with cleanup rules so that stale review apps are automatically removed after a configurable period.

A professionally configured GitLab CI/CD setup:

• Template-based pipelines — reusable CI templates with project-specific overrides
• DAG optimization — dependency-based job execution for faster pipeline completion
• Review apps — per-merge-request environments with automatic lifecycle management
• Runner optimization — Docker Machine autoscaling or Kubernetes executor with proper resource limits
• Caching strategy — distributed cache with per-branch and per-job-type cache keys
• Security scanning — SAST, DAST, dependency scanning, and container scanning integrated
• Merge request pipelines — lightweight pipelines for MRs with full pipelines on merge

Use workflow:rules at the top of your .gitlab-ci.yml to control when pipelines run at all. Without it, every push and every merge request creates a pipeline, leading to duplicate runs. Configure workflow:rules to run pipelines for merge requests and for pushes to the default branch, but not for both simultaneously.

GitLab's interruptible keyword is underused but powerful. Mark jobs as interruptible: true so that when a new commit is pushed to the same branch, the old pipeline is automatically cancelled. This prevents wasted CI minutes on outdated commits and is especially valuable for merge request pipelines where developers push multiple commits in quick succession.