Gitlab Ci Optimizer

Optimize GitLab CI/CD pipelines — analyze .gitlab-ci.yml for speed, cost, caching, parallelism, DAG dependencies, and runner efficiency. Use when asked to sp...

Analyze GitLab CI/CD pipeline configurations to find speed, cost, and reliability improvements. Examines .gitlab-ci.yml for caching gaps, missing parallelism, inefficient job ordering, bloated Docker images, redundant work, and misconfigured runners. Produces a concrete optimization plan with estimated time savings.

Use when: "speed up our CI", "pipeline takes too long", "optimize gitlab ci", "review our .gitlab-ci.yml", "reduce build costs", "fix flaky pipeline", or when a pipeline configuration needs improvement.

Analysis Steps

1. Parse Pipeline Structure

Read the .gitlab-ci.yml and any included files to build a complete picture:

# Find the main CI config
cat .gitlab-ci.yml

# Find all included CI files
grep -r "include:" .gitlab-ci.yml
find . -name "*.gitlab-ci.yml" -o -name ".gitlab-ci*.yml" | head -20
find . -path "*/.gitlab/ci/*.yml" | head -20

# Check for CI/CD variables defined in the file
grep -E "variables:" .gitlab-ci.yml

# List all jobs and their stages
grep -E "^[a-zA-Z_][a-zA-Z0-9_-]*:" .gitlab-ci.yml | grep -v "^#" | grep -v "stage:" | grep -v "variables:" | grep -v "include:" | grep -v "default:" | grep -v "workflow:"

For each job, extract: name, stage, runner tags, Docker image, needs/dependencies, cache/artifacts config, rules/conditions, estimated duration, and whether it runs on every commit or only specific branches.

2. Analyze Stage Dependencies

Map the execution flow to find bottlenecks:

Stage 1: build       [job-a: 3min] [job-b: 5min]
                      ↓              ↓
Stage 2: test        [job-c: 8min, needs: job-a] [job-d: 2min, needs: job-b]
                      ↓
Stage 3: deploy      [job-e: 1min, needs: job-c, job-d]

Critical path: job-b (5m) → job-d (2m) → job-e (1m) = 8 min
              OR: job-a (3m) → job-c (8m) → job-e (1m) = 12 min ← BOTTLENECK

Total wall time: 12 min (limited by the longest path)
Total compute time: 3 + 5 + 8 + 2 + 1 = 19 min (what you pay for)

Key questions:

• Which job is the longest on the critical path? (optimize this first)
• Are there jobs running sequentially that could run in parallel?
• Are there jobs in the same stage that have no actual dependency on each other?

3. Evaluate Caching

Check for these common caching problems:

Missing cache entirely:

# BAD: Downloads all dependencies every run
install:
  script:
    - npm ci

# GOOD: Cache node_modules between runs
install:
  script:
    - npm ci
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
    policy: pull-push

Cache key analysis:

Key Strategy	When to Use	Invalidation
$CI_COMMIT_REF_SLUG	Branch-specific caches	New branch = cold start
files: [package-lock.json]	Dependency caches	Only when lockfile changes
$CI_JOB_NAME	Job-specific caches	Never (manual clear)
prefix: $CI_COMMIT_REF_SLUG + files:	Best of both worlds	Branch + lockfile change

Cache policy optimization: Jobs that only READ the cache should use policy: pull (saves upload time). Only the install job should use policy: pull-push.

Cache vs Artifacts decision:

• Cache = best-effort, speeds up repeated runs, may not be available
• Artifacts = guaranteed, passes files between jobs in the same pipeline
• Rule: Use artifacts for build outputs that downstream jobs NEED. Use cache for dependencies that are expensive to re-download.

4. Optimize Docker Images

Problem: Using large base images

# BAD: 1.2 GB image, takes 45 seconds to pull
build:
  image: node:18

# BETTER: 180 MB image, pulls in 5 seconds
build:
  image: node:18-alpine

# BEST: Pre-built image with your dependencies baked in
build:
  image: registry.gitlab.com/my-org/ci-images/node:18

Create custom CI images when before_script takes >30s or you install system packages every run. Use GitLab's dependency proxy (${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/node:18-alpine) to cache Docker Hub images and avoid rate limits.

5. Implement DAG Dependencies

Replace stage-based ordering with needs: for parallel execution:

# Without DAG: test-frontend waits for ALL build jobs (including build-backend)
# With DAG: test-frontend starts immediately after build-frontend finishes

test-frontend:
  stage: test
  needs: [build-frontend]     # Start as soon as build-frontend finishes

test-backend:
  stage: test
  needs: [build-backend]      # Start as soon as build-backend finishes

deploy:
  stage: deploy
  needs: [test-frontend, test-backend]  # Start when BOTH tests pass

Impact: A pipeline with 5 jobs (3+5+8+2+1 min) drops from 19 min wall time to 12 min (37% faster).

6. Apply Parallelism

Test splitting with parallel:

test:
  stage: test
  parallel: 4
  script:
    - TOTAL=$CI_NODE_TOTAL
    - INDEX=$CI_NODE_INDEX
    # Split test files across parallel jobs
    - |
      TEST_FILES=$(find tests/ -name "*.test.js" | sort | awk "NR % $TOTAL == $INDEX")
      npx jest $TEST_FILES
  artifacts:
    reports:
      junit: junit.xml

Use parallel:matrix for multi-environment testing (e.g., NODE_VERSION: ["16", "18", "20"] x DB: ["postgres", "mysql"]).

7. Reduce Redundant Work

Three key techniques:

1. Path-based filtering — only run jobs when relevant files change:

test-frontend:
  rules:
    - changes: ["frontend/**/*", "package-lock.json"]
      when: always
    - when: never

2. Auto-cancel outdated pipelines:

workflow:
  auto_cancel:
    on_new_commit: interruptible

3. Pass artifacts, don't rebuild — use artifacts: paths: [dist/] with expire_in: 1 hour and needs: in downstream jobs.

8. Optimize Runner Configuration

Match runner size to job requirements:

Job Type	Recommended Size	Why
Build (compiled)	large (4 CPU)	Compilation is CPU-bound
Unit tests	medium (2 CPU)	Moderate CPU, moderate RAM
Lint/format	small (1 CPU)	Trivial compute
Integration tests	large (4 CPU)	Runs services, needs RAM
Deploy	small (1 CPU)	Just runs scripts/API calls

9. Common Anti-Patterns

Anti-Pattern	Problem	Fix
apt-get install in every job	30-60s wasted per job	Bake into custom Docker image
No cache: on dependency install	Downloads 500MB+ every run	Add cache with lockfile key
All jobs in one stage	Maximum sequential execution	Split into stages + use DAG
artifacts: paths: ["."]	Uploads entire repo as artifact	Only artifact what downstream needs
when: manual without allow_failure	Blocks entire pipeline	Add allow_failure: true for optional manual jobs
No expire_in on artifacts	Storage grows forever	Set expire_in: 1 day for CI artifacts
Using only/except instead of rules	Confusing precedence, deprecated	Migrate to rules: syntax
retry: 2 on flaky tests	Masks real problems, slows pipeline	Fix the flaky test, don't retry
GIT_STRATEGY: clone (default)	Full clone every time	Use GIT_STRATEGY: fetch or GIT_DEPTH: 20
Monorepo without path filtering	Every change triggers all jobs	Use rules: changes: per component

Output Format

# GitLab CI Optimization Report

## Pipeline Overview
- **Stages:** {list}  |  **Jobs:** {count}  |  **Wall Time:** {duration}
- **Critical Path:** {job-a -> job-b -> job-c}

## Findings (ranked by impact)
### 1. {Finding Title} — Impact: {High|Medium|Low}, saves ~{X} min/pipeline
- **Current:** {what it does now}
- **Recommended:** {what it should do}

## Estimated Savings
- Wall time: {X} min -> {Y} min ({Z}% reduction)
- Monthly cost: ${X} -> ${Y} (${Z} saved)
- Developer wait time saved: {hours}/day

Tips

• Measure before optimizing — get baseline pipeline duration from GitLab's CI/CD analytics
• Optimize the critical path first — speeding up non-critical-path jobs saves compute cost but not wall time
• Use interruptible: true on all jobs except deploy — auto-cancels old pipelines when new commits arrive
• Set GIT_DEPTH: 20 globally to avoid full clones (unless you need full git history)
• Use rules:changes: in monorepos to skip unaffected jobs — this is the single biggest optimization for monorepos
• Merge before_script into custom Docker images when the commands don't change between runs
• Profile your scripts — add time prefix to commands to find which step is slow
• Check artifacts:expire_in on all jobs — unlimited artifacts eat storage and slow uploads
• Consider GitLab's parallel keyword to split test suites — 4x parallelism = ~3.5x speedup (Amdahl's law)
• Use dependency proxy for Docker Hub images to avoid rate limiting and speed up pulls