Linux in CI/CD Pipelines

Every CI/CD pipeline — GitHub Actions, GitLab CI, Jenkins — eventually resolves down to the same thing: a Linux machine, checking out code, running a sequence of shell commands, and reporting whether they exited with status 0. The YAML that defines a pipeline is a thin orchestration layer; the actual work — installing dependencies, running tests, building an image, deploying an artifact — happens the same way it would on a server you SSH into by hand. Understanding CI/CD as "Linux administration with no human at the keyboard" turns a debugging session on a mysterious pipeline failure from guesswork into the same systematic process covered in Troubleshooting Workflow — because that is exactly what it is.

What You Will Learn

By the end of this article you will be able to:

• explain what a CI/CD runner actually is at the operating-system level;
• read a pipeline's shell steps as ordinary Bash, with the same exit-code discipline covered earlier in this course;
• build and push a container image from inside a pipeline;
• explain the security implications of secrets and ephemeral runners;
• diagnose a failing pipeline step using the same tools used for any other Linux process.

A Runner Is Just a Linux Host

A CI/CD runner — GitHub Actions calls it a "runner," GitLab calls it a "runner" too, Jenkins calls it an "agent" — is a machine (a container, a VM, or occasionally bare metal) that checks out a Git repository and executes a sequence of shell commands defined in a configuration file. Nothing about the execution environment is magic:

Trigger (push, PR, schedule)
   │
   ▼
Scheduler assigns a runner (a Linux VM or container)
   │
   ▼
Runner: git clone/checkout
   │
   ▼
Runner: execute each pipeline step as an ordinary shell command
   │
   ▼
Runner: report exit code of each step; non-zero halts the pipeline (by default)

This is why Exit Codes and Error Handling matters directly for CI/CD: a pipeline step that suppresses a command's non-zero exit status (a stray || true, or a pipeline that only checks the last command in a pipe without pipefail) will report success to GitHub Actions or GitLab even though the underlying command failed — the CI system has no independent way to know a step "actually" failed beyond the exit code it observes.

# .github/workflows/ci.yml — a minimal example
name: CI
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: |
          set -euo pipefail
          pip install -r requirements.txt
          pytest

set -euo pipefail at the top of the run block is not a CI-specific convention — it is the same defensive Bash pattern from Exit Codes and Error Handling, applied here so that a failed pip install halts the job immediately instead of proceeding to run tests against half-installed dependencies and producing a confusing, unrelated failure.

Runners Are (Usually) Containers, and That Matters

Most hosted CI providers run each job inside a fresh container or VM, discarded after the job completes. This has two direct, practical consequences that connect straight back to Linux Containers:

• No state survives between runs, unless explicitly cached or restored — a package installed by hand in one run is gone in the next, which is why pipelines declare their dependencies explicitly (requirements.txt, package.json) rather than relying on a runner's pre-existing state.
• Every run starts from the same known image, which is what makes CI results reproducible — a test that fails in CI but "works on my machine" is frequently explained by exactly the dependency-drift problem containers were built to solve in the first place.

# A typical hosted runner is already a container itself
cat /proc/1/cgroup

0::/

An empty or minimal cgroup path at PID 1 on a hosted runner is a strong signal the "machine" you were just handed is itself a container, not a dedicated VM — worth checking when a pipeline step behaves unexpectedly around resource limits, since a shared, size-limited runner container can silently throttle a build that assumes it has a full host's CPU and memory to itself.

  - name: Cache pip packages
    uses: actions/cache@v4
    with:
      path: ~/.cache/pip
      key: pip-${{ hashFiles('requirements.txt') }}

Building and Pushing an Image from a Pipeline

A common pipeline stage builds the same kind of image covered in Docker and Podman in Practice, then pushes it to a registry for deployment:

  build:
    runs-on: ubuntu-latest
    needs: test
    steps:
      - uses: actions/checkout@v4
      - name: Log in to registry
        run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login myregistry.example.com -u ci --password-stdin
      - name: Build and push
        run: |
          docker build -t myregistry.example.com/myapp:${{ github.sha }} .
          docker push myregistry.example.com/myapp:${{ github.sha }}

Tagging the image with ${{ github.sha }} — the exact commit hash — rather than a mutable tag like latest is deliberate: it makes every deployed image traceable back to the exact source code it was built from, which matters enormously the first time a production incident requires answering "which commit is actually running right now?"

docker login ... --password-stdin reading the token from stdin, rather than as a command-line argument, avoids the token appearing in the process list (ps aux) or in shell history — the same principle behind never passing a password as a bare CLI argument covered in SSH Key Authentication.

Secrets: The Highest-Value Target in Any Pipeline

A CI pipeline typically holds credentials with real production access — registry tokens, cloud provider API keys, deployment SSH keys — which makes the pipeline configuration itself a security-sensitive surface, not just the servers it deploys to.

# Wrong: secret visible in the workflow file and in every log line that echoes $TOKEN
run: curl -H "Authorization: Bearer sk_live_abc123..." https://api.example.com

# Right: secret injected from the provider's secret store at run time, never echoed
run: curl -H "Authorization: Bearer ${{ secrets.API_TOKEN }}" https://api.example.com

Most CI providers automatically mask a registered secret's value in log output — but only for the exact string; a secret that gets base64-encoded, concatenated, or otherwise transformed before being printed can still leak through the mask, which is a real and recurring failure mode worth testing for deliberately rather than assuming the platform's masking is infallible.

Diagnosing a Failing Pipeline

A failing pipeline step is debugged the same way any Linux process failure is debugged, because it is one:

1. Read the actual exit code and output, not just the "failed" status — the full log for the failing step almost always states the real underlying command that returned non-zero.
2. Reproduce locally. Since the runner is a known, often publicly documented image (ubuntu-latest on GitHub Actions, for instance, has a published, versioned package list), the same steps can usually be run inside an equivalent local container to reproduce the failure without waiting on the CI queue:

docker run --rm -it -v "$PWD:/workspace" -w /workspace ubuntu:24.04 bash
# then run the same commands the pipeline runs, one at a time

1. Check for environment differences, not just code differences — a test that depends on a locally installed tool, a timezone assumption, or a hardcoded localhost path frequently passes on a developer's machine and fails on a clean runner precisely because the runner has none of that implicit local state.

Common Mistakes

• Suppressing a step's real exit code with || true "to make the pipeline green." This does not fix the underlying failure — it hides it, and the next person to touch the pipeline has no signal that anything is wrong until the hidden failure causes damage somewhere else.
• Committing a secret "temporarily" to test a pipeline. As covered above, Git history retains it permanently; the correct fix after any accidental commit is rotating the credential, not just removing the line in a follow-up commit.
• Assuming a hosted runner has the same resources or state as a personal machine. Hosted runners are typically resource-constrained containers or VMs with none of a developer's locally installed tools, cached packages, or environment variables.
• Tagging every built image latest. This makes it impossible to know, after the fact, exactly which commit produced the image currently running in production — an immutable, commit-derived tag solves this directly.

Exercises

1. Take a Bash script you have already written for this course (the backup script or healthcheck script work well) and wrap it in a minimal GitHub Actions workflow that runs it on every push. Confirm the workflow reports failure correctly by deliberately introducing a bug.
2. Explain, in your own words, why docker login --password-stdin is safer than passing a password with -p on the same command line, referencing what ps aux would show in each case.
3. Write a one-paragraph incident response plan for "a secret was accidentally committed to a public repository three commits ago," listing the steps in the order they need to happen.
4. Pull the same base image your CI provider documents using (for example ubuntu:24.04), and run your pipeline's shell steps manually inside it with docker run -it. Note any command that behaves differently than it does on your own machine, and explain why.

Summary

A CI/CD runner is an ordinary Linux host — usually itself a container — executing shell commands in sequence and reporting each one's exit code; the pipeline YAML is orchestration on top of the same shell scripting, exit-code discipline, and container tooling covered throughout this course. Ephemeral, disposable runners and provider-managed secret stores exist because a persistent build machine and a hardcoded credential are exactly the same risk as an unpatched, over-privileged server — the mitigations are identical, just applied to build infrastructure instead of production infrastructure. Linux in the Cloud continues from here, since the artifact a pipeline like this produces almost always ends up running on a cloud instance provisioned the way that article describes.

References

• GitHub Actions Documentation: Understanding GitHub Actions
• GitHub Actions Documentation: Using secrets in GitHub Actions
• GitLab Documentation: CI/CD variables
• Docker Docs: Docker login