"Building a CI/CD Pipeline From Scratch: A Practical Guid...

Building a CI/CD Pipeline From Scratch: A Practical Guide for Developers

Originally inspired by Akoode's CI/CD pipeline guide — rewritten here with more depth, code, and less hand-waving.

I've seen teams spend hours manually running tests, zipping build artifacts, SSHing into servers, and crossing fingers before every deploy. CI/CD pipelines exist to kill that workflow. This guide skips the theory lecture and gets into how to actually build one.

We'll use GitHub Actions as the CI/CD platform — it's free for public repos, tightly integrated with GitHub, and requires zero external infrastructure to get started.

What CI/CD Actually Does (Plain English)

CI (Continuous Integration): Every time code is pushed or a PR is opened, automatically run your build and tests. Catch breakage early, not in prod.
CD (Continuous Delivery/Deployment): After CI passes, automatically ship the artifact to staging or production — no human clicking "deploy" required.

The pipeline is just a sequence of automated steps triggered by a git event.

Pipeline Architecture

git push / PR open
│
▼
┌─────────────┐
│ Trigger │ ← GitHub webhook fires
└─────┬───────┘
│
▼
┌─────────────┐
│ Build │ ← Install deps, compile, bundle
└─────┬───────┘
│
▼
┌─────────────┐
│ Test │ ← Unit, integration, lint
└─────┬───────┘
│
▼
┌─────────────┐
│ Deploy │ ← Push to staging/prod
└─────────────┘
Each stage is a job. Jobs run on runners (GitHub-hosted VMs or your own). They can run in parallel or sequentially with dependencies between them.

Setting Up Your First Pipeline with GitHub Actions

Create this file in your repo:

.github/
workflows/
ci-cd.yml

Minimal CI Pipeline (Node.js Example)

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build-and-test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run linter
        run: npm run lint

      - name: Run tests
        run: npm test

      - name: Build
        run: npm run build

That's it. Push this file, and every PR gets auto-tested. No server, no webhook config.

Adding CD: Deploy to a Server

After CI passes, deploy to production. Here we'll SSH into a VPS and pull + restart:

  deploy:
    needs: build-and-test       # only runs if CI passes
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'   # only on main branch

    steps:
      - name: Deploy via SSH
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ secrets.DEPLOY_HOST }}
          username: ${{ secrets.DEPLOY_USER }}
          key: ${{ secrets.DEPLOY_SSH_KEY }}
          script: |
            cd /var/www/myapp
            git pull origin main
            npm ci --omit=dev
            pm2 restart myapp

Store your SSH key and server IP in GitHub Secrets (Settings → Secrets and variables → Actions). Never hardcode credentials in the YAML.

Docker-Based Deploy (More Portable)

If you're deploying containers:

  build-and-push:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Log in to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_TOKEN }}

      - name: Build and push Docker image
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: yourusername/myapp:${{ github.sha }}

Using the commit SHA as the image tag gives you a clean audit trail — every deploy is traceable to a specific commit.

Environment Separation

Don't deploy everything to production. Use branch-based environment targeting:

on:
  push:
    branches:
      - main       # → production
      - staging    # → staging env
      - 'feat/**'  # → preview envs (optional)

Pair with GitHub Environments (Settings → Environments) to add manual approval gates before production:

  deploy-prod:
    environment:
      name: production
      url: https://myapp.com

GitHub will pause and require an approver before proceeding. Useful for regulated teams or high-stakes deploys.

Caching Dependencies

Don't reinstall node_modules from scratch on every run. Cache it:

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'        # ← this line handles caching automatically

For Python:

      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'

This alone can cut pipeline runtime by 60–70% on most projects.

Matrix Testing: Test Across Multiple Versions

Need to support Node 18 and 20? Don't write two jobs:

  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [18, 20, 22]

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci && npm test

GitHub runs these in parallel — fast and zero duplication.

Secrets Management

Rules:

Store secrets in GitHub Secrets, not in .env files committed to the repo
Use environment-scoped secrets for prod vs staging differences
Rotate secrets regularly (SSH keys, API tokens)
Never echo secrets in run steps — they'll be masked in logs, but it's still bad practice

      - name: Deploy
        env:
          API_KEY: ${{ secrets.PROD_API_KEY }}
        run: ./deploy.sh

When to Use CI/CD

✅ Any team with more than one developer

✅ Frequent deploys (more than once a week)

✅ You have a test suite (even a small one)

✅ Multiple environments (dev, staging, prod)

✅ Open source projects where contributors submit PRs

When NOT to Use (or Keep It Simple)

❌ Solo hobby project with no test suite — a basic deploy script is fine

❌ Legacy monolith where builds take 45 minutes — fix the build first

❌ Highly regulated environments where automated prod deploys are prohibited — use CD to staging only, with manual prod promotion

Common Mistakes

1. Not pinning action versions

# Bad — can break silently when the action updates
uses: actions/checkout@main

# Good — locked to a specific version
uses: actions/checkout@v4

2. Running everything on every push

Use path filters to skip unnecessary runs:

on:
  push:
    paths:
      - 'src/**'
      - 'package.json'

3. Storing secrets in env files

Don't commit .env.production to the repo. Use GitHub Secrets + a secrets manager (HashiCorp Vault, AWS Secrets Manager) for anything sensitive.

4. No rollback plan

Tag your Docker images with the git SHA. If prod breaks, you can redeploy the previous image in 30 seconds.

Full Pipeline at a Glance

name: CI/CD Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run lint
      - run: npm test

  deploy:
    needs: test
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    environment: production
    steps:
      - uses: actions/checkout@v4
      - name: Build Docker image
        run: docker build -t myapp:${{ github.sha }} .
      - name: Push to registry
        run: |
          echo ${{ secrets.DOCKER_TOKEN }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
          docker push myapp:${{ github.sha }}
      - name: Deploy
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ secrets.DEPLOY_HOST }}
          username: ${{ secrets.DEPLOY_USER }}
          key: ${{ secrets.DEPLOY_SSH_KEY }}
          script: |
            docker pull myapp:${{ github.sha }}
            docker stop myapp || true
            docker run -d --name myapp -p 3000:3000 myapp:${{ github.sha }}

Practical Takeaways

Start small: even a single npm test in CI adds real value
needs: keyword is your sequencing primitive — use it
Branch protection rules + required CI checks = no broken code on main
Commit SHA tagging on Docker images = instant rollback capability
Cache dependencies — it's free performance
Use GitHub Environments for approval gates before prod

The goal isn't a perfect pipeline on day one. It's getting something automated, then adding stages as your confidence and test coverage grow.

推荐订阅源

DEV Community