Where Bitbucket Pipelines fits in the Atlassian stack

Bitbucket is Atlassian’s Git hosting product (Cloud SaaS or Data Center on your infrastructure). Bitbucket Pipelines is the native CI/CD service: pipelines are defined in Git, triggered by pushes and pull requests, and executed on managed or self-hosted runners.

Do not confuse Pipelines with Bamboo, Atlassian’s older standalone CI server. Many enterprises still run Bamboo for Jira-linked release trains; greenfield teams on Bitbucket Cloud typically standardize on Pipelines because configuration lives beside application code and billing is tied to build minutes rather than agent licenses.

If you already know Git from Git & GitHub in depth, the mental model transfers: branches, tags, and merge strategies drive when automation runs; Pipelines defines what runs.

Architecture: triggers, runners, and containers

Every pipeline run starts from an event—usually a push, pull request, or a manual/custom trigger. Bitbucket evaluates bitbucket-pipelines.yml, resolves which pipeline definition matches the branch or tag, allocates a runner, and executes steps sequentially unless you declare parallelism.

• Cloud runners — Atlassian-hosted Linux containers (size: 1×, 2×, 4×, 8× memory/CPU tiers). Each step runs in a fresh container image you specify (often atlassian/default-image or a language-specific image).
• Self-hosted runners — You register runners in your VPC or datacenter for private network access, larger machines, or compliance boundaries. The YAML is largely the same; labels route steps to the right pool.
• Docker as the execution unit — Steps are not “SSH into a pet server”; they are container commands. Custom images must be pullable from a registry Bitbucket can reach (Docker Hub, ECR, GCR, Artifactory, etc.).

Understanding this matters for debugging: a failed step is usually a container exit code, not a daemon on a shared Jenkins agent. Logs, caches, and artifacts are scoped to the run (with explicit handoff via artifacts keywords).

The configuration file: anatomy of bitbucket-pipelines.yml

The file lives at the repository root (or path configured in repo settings). Top-level keys you will use daily:

• image — default Docker image for steps that do not override it.
• definitions — reusable YAML anchors: caches, services, step templates.
• pipelines — maps branches, tags, bookmarks, and custom triggers to step graphs.
• options — global knobs (e.g. max-time, size, docker).

Minimal example—test on every branch, deploy only from main:

image: node:20

definitions:
  caches:
    npm: ~/.npm

pipelines:
  default:
    - step:
        name: Test
        caches:
          - npm
        script:
          - npm ci
          - npm test

  branches:
    main:
      - step:
          name: Test
          caches:
            - npm
          script:
            - npm ci
            - npm test
      - step:
          name: Build & publish
          script:
            - npm run build
            - docker build -t myapp:${BITBUCKET_COMMIT:0:7} .
          services:
            - docker
      - step:
          name: Deploy production
          deployment: production
          trigger: manual
          script:
            - ./deploy.sh production

Key built-in variables include BITBUCKET_BRANCH, BITBUCKET_COMMIT, BITBUCKET_PR_ID, BITBUCKET_BUILD_NUMBER, and BITBUCKET_REPO_SLUG. Use them for image tags, environment names, and audit trails.

Steps, stages, and parallelism

A step is one container run with a script block (shell commands). A parallel block runs multiple steps at once—ideal for splitting test suites by directory or running lint and unit tests concurrently.

pipelines:
  default:
    - parallel:
        - step:
            name: Unit tests
            script:
              - npm run test:unit
        - step:
            name: Lint
            script:
              - npm run lint
    - step:
        name: Integration (needs both)
        script:
          - npm run test:integration

Steps in a list run sequentially; failure stops the pipeline unless you use on-fail strategies (see below). For fan-out/fan-in patterns, combine parallel with a final aggregating step that downloads artifacts from earlier steps.

Branch pipelines, tags, custom pipelines, and PRs

pipelines.default catches branches without a specific rule. pipelines.branches keys off branch names (glob patterns supported). pipelines.tags runs on tag pushes—common for release SemVer tags. pipelines.custom defines named pipelines you trigger manually or via API/schedule—useful for nightly security scans or one-off data migrations.

Pull request pipelines use the pull-requests section:

pipelines:
  pull-requests:
    '**':
      - step:
          name: PR validation
          script:
            - npm ci
            - npm test
    'hotfix/**':
      - step:
          name: Expedited checks
          script:
            - npm run test:smoke

Pair PR pipelines with branch permissions and merge checks in Bitbucket so merges require green builds. That is the social layer on top of YAML—the same principle as protected branches on GitHub.

Caches, artifacts, and services

Caches

Caches speed up dependency downloads (npm, pip, Maven, Gradle, Go modules). Define named caches under definitions.caches and reference them per step. Caches are best-effort: treat them as performance hints, not guaranteed storage. Pin lockfiles so cache hits stay meaningful.

Artifacts

Artifacts pass files between steps or to downloads in the UI:

- step:
    name: Build
    script:
      - make package
    artifacts:
      - dist/**
- step:
    name: Deploy
    script:
      - ls dist/
      - aws s3 sync dist/ s3://my-bucket/

Large binaries belong in object storage or a container registry—not giant artifact tarballs on every commit.

Services (sidecar containers)

The services block starts auxiliary containers (PostgreSQL, Redis, Docker-in-Docker). Example for integration tests:

definitions:
  services:
    postgres:
      image: postgres:16
      variables:
        POSTGRES_PASSWORD: test
        POSTGRES_DB: app_test

pipelines:
  default:
    - step:
        services:
          - postgres
        script:
          - export DATABASE_URL=postgres://postgres:test@localhost:5432/app_test
          - npm run test:integration

Service hostnames are typically localhost from the step container’s perspective; consult Atlassian docs for port mappings per service type.

Docker builds and registries

Building images inside Pipelines requires the Docker service (services: [docker]) and often elevated memory (size: 2x or higher). Login to registries in-script or via secured variables:

- step:
    name: Push image
    services:
      - docker
    script:
      - echo "$AWS_ACCESS_KEY_ID" | docker login -u AWS -p "$(aws ecr get-login-password)" "$ECR_REGISTRY"
      - docker build -t "$ECR_REGISTRY/myapp:$BITBUCKET_COMMIT" .
      - docker push "$ECR_REGISTRY/myapp:$BITBUCKET_COMMIT"

Prefer OIDC federation to AWS (and similar for Azure/GCP) instead of static access keys committed as variables—see the security section below.

Deployments, environments, and promotion

The deployment: keyword on a step binds it to a named deployment environment (e.g. test, staging, production). Environments unlock:

• Deployment variables scoped per environment (different API URLs, kube contexts).
• Deployment permissions (who may deploy to production).
• Visibility in Bitbucket’s Deployments UI—history, diffs, and rollback conversations tied to Jira if integrated.

Use trigger: manual on production steps so automation stops at a human gate unless policy allows continuous deployment. Combine with deployment: production and branch restrictions on main for a two-key process: merge review + deploy click.

For Kubernetes or VM deploys, steps often call kubectl, Helm, Terraform, or Ansible. If you treat cluster desired state as Git-managed manifests, read GitOps principles—Pipelines builds and tests; a controller or a follow-up sync job applies what Git declares.

Variables and secrets

Bitbucket supports variables at workspace, repository, and deployment environment levels:

• Secured variables — masked in logs; not returned to forks in PR builds from untrusted contributors (understand the fork PR security model).
• Default variables — non-secret configuration.
• Deployment variables — override per environment without branching YAML.

Never echo secrets in scripts. Rotate credentials when someone leaves the team or a log leak is suspected. For infrastructure secrets in cluster land, use external secret operators—not plain Git.

OIDC: short-lived cloud credentials

Bitbucket Pipelines can assume cloud IAM roles via OpenID Connect—eliminating long-lived AWS_ACCESS_KEY_ID in repo settings. The pattern:

1. Create an OIDC identity provider in AWS (or equivalent) trusting Bitbucket’s issuer.
2. Map repository/workspace UUID and branch patterns in the role trust policy.
3. Enable OIDC in the pipeline step and call aws sts assume-role-with-web-identity (or use a helper pipe).

- step:
    name: Deploy with OIDC
    oidc: true
    script:
      - export AWS_REGION=us-east-1
      - export AWS_ROLE_ARN=arn:aws:iam::123456789012:role/bitbucket-deploy
      - aws sts assume-role-with-web-identity \
          --role-arn "$AWS_ROLE_ARN" \
          --role-session-name "bb-${BITBUCKET_BUILD_NUMBER}" \
          --web-identity-token "$BITBUCKET_STEP_OIDC_TOKEN" \
          --duration-seconds 900 > creds.json
      # export keys from creds.json, then deploy

Tighten trust policies: require specific repositories, branches (main only for prod), and optionally deployment environment claims.

Conditions, failures, and control flow

• condition — skip or run steps based on changesets (changesets include/exclude paths) or custom expressions—essential for monorepos.
• fail-fast on parallel steps — cancel siblings when one fails.
• on-fail / after-script — cleanup, notifications, or artifact upload even when the main script fails (similar in spirit to GitHub Actions if: always()).
• max-time — cap runaway jobs; pair with sensible test parallelization.

- step:
    name: Frontend only
    condition:
      changesets:
        includePaths:
          - "web/**"
    script:
      - cd web && npm test

Schedules, webhooks, and API triggers

Scheduled pipelines run cron-style jobs (dependency audits, backup drills). Custom pipelines expose buttons in the UI for operators. The REST API lets external systems (Jira automation, release orchestrators) trigger runs with tokens scoped to the workspace.

Outgoing webhooks notify Slack, MS Teams, or internal event buses on pipeline events—wire these for SRE visibility, not only developer convenience.

Reusable config: pipes, templates, and YAML anchors

Pipes are Atlassian-curated (and community) wrapper images for common tasks—deploy to AWS S3, scan with SonarQube, push to Docker Hub—without rewriting boilerplate:

- pipe: atlassian/aws-s3-deploy:1.6.0
  variables:
    AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
    AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
    AWS_DEFAULT_REGION: us-east-1
    S3_BUCKET: my-static-site
    LOCAL_PATH: dist

Use pipes when they match your threat model; for regulated environments, pin versions and vendor your own scripts. YAML anchors under definitions.steps deduplicate repeated step blocks across branches.

Security model and supply chain

• Fork PRs — untrusted forks must not exfiltrate secured variables; understand which variables are available on PR builds and use approval gates for external contributors.
• Branch permissions — restrict who can push to main; require passing builds and reviewers.
• IP allowlists (Cloud) — restrict who can trigger or view if compliance requires.
• Dependency scanning — integrate Snyk, Sonar, or container scanners as dedicated steps; fail builds on critical CVEs when policy says so.
• Image pinning — use digests or immutable tags for base images and pipes; avoid :latest in production paths.
• Least privilege cloud roles — OIDC roles scoped to deploy actions only; no * on production accounts.

Shift-left security aligns with the broader DevOps arc in Historical foundations of DevOps and day-to-day delivery outcomes in DevOps life & business value.

Bitbucket Pipelines vs GitHub Actions vs GitLab CI

Choose based on where your Git lives and who owns the platform contract—not YAML syntax alone. Teams standardized on Atlassian often pick Pipelines to keep permissions, audit, and deployment views in one admin boundary.

Minutes, sizing, and cost discipline

Bitbucket Cloud bills on build minutes consumed by step duration × runner size multiplier. Practices that keep spend predictable:

• Path filters so unrelated monorepo changes do not run every suite.
• Cache dependencies; avoid npm install without lockfiles.
• Right-size containers—do not default to 8× for echo scripts.
• Split fast PR checks from slow nightly custom pipelines.
• Self-hosted runners for heavy integration tests if minutes pricing exceeds infra cost.

FinOps-minded teams track minutes per service; see the FinOps series for framing cloud spend as a product concern.

End-to-end reference pipeline (multi-stage)

image: golang:1.22

definitions:
  caches:
    gomod: ~/.cache/go-build
  steps:
    - step: &lint-test
        name: Lint and unit tests
        caches:
          - gomod
        script:
          - go mod download
          - golangci-lint run ./...
          - go test ./... -race

pipelines:
  pull-requests:
    '**':
      - step: *lint-test

  branches:
    main:
      - step: *lint-test
      - step:
          name: Build image
          services:
            - docker
          script:
            - export IMAGE="registry.example.com/api:${BITBUCKET_COMMIT:0:12}"
            - docker build -t "$IMAGE" .
            - docker push "$IMAGE"
          artifacts:
            - image.txt
      - step:
          name: Deploy staging
          deployment: staging
          oidc: true
          script:
            - echo "registry.example.com/api:${BITBUCKET_COMMIT:0:12}" > image.txt
            - ./helm-upgrade.sh staging
      - step:
          name: Deploy production
          deployment: production
          trigger: manual
          oidc: true
          script:
            - ./helm-upgrade.sh production

  custom:
    security-scan:
      - step:
          name: OWASP dependency check
          script:
            - ./run-dependency-check.sh

Operational checklist (production-minded)

1. Enable Pipelines on the repo; validate YAML in a branch before touching main.
2. Require green PR builds + reviewers on the default branch.
3. Store secrets as secured variables; use OIDC for cloud deploy roles.
4. Tag releases; use deployment environments with manual production triggers if needed.
5. Export logs/metrics to your observability stack; alert on repeated failures and queue time.
6. Document rollback: redeploy prior image tag or revert Git and rerun pipeline.
7. Game-day a failed deploy—pair with Incidents & disaster response.

Common pitfalls

• Assuming caches are durable — always reproducible from lockfiles.
• Running production deploy steps on every branch — scope with branches and deployment.
• Docker-in-Docker without the docker service — mysterious “cannot connect to daemon” errors.
• Huge artifacts — slow uploads; use registries and object storage.
• Static cloud keys in variables — rotate pain and leak risk; adopt OIDC.
• Ignoring fork PR security — external contributors and secrets require policy.
• One giant step — hard to debug; split lint, test, build, deploy for clear failure domains.

Learning path

1. Enable Pipelines; commit a two-step YAML (install + test) on a feature branch.
2. Add PR pipeline; enforce merge checks.
3. Introduce caches and a service container for integration tests.
4. Build and push a container image; promote via deployment environments.
5. Replace static AWS keys with OIDC; narrow IAM trust policies.
6. Add custom scheduled security scan; wire Slack webhook on failure.
7. If on Kubernetes, connect deploy steps to GitOps or Helm patterns from platform docs.

Further reading

• Atlassian — Bitbucket Pipelines documentation (YAML reference, OIDC, self-hosted runners)
• Atlassian — Pipes catalog and deployment guides
• AWS / Azure / GCP docs — federated identity for CI OIDC providers
• DORA research — metrics for pipeline health (lead time, change fail rate)

Blog index · GitHub CI/CD · GitLab CI/CD · Git & GitHub · GitOps principles

Back to blog list