Team-Scoped Workers via Helm

A team-scoped worker is a dedicated worker pool restricted to one team’s steps, deployed as its own Helm release rather than added to a shared, general-purpose pool.

Why

A shared worker pool means one team’s heavy or misbehaving pipeline can starve everyone else’s queue time. Dedicating capacity per team gives each one a pool that scales and fails independently — the same isolation instinct behind tiering resources by size, just drawn along team lines instead of load tiers. It’s worth reaching for once shared-pool contention becomes a recurring complaint rather than an occasional blip.

How

The stock concourse-chart needs no fork for this — deploy a second (or Nth), worker-only release pointed at the same web/ATC, and restrict it to one team by setting the worker’s team through the chart’s generic concourse: values passthrough, which maps directly to the underlying binary’s environment variables. Pair it with an autoscaling range instead of a fixed replica count, and size the worker itself generously — a dedicated pool can run each worker heavier (multiple CPUs, tens of GB of memory) since it isn’t sharing a node with anyone else’s work:

concourse:
  worker:
    team: team-a
    autoscaling:
      maxReplicas: 20
      minReplicas: 5

worker:
  labels:
    team: team-a
  resources:
    requests:
      cpu: "8"
      memory: "24Gi"
    limits:
      cpu: "8"
      memory: "24Gi" # requests == limits: Guaranteed QoS, see Setup & Sizing

persistence:
  labels:
    team: team-a

Decoupling the worker’s identity from the release

If the worker authenticates to cloud infrastructure (pulling images, writing artifacts) via a Kubernetes service account bound to an IAM identity, create that service account once, outside the worker’s own Helm release, and have the chart reference it rather than create it:

rbac:
  # The worker's Kubernetes service account is created once, pool-independent,
  # and referenced here rather than templated per release — so a worker pool
  # can be recreated or resized without provisioning a new IAM binding each time.
  create: false
  workerServiceAccountName: concourse-team-a-worker

This is a small thing that saves a real class of pain: without it, every worker-pool change becomes a Helm change and an IAM change, and the two don’t always land atomically.

Delivering the worker key

A worker-only release doesn’t get its own key — it authenticates with the exact same worker private key and web-node host public key as the core worker pool, since they’re all registering against the same web/ATC. Since the team release is separate from core, that key is pulled into the cluster from a secrets manager rather than baked into chart values — so rotation happens once, in one place, and every pool (core and team alike) picks it up without a redeploy. The worker itself — core pool and every team pool — runs in one shared namespace, since they all need to reach the same web/ATC:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: concourse-team-a-worker
  namespace: concourse-prod
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: cluster-secret-store
  target:
    name: concourse-team-a-worker
    template:
      mergePolicy: Merge
      engineVersion: v2
      data:
        worker-key: '{{ .workerKey | b64dec }}'
  data:
    - secretKey: host-key-pub
      remoteRef:
        key: concourse-ci-common
        property: hostKeyPub
    - secretKey: workerKey
      remoteRef:
        key: concourse-ci-common
        property: workerKey

The same credential-scoping instinct as Jenkins applies here: the team’s worker gets exactly the key and identity it needs to authenticate as a worker, not a broad, shared credential.

Team-scoped pipeline secrets

The worker’s own key isn’t the only secret a dedicated team pool needs — pipelines running on it typically need their own source-control auth and alerting credentials too, pulled from a team-scoped path in the secret store rather than the shared common one the worker key uses. Unlike the worker itself, these secrets live in their own per-team namespace, separate from the shared namespace the workers run in:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: concourse-secrets-team-a
  namespace: concourse-prod-team-a
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: cluster-secret-store
  target:
    name: concourse-secrets-team-a
  data:
    - secretKey: github-app-id
      remoteRef:
        key: concourse-ci-common
        property: githubAppId
    - secretKey: github-app-ssh-key
      remoteRef:
        decodingStrategy: Base64
        key: concourse-ci-common
        property: githubAppSSH
    - secretKey: slack-alert-url
      remoteRef:
        key: concourse-ci-common
        property: slackAlertURL
    - secretKey: team-alerts-webhook
      remoteRef:
        key: concourse-team-a-secrets
        property: alertsWebhook

Most of these (GitHub App auth, the shared Slack integration) come from the same common path every team pool reads from — only the last one, the team’s own alert destination, comes from a path scoped to that team specifically. That split mirrors the worker-key pattern: shared infrastructure credentials live in one common place, team-specific ones live in their own.

Related

• Setup & Sizing
• Jenkins: credential scoping