Setup & Sizing
Because every task spins up a fresh container, Concourse’s performance is dominated by worker-level resource limits and garbage collection — not raw controller sizing the way Jenkins is.
Worker resource limits
Section titled “Worker resource limits”Set explicit CPU/memory requests and limits per worker rather than trusting node defaults — because every task runs in its own container, worker-level limits are what actually prevent one runaway task from starving the rest of the pipeline’s throughput. Setting requests equal to limits (Guaranteed QoS) is worth the loss of bin-packing efficiency on a dedicated worker pool: predictable per-worker performance matters more than squeezing extra density out of the node, since Concourse is already isolating work at the task-container level.
Garbage collection
Section titled “Garbage collection”Concourse GCs volumes and containers on its own, but the interval is worth tuning explicitly on constrained infrastructure rather than trusting defaults sized for something larger. A GC interval too long on a small worker pool lets ephemeral-container churn outpace cleanup and quietly eat disk.
Picking a tier
Section titled “Picking a tier”- Small (a couple of pipelines, low concurrency). A handful of workers with modest resource requests is enough; default GC interval is fine.
- Medium (several teams, steady pipeline traffic). Worker count starts to matter more than per-worker size — favor more, smaller workers over fewer large ones so one team’s task burst doesn’t starve another’s.
- Large (many pipelines, high concurrency, dedicated team capacity). Individual workers can run heavy (in the range of several CPUs and tens of GB of memory each), sized to autoscale within a min/max replica band rather than a fixed count — this is also where team-scoped workers start to earn their complexity, splitting worker capacity by team rather than growing one shared pool indefinitely.
Keep pipelines narrow and focused
Section titled “Keep pipelines narrow and focused”A pipeline representing a handful of jobs schedules and visualizes faster than one enormous pipeline trying to model an entire system’s worth of stages — this is as much about legibility as it is about scheduling performance.
Related