Includes, templates and components

Share pipeline code across projects, and pin it so nobody else changes your pipeline.

Intermediate14 min · lesson 5 of 17
Lesson files
The scripts, test data and local test servers this lesson uses, exactly as they ran on the lab machine (21 files, 7 KB): reuse.tar.gz. The lab VM shares no folders with your computer, so fetch them inside the VM: cd ~/lab && curl -fsSLO https://secopslog.com/lab-files/ci-gitlab/reuse.tar.gz && tar -xzf reuse.tar.gz, which creates ~/lab/reuse/. SHA-256: e37ba454df69c51a2ac7307186c505ec920b441fa453d92202425ddee5b504c7

Thirty projects copy the same test job. When the Python version changes, someone opens thirty merge requests. So the platform team moves the job into a shared project and every pipeline includes it. That fixes the copies and creates a new kind of dependency: whoever controls the shared file now controls thirty pipelines, including the jobs that hold their secrets.

A shared templates project

The lab has two projects. ci-templates holds shared configuration and has two tags on the same commit: v1.0.0 for the plain include, and 1.0.0 because components are versioned with plain semantic version numbers. app uses it in several ways:

templates/python.yml
# Shared job definitions. Hidden jobs (names starting with a dot) never run on their own.
.python:
image: secopslog-py:3.15
before_script:
- python --version
.python-test:
extends: .python
script:
- python -m pytest -q
.audit:
script:
- echo "commit $CI_COMMIT_SHORT_SHA built by $GITLAB_USER_LOGIN"
templates/lint.yml
# A CI/CD component: a template with typed inputs, included by path and version.
spec:
inputs:
paths:
default: "."
stage:
default: test
---
ruff:
stage: $[[ inputs.stage ]]
image: secopslog-py:3.15
script:
- ruff check $[[ inputs.paths ]]
app/.gitlab-ci.yml
include:
- project: root/ci-templates
ref: v1.0.0 # a tag: this file changes only when you change this line
file: /templates/python.yml
- component: $CI_SERVER_FQDN/root/ci-templates/lint@1.0.0
inputs:
paths: calc.py
- project: root/ci-templates
ref: main # a branch: whoever can push to it changes your pipeline
file: /templates/notify.yml
test:
extends: .python-test
report:
extends: .python
script:
- !reference [.audit, script]
- echo "tests are in the test job"
MechanismWhat it doesHere
include: projectPulls YAML from another project at a refpython.yml at tag v1.0.0, notify.yml at branch main
extendsMerges a (usually hidden) job into this onetest extends .python-test, which extends .python
!referenceReuses one part of another job, such as its scriptreport takes .audit's script lines
componentA versioned template with typed inputslint@1.0.0 with paths: calc.py

$CI_SERVER_FQDN in the component path is this GitLab's host name, and the .post stage is a built-in stage that always runs last. Jobs whose names start with a dot are hidden: they never run, they only exist to be extended. A component lives in a templates/ directory and declares its inputs in a spec: header. The component uses its inputs through $[[ inputs.name ]] placeholders, and the caller passes values under inputs:. An unknown input name fails the pipeline instead of passing silently.

Run it, and see what it really runs

ubuntu@secopslog-cicd:~/lab/reuse · GitLab 19.5.3
$ ./up.sh
runner lab-runner: online (tags: lab,docker) GitLab 19.5.3 is up on http://127.0.0.1:8929
$ ./seed.sh ci-templates templates/ && git -C templates tag 1.0.0 && git -C templates tag v1.0.0 && git -C templates push -q origin 1.0.0 v1.0.0 && git -C templates tag --list | cat
pushed ci-templates: f6b65e0 first commit 1.0.0 v1.0.0
$ ./seed.sh app app/ && ./pipeline.sh app/
pushed app: b63e63a first commit --- job ruff (test): success $ ruff check calc.py All checks passed! --- job test (test): success $ python --version Python 3.15.2 $ python -m pytest -q . [100%] 1 passed in 0.01s --- job report (test): success $ python --version Python 3.15.2 $ echo "commit $CI_COMMIT_SHORT_SHA built by $GITLAB_USER_LOGIN" commit b63e63a7 built by root $ echo "tests are in the test job" tests are in the test job --- job notify (.post): success $ echo "notify: pipeline $CI_PIPELINE_ID finished" notify: pipeline 1 finished pipeline #1: success

Four jobs, none of them written out in full in app's file. To review what a pipeline will actually run, read the merged configuration, which the lint API returns with every include pulled in:

ubuntu@secopslog-cicd:~/lab/reuse · GitLab 19.5.3
$ ./gl.sh '/projects/root%2Fapp/ci/lint' | jq -r '.merged_yaml' | awk '/^(test|ruff):/{p=1; print; next} /^[^ ]/{p=0} p'
ruff: stage: test image: secopslog-py:3.15 script: - ruff check calc.py test: image: secopslog-py:3.15 before_script: - python --version extends: ".python-test" script: - python -m pytest -q

That expanded view is what a reviewer should look at for a pipeline change, because the file in the merge request may be three lines that pull in three hundred.

A branch include moves under you

notify.yml is included from main, so it changes whenever someone pushes to the templates project. Play that someone:

ubuntu@secopslog-cicd:~/lab/reuse · GitLab 19.5.3
$ cp notify-changed.yml templates/templates/notify.yml && ./commit.sh templates/ "notify: send more"
pushed: 101e2aa notify: send more to main
$ ./gl.sh "/projects/root%2Fapp/pipeline" -X POST -d ref=main -o /dev/null && sleep 3 && ./pipeline.sh app/ | sed -n "/--- job notify/,\$p"
--- job notify (.post): success $ echo "notify: pipeline $CI_PIPELINE_ID finished" notify: pipeline 2 finished $ echo "also sending CI_JOB_TOKEN (${#CI_JOB_TOKEN} characters) to collector.example" also sending CI_JOB_TOKEN (29 characters) to collector.example pipeline #2: success

Nothing changed in app. No merge request there, no review there, and its next pipeline ran a new command that talks about its job token. In a real attack that line would send the token, or any variable the job can see, to the attacker. Every project that includes from a branch trusts everyone who can push to that branch, on top of its own developers.

ReferenceCan change under you?Use for
Branch (ref: main)Yes, on every pushYour own experiments, never shared CI
Tag (ref: v1.0.0, @1.0.0)Only if tags can be moved: protect themNormal use, together with protected tags
Commit SHANoWhere you need certainty, such as release pipelines
Watch out
A tag pins only as long as nobody moves it. Protecting the version tag pattern in the templates project limits who can move it to that project's Maintainers, who can still delete a tag and create it again on another commit. Only a full 40-character SHA can't change. For components, ~latest and partial versions such as @1.0 move with every new release. Pinning the top-level include doesn't pin what it pulls in either: an included file can include others from a branch, or use an image by tag. Always write ref: (without it you get the default branch), and pin images by digest.

Hands-on

Pin notify.yml to the commit before the change (git -C templates rev-parse HEAD~1) by putting that SHA in ref:, push, and check the notify job. Then push another change to templates/templates/notify.yml on main and run app's pipeline again with ./run-pipeline.sh app. Does app see it? Finish with ./down.sh.

What you should see: after the pin, ref: holds a 40-character SHA and notify prints notify: pipeline 3 finished, the version from before the change. After a new commit to notify.yml on the templates project's main, app's next pipeline still prints the same single line. and one more thing never appears, because app asks for a commit, and a commit can't change. The cost is that you also don't get fixes until you move the SHA yourself.

Remember

Shared CI configuration is code that runs with your project's secrets. Include it by tag or SHA, protect the tags, and review the merged configuration, not just the file that includes it.

Quick check
01A project includes templates/deploy.yml from a shared project with ref: main. What does that mean for its security?
Correct — The lab showed a new command appearing in app without any change in app.
Incorrect — Includes are fetched each time a pipeline is created.
Incorrect — Whoever can push or merge to that branch can affect it, which depends on its protection.
Incorrect — Includes work the same on every branch.
02What is a hidden job, such as .python-test?
Incorrect — Hidden jobs do not run at all.
Incorrect — That would be a rule, not a hidden job.
Correct — The leading dot makes GitLab skip it as a job.
Incorrect — Template jobs are ordinary jobs. Hidden is about the dot.
03A reviewer approves a merge request that adds two include: lines. Why is the merged configuration worth reading?
Incorrect — The API only validates and expands. It does not enforce reviews.
Incorrect — Merging is how GitLab reads the file, with no speed difference.
Incorrect — Includes apply to the merge request pipeline too.
Correct — What runs is the expanded configuration.

Related