Modules & the Forge

Reusable, shareable units.

A module is a labelled toolbox. One job per box, everything that job needs sealed inside, and a card taped to the lid naming the box, its version, and which other boxes have to arrive on the same van. In Puppet the toolbox is a directory. Inside sit manifests (the Puppet code itself), templates, static files, default data, and Ruby plugins. The card is a file called metadata.json. Hand that directory to a machine that has never heard of SSH hardening in its life, and the machine comes back hardened.

The Puppet Forge, at forge.puppet.com, is the public shelf those boxes sit on. Thousands of them, versioned, free, covering most of what you would otherwise write yourself. You will use it. You should also understand the deal you are signing. Taking a box off a public shelf and wiring it into your codebase hands root on every machine you manage to whoever packed it. Two things make this lesson worth your time. The layout, which is the only reason Puppet can find your code at all. And the supply chain, which is where the bad afternoons come from.

The Directory Names Are Load-Bearing

The layout is not a style guide. It is a lookup table. Think of a library call number: it tells you the floor, the aisle, and the exact spot on the shelf, and nobody has to keep a separate index of where each book was put. Puppet's autoloader (the part that finds code on disk without being told where to look) works the same way, turning a class name into a file path by a fixed rule. Class acme_baseline lives in manifests/init.pp. Class acme_baseline::config lives in manifests/config.pp. Class acme_baseline::sshd::hardening lives in manifests/sshd/hardening.pp. Double colons become directory separators, the final segment becomes the .pp file name, and the first segment is always the module's own directory name. Put a class anywhere else and Puppet will not find it, will not warn you, and will fail at compile time complaining about a class you can see with your own eyes.

Do not build the directory by hand. The Puppet Development Kit (PDK, the official tool for scaffolding and testing modules) writes a valid skeleton, a test harness, and continuous integration config in one command, and gives you a validate-and-test loop that catches the mistakes you would otherwise ship.

cd /home/ops/modules
pdk new module acme_baseline --skip-interview
cd acme_baseline
pdk new class config

pdk (INFO): Creating new module: acme_baseline
pdk (INFO): Module 'acme_baseline' generated at path '/home/ops/modules/acme_baseline', from template 'https://github.com/puppetlabs/pdk-templates'.
pdk (INFO): In your module directory, add classes with the 'pdk new class' command.
 
pdk (INFO): Creating '/home/ops/modules/acme_baseline/manifests/config.pp' from template.
pdk (INFO): Creating '/home/ops/modules/acme_baseline/spec/classes/config_spec.rb' from template.

# after filling in the directories this module actually needs
tree --dirsfirst /home/ops/modules/acme_baseline

/home/ops/modules/acme_baseline
├── data
│   ├── os
│   │   └── RedHat.yaml
│   └── common.yaml
├── files
│   └── issue.net
├── functions
│   └── port_valid.pp
├── lib
│   └── facter
│       └── acme_hardening.rb
├── manifests
│   ├── config.pp
│   ├── init.pp
│   └── sshd.pp
├── spec
│   ├── classes
│   │   └── config_spec.rb
│   ├── default_facts.yml
│   └── spec_helper.rb
├── templates
│   └── sshd_config.epp
├── types
│   └── port.pp
├── CHANGELOG.md
├── Gemfile
├── hiera.yaml
├── metadata.json
├── Rakefile
├── README.md
└── REFERENCE.md
 
11 directories, 20 files

Every one of those names means something to Puppet. manifests/ holds classes and defined types, addressed by path as above. templates/ holds EPP (Embedded Puppet) and ERB (Embedded Ruby) templates, which are files with holes in them that Puppet fills at compile time. files/ holds static content, copied out as-is. Both are referenced by module name and file name only, with the directory silently dropped: templates/sshd_config.epp is called as epp('acme_baseline/sshd_config.epp'), and files/issue.net is fetched as source => 'puppet:///modules/acme_baseline/issue.net'. types/port.pp defines the data type alias Acme_baseline::Port, so a parameter can be checked against your own rules instead of a bare Integer. functions/port_valid.pp defines the Puppet-language function acme_baseline::port_valid. lib/ holds Ruby: custom facts under lib/facter, custom functions under lib/puppet/functions, resource types under lib/puppet/type and their providers under lib/puppet/provider. A tasks/ directory, which this skeleton has not needed yet, holds Bolt tasks, one-off scripts you run on demand that never appear in a catalog. data/ and hiera.yaml carry the module's own defaults, which is the next section. REFERENCE.md is generated from the comments in your code by puppet-strings.

# the class name is right, the file name is not
grep -n 'class' manifests/hardening.pp
echo 'include acme_baseline::config' > /tmp/site.pp
puppet apply --modulepath /home/ops/modules /tmp/site.pp

1:class acme_baseline::config {
Error: Evaluation Error: Error while evaluating a Function Call, Could not find class ::acme_baseline::config for web01.example.com (file: /tmp/site.pp, line: 1, column: 1) on node web01.example.com

git mv manifests/hardening.pp manifests/config.pp
puppet apply --modulepath /home/ops/modules --noop /tmp/site.pp

Notice: Compiled catalog for web01.example.com in environment production in 0.41 seconds
Notice: /Stage[main]/Acme_baseline::Config/File[/etc/issue.net]/ensure: current_value 'absent', should be 'file' (noop)
Notice: Class[Acme_baseline::Config]: Would have triggered 'refresh' from 1 event
Notice: Stage[main]: Would have triggered 'refresh' from 1 event
Notice: Applied catalog in 0.05 seconds

That is the whole debugging story for autoloading. If Puppet says it cannot find a class you have definitely written, check the file path before you check anything else. Nine times in ten the file is one directory too deep, or somebody renamed the module directory and left the old prefix on the class.

Defaults That Travel With the Module

A good module arrives like a form with the common answers already filled in, leaving you to cross out only the ones that are wrong for your estate. That is the job of data/ and the module's own hiera.yaml. Hiera (Puppet's data lookup system) is a stack of answer sheets read in a fixed order: you ask for a key, Hiera works down the stack, and the first sheet holding that key wins. A module can ship its own small stack. That is how a single module knows Red Hat calls the SSH service sshd while Debian and Ubuntu call it ssh, with no thicket of if statements in the manifest.

---
version: 5
 
defaults:
  datadir: data          # relative to the module root
  data_hash: yaml_data
 
hierarchy:
  - name: 'OS family overrides'
    path: "os/%{facts.os.family}.yaml"
 
  - name: 'Common defaults'
    path: 'common.yaml'

---
# keys are fully qualified class parameters: <class>::<parameter>
acme_baseline::sshd_package: 'openssh-server'
acme_baseline::sshd_config:  '/etc/ssh/sshd_config'
acme_baseline::sshd_service: 'ssh'

---
# Package and config path match common.yaml, so only the unit name is listed.
# Red Hat calls it sshd; Debian and Ubuntu call it ssh.
acme_baseline::sshd_service: 'sshd'

When a class parameter is given no explicit value, Puppet looks the key up automatically before falling back to the default written in the class signature. It checks three layers in order: the global layer at /etc/puppetlabs/puppet/hiera.yaml, then the environment layer, then the module layer. Module data comes last, which is exactly the property you want. The author's defaults hold until you set the same key in your own environment data, and then yours wins, with no fork and no patch. This replaced the old params.pp pattern, where defaults lived in a separate class stuffed with case statements. Inherit a module still doing that, and module data is the modern fix.

Two Ways In, and Only One Survives Contact

For a laptop, a lab box, or anything running standalone with puppet apply, the module tool pulls straight from the Forge and walks the dependency graph for you.

puppet module install puppetlabs-apache --version 13.2.0

Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└─┬ puppetlabs-apache (v13.2.0)
  ├── puppetlabs-concat (v10.0.1)
  └── puppetlabs-stdlib (v10.0.1)

Three quirks to file away. The Forge writes a module's full name with a hyphen, puppetlabs-apache, and that hyphen form is what goes in the name field of metadata.json. Dependency entries inside metadata.json use the slash form, puppetlabs/stdlib, and so do Puppetfile lines. Same module either way. On disk, though, the directory is called apache, not puppetlabs-apache, because the autoloader keys off the short name alone. Two authors publishing a module called apache therefore cannot coexist in one environment, which becomes a real constraint the day you fork a community module and keep its name. Last quirk, and it catches people in regulated shops: the integrity check the module tool runs on a downloaded release archive is an MD5 comparison, which guards against a corrupted download and is refused outright on a host running in FIPS mode.

For anything with users on it, that command is the wrong tool. You pin exact versions in a Puppetfile, commit it to a control repository, and let r10k (the deployment tool that turns a git branch plus a Puppetfile into a live Puppet environment) build the environment. A module change becomes a commit, a review, and a deploy, like every other change.

# Forge modules: exact versions. No ranges, no :latest.
# r10k does NOT resolve dependencies, so list every one yourself.
mod 'puppetlabs/stdlib', '10.0.1'
mod 'puppetlabs/concat', '10.0.1'
mod 'puppetlabs/apache', '13.2.0'
 
# Git module: pin a commit, not a branch.
# A branch is a moving pointer; whoever pushes last decides what you deploy.
mod 'acme_baseline',
  git:    'https://git.example.com/infra/puppet-acme_baseline.git',
  commit: '3f1c9a2e6b7d4f8a0c5e1b9d2a4f6c8e0b3d5f71'

cd /etc/puppetlabs/code/environments/production
r10k puppetfile check
r10k puppetfile install --verbose

Syntax OK
INFO     -> Deploying module to /etc/puppetlabs/code/environments/production/modules/stdlib
INFO     -> Deploying module to /etc/puppetlabs/code/environments/production/modules/concat
INFO     -> Deploying module to /etc/puppetlabs/code/environments/production/modules/apache
INFO     -> Deploying module to /etc/puppetlabs/code/environments/production/modules/acme_baseline

Read the comment in that Puppetfile again, because it catches almost everyone once. r10k installs exactly what you listed and nothing else. It does not read metadata.json and it does not resolve dependencies. puppet module install does both, which is why the habit transfers badly. Forget stdlib and the deploy succeeds, r10k reports nothing wrong, and every catalog that touches apache fails on the next agent run. There is a second trap in the same file. A forge 'https://...' line at the top of a Puppetfile is ignored unless forge.allow_puppetfile_override is set to true in r10k.yaml, so people add it, feel safer, and have changed nothing. Set the Forge URL once in r10k.yaml under forge.baseurl instead. The check that catches a missing dependency before your nodes do takes one command.

puppet module list --tree

Warning: Missing dependency 'puppetlabs-stdlib':
  'puppetlabs-apache' (v13.2.0) requires 'puppetlabs-stdlib' (>= 4.13.1 < 11.0.0)
  'puppetlabs-concat' (v10.0.1) requires 'puppetlabs-stdlib' (>= 9.0.0 < 11.0.0)
/etc/puppetlabs/code/environments/production/modules
└─┬ puppetlabs-apache (v13.2.0)
  ├── UNMET DEPENDENCY puppetlabs-stdlib (>= 4.13.1 < 11.0.0)
  └─┬ puppetlabs-concat (v10.0.1)
    └── UNMET DEPENDENCY puppetlabs-stdlib (>= 9.0.0 < 11.0.0)
/etc/puppetlabs/code/modules (no modules installed)
/opt/puppetlabs/puppet/modules (no modules installed)

One Puppetfile Line, Root on Every Node

Here is the part that surprises people. Before a machine can be told what to do, it has to describe itself, and some of those self-descriptions come from Ruby that modules ship under lib/. So the first thing every Puppet run does is download the plugin files from every module in the environment, write them into the agent's cache, and load them. Every module in the environment. Not every module the node uses. Classification, the process that decides which classes a node gets, has nothing to do with it. This is plugin sync, and it happens before a catalog exists.

puppet agent -t --environment canary --noop

Info: Using environment 'canary'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter/acme_hardening.rb]/ensure: defined content as '{sha256}5f0a1c8e3b7d9042ac6155e0b2d47f31c9a8e6b40d5f2731ac8e6b40d5f27319'
Info: Loading facts
Info: Caching catalog for web01.example.com
Info: Applying configuration version '1784613295'
Notice: /Stage[main]/Acme_baseline::Config/File[/etc/issue.net]/ensure: current_value 'absent', should be 'file' (noop)
Notice: Applied catalog in 1.87 seconds

Look at the order of those lines. The Ruby file landed in the agent's cache, and Facter loaded it on the very next line, before the catalog was ever requested. Add one line to a Puppetfile, deploy it, and every agent in that environment runs that module's Ruby as root on its next check-in, which on default settings is within thirty minutes. The Puppetfile is the gate. Node classification is not.

The blast radius runs in two directions and both are worth knowing precisely. Ruby under lib/facter, lib/puppet/type and lib/puppet/provider is loaded by the agent, as root, on the managed machine. Type definitions are loaded on the server as well, because the compiler validates and munges resource parameters while building the catalog, so that particular code runs on both sides of the wire. Ruby under lib/puppet/functions runs on the compiler, as the puppet user, on the machine that holds every node's data and usually the Certificate Authority (the service whose signature every agent in the estate trusts). exec resources declared in manifests run on the agent, as root, during apply. Nothing on that list is sandboxed, rate limited, or reviewed by anyone but you.

The Forge does not sign releases. What its API hands you is an MD5 and a SHA-256 of the release archive, which proves you downloaded the file the Forge described and proves nothing about who wrote it or what is inside. The rest is reputation: download counts, a validation score, whether the release was built with PDK, and a source repository you can go and read. Treat all of that as signal, never as guarantee. Before you pin something new, unpack the archive and spend a minute on three questions. What does grep -rn 'exec {' manifests/ turn up, and does every hit look sane. What sits under lib/, since that is the code that runs everywhere regardless of classification. Does anything download at runtime, which quietly converts your build-time pin into a promise somebody else can break later. Fail any of those and the honest answer is a reviewed fork you host yourself, not a shrug.

Checking That the Deploy Matches the Release

On a server nobody should be editing, a modified file is a finding. Puppet ships a command aimed straight at that question, and it comes with a catch big enough to plan around.

puppet module changes /etc/puppetlabs/code/environments/production/modules/apache

Error: No file containing checksums found.
Error: Try 'puppet help module changes' for usage

puppet module changes compares the files on disk against MD5 checksums recorded in the module's checksums.json, falling back to a checksums key inside metadata.json. A current release ships neither. The retired puppet module build wrote checksums.json; pdk build, which every publisher uses now, does not, so anything released in recent years arrives with nothing to compare against. Crack open puppetlabs-apache 13.2.0 and you will find no checksums.json and no checksums key. The command still works on old modules, so keep it in the drawer, but the check you can actually rely on today uses the hash the Forge publishes for the release archive.

cd /tmp
curl -sLO https://forgeapi.puppet.com/v3/files/puppetlabs-apache-13.2.0.tar.gz
sha256sum puppetlabs-apache-13.2.0.tar.gz
curl -s https://forgeapi.puppet.com/v3/releases/puppetlabs-apache-13.2.0 | jq -r .file_sha256
tar xzf puppetlabs-apache-13.2.0.tar.gz
diff -r puppetlabs-apache-13.2.0 /etc/puppetlabs/code/environments/production/modules/apache

a73937addcb6b3af144e3afea19f73f0e8a0a646c87b67d1c7adf1cac893357a  puppetlabs-apache-13.2.0.tar.gz
a73937addcb6b3af144e3afea19f73f0e8a0a646c87b67d1c7adf1cac893357a
diff -r puppetlabs-apache-13.2.0/manifests/init.pp /etc/puppetlabs/code/environments/production/modules/apache/manifests/init.pp
468c468
<   Boolean $service_enable                                                    = true,
---
>   Boolean $service_enable                                                    = false,

The two hashes match, so the archive is the one the Forge published. One file then differs from it, which means either a colleague hot-patched production and forgot to mention it, or something changed that file and it was not a colleague. Both are worth a conversation before lunch. Two honest limits. A module deployed from git has no published archive to compare against, so git status inside the deployed checkout is the equivalent check there. And all of this catches tampering after deployment, never a bad release upstream. The property that helps you there is immutability: a published Forge version is the same archive forever, so 13.2.0 cannot change underneath a pin. An author who wants different code on your servers has to publish 13.2.1 and talk you into moving.

Shipping Your Own

Your own modules go through the same pipeline, which is the point of using the standard layout even for code that will never leave the building. metadata.json is the card on the lid, and everything downstream reads it.

{
  "name": "acme-acme_baseline",
  "version": "1.2.0",
  "author": "acme",
  "license": "Apache-2.0",
  "summary": "CIS-aligned baseline hardening for Linux servers",
  "source": "https://git.example.com/infra/puppet-acme_baseline",
  "dependencies": [
    { "name": "puppetlabs/stdlib", "version_requirement": ">= 9.0.0 < 11.0.0" }
  ],
  "operatingsystem_support": [
    { "operatingsystem": "RedHat", "operatingsystemrelease": ["8", "9", "10"] },
    { "operatingsystem": "Ubuntu", "operatingsystemrelease": ["22.04", "24.04"] }
  ],
  "requirements": [
    { "name": "puppet", "version_requirement": ">= 8.0.0 < 9.0.0" }
  ],
  "pdk-version": "3.4.0",
  "template-url": "https://github.com/puppetlabs/pdk-templates.git#main"
}

pdk validate     # runs every validator: metadata, manifests, EPP, Ruby, tasks, YAML
pdk test unit    # rspec-puppet: compiles catalogs in memory, touches no real node

pdk (INFO): Running all available validators...
pdk (INFO): Using Ruby 3.2.5
pdk (INFO): Using Puppet 8.10.0
[✔] Checking metadata syntax (metadata.json tasks/*.json).
[✔] Checking module metadata style (metadata.json).
[✔] Checking Puppet manifest syntax (**/*.pp).
[✔] Checking Puppet plan syntax (plans/**/*.pp).
[✔] Checking Puppet manifest style (**/*.pp).
[✔] Checking Puppet EPP syntax (**/*.epp).
[✔] Checking Ruby code style (**/**.rb).
[✔] Checking task names (tasks/**/*).
[✔] Checking task metadata style (tasks/*.json).
[✔] Checking YAML syntax (**/*.yaml **/*.yml).
 
pdk (INFO): Using Ruby 3.2.5
pdk (INFO): Using Puppet 8.10.0
[✔] Preparing to run the unit tests.
[✔] Running unit tests.
  Evaluated 6 tests.
 
6 tests, 0 failures

pdk build

pdk (INFO): Building acme_baseline version 1.2.0
pdk (INFO): Build of acme-acme_baseline has completed successfully. Built package can be found here: /home/ops/modules/acme_baseline/pkg/acme-acme_baseline-1.2.0.tar.gz

Bump the version in metadata.json under semantic versioning before every release: patch for a fix that changes no interface, minor for a backward-compatible addition, major when you rename a parameter or change what a class does by default. The Forge refuses a version number it has already seen, so a forgotten bump surfaces as an upload error rather than a silent overwrite, which is a small mercy. Check one thing before your first publish. pdk build packs the directory through .pdkignore, falling back to .gitignore when there is no .pdkignore, and whatever survives that filter ends up in the archive and on every machine that installs it. A test fixture with a real password in it. A private key you dropped in while debugging. A stray .env. That filter is the only one you get. If you are running OpenVox, the community fork that appeared after Puppet 8's licence change, none of this changes: same module format, same layout, same Forge, same commands.

Before any of it reaches production, deploy the Puppetfile to a canary environment with r10k deploy environment canary --puppetfile, point one disposable node at it using puppet agent -t --environment canary, and read the entire run. What got written into the plugin cache. What the catalog changed. How long it took. Check the first line of the run says canary, because an External Node Classifier (the service that decides a node's environment and classes) can override the environment you asked for and hand you production by mistake. A module that surprises you on a canary costs you one machine you were going to rebuild anyway. The same surprise in production costs you the fleet, at whatever your run interval is.

Try this

Run pdk new module acme_baseline --skip-interview on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: r10k decides what lives in the modules directory, and it is not you. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related