Variables, facts & Facter

Node data and conditionals.

Walk into a kitchen you have never cooked in and the first thing you do is open cupboards. Gas or induction? How big is the biggest pot? Is there an oven at all? The dish survives. The method bends to whatever you find. Facter does that walk for you, on every machine, on every run. It inventories the node and hands the compiler a structured pile of data called facts, which are the things Puppet knows about a machine before it decides anything: operating system, memory, disks, network interfaces, whether the box is a virtual machine, which kernel it runs. Several hundred values on a typical Linux host. Your manifests read those facts and decide what to declare. httpd here, apache2 there, one worker process on the 2 GB test box. Classes group resources together. Facts and conditionals decide which resources a class actually declares, and that is how one piece of code covers a whole fleet without a single per-host special case. (If you run OpenVox, the community fork that appeared after Puppet's licence change, everything below is identical: same commands, same fact names.)

What Facter Actually Sees

Facter is a standalone command, so you can look at exactly what the compiler will be handed before you write a line of logic against it. Ask for one fact and you get that value on its own line. This matters more than it sounds. Most broken conditionals are not broken logic. They are a perfectly correct test written against a value the author never looked at.

facter os.family
facter os.release.major
facter networking.ip
facter processors.count
facter virtual

RedHat
9
10.20.4.31
8
kvm

Modern facts are structured, which means a fact can be a hash (a labelled drawer with smaller labelled boxes inside it) or an array, rather than only a flat string. os is a hash holding family, name, architecture and a nested release hash. On the command line you drill in with dots. In the Puppet language you use square brackets. Ask for the parent and Facter prints the whole tree, with => between each key and its value. That tree is the map you are indexing into when you write $facts['os']['release']['major'], so look at it once per fact instead of guessing the path.

facter os

{
  architecture => "x86_64",
  family => "RedHat",
  hardware => "x86_64",
  name => "AlmaLinux",
  release => {
    full => "9.4",
    major => "9",
    minor => "4"
  },
  selinux => {
    config_mode => "enforcing",
    config_policy => "targeted",
    current_mode => "enforcing",
    enabled => true,
    enforced => true,
    policy_version => "33"
  }
}

Older manifests read facts as bare top-scope variables: $::osfamily, $::ipaddress, $::operatingsystemmajrelease. Those are the legacy facts, and in Puppet 8 they are a loaded gun. Facter itself still knows every one of them. It keeps them out of normal output, and --show-legacy brings them back.

# Facter still resolves the flat names, it only hides them by default
facter --show-legacy | grep -E '^(osfamily|operatingsystem|operatingsystemmajrelease|ipaddress) '

ipaddress => 10.20.4.31
operatingsystem => AlmaLinux
operatingsystemmajrelease => 9
osfamily => RedHat

Puppet is the half that changed. Puppet 7 passed legacy facts to the compiler along with everything else. Puppet 8 ships with include_legacy_facts set to false, so the fact set the compiler receives holds structured facts only. Nothing errors. $::osfamily comes back undef, your case drops through to default, and the run reports success. Watch it happen on a node where Facter is perfectly willing to tell you the answer.

puppet apply -e '
$legacy     = $::osfamily
$structured = $facts["os"]["family"]
notice("legacy [${legacy}] structured [${structured}]")
'

Notice: Scope(Class[main]): legacy [] structured [RedHat]
Notice: Compiled catalog for web01.example.com in environment production in 0.02 seconds
Notice: Applied catalog in 0.01 seconds

You can set include_legacy_facts = true in puppet.conf to buy time during a migration, and plenty of shops did exactly that on upgrade day. Fix the code instead, because the default moving against you is only the newest reason. A bare $osfamily with no colons resolves in local scope first, so a class parameter of the same name wins silently and your manifest reads something that was never a fact at all. And the flat names throw away structure: $::operatingsystemmajrelease hands you 9 and nothing else, while $facts['os']['release'] hands you full, major and minor together. The same swap applies inside hiera.yaml, where %{::osfamily} has to become %{facts.os.family} or your data hierarchy quietly stops matching anything.

Branching Without Per-Host Special Cases

A Puppet variable starts with $ and is assigned exactly once per scope. Write $pkg = 'httpd' and then $pkg = 'apache2' further down the same file and the compile stops with Cannot reassign variable. That feels wrong if you arrive from Bash or Python. It is deliberate. A manifest describes an end state, and a name that meant two different things at two points in the file would make that description ambiguous.

case is the workhorse for operating system forks. The selector, written ? { }, is the compact form for mapping one value straight onto another. if/elsif/else and unless cover boolean tests. Two things to know before you write one. String comparison in Puppet ignores case, so 'RedHat' matches 'redhat' and you do not need to normalise fact values first. And whichever form you reach for, give it a default arm with fail() in it. An unsupported platform should stop the compile with a message a human can act on, not slide through and apply a half-built catalog that leaves the machine in a state nobody designed or tested.

# Roles and profiles: a profile wires up one technology on one node.
class profile::webserver (
  Integer[1] $max_workers = $facts['processors']['count'],
) {
 
  case $facts['os']['family'] {
    'RedHat': {
      $pkg  = 'httpd'
      $svc  = 'httpd'
      $conf = '/etc/httpd/conf.d/workers.conf'
    }
    'Debian': {
      $pkg  = 'apache2'
      $svc  = 'apache2'
      $conf = '/etc/apache2/conf-available/workers.conf'
    }
    default: {
      fail("profile::webserver has no support for ${facts['os']['name']} ${facts['os']['release']['full']}")
    }
  }
 
  # Selector: one value mapped onto another. Regex cases are allowed.
  $tuning = $facts['virtual'] ? {
    'physical'           => 'baremetal',
    /^(kvm|vmware|xen)$/ => 'virtual',
    default              => 'conservative',
  }
 
  # Small boxes get one worker no matter how many cores the hypervisor claims.
  $workers = $facts['memory']['system']['total_bytes'] < 2147483648 ? {
    true    => 1,
    default => $max_workers,
  }
 
  package { $pkg:
    ensure => installed,
  }
 
  file { $conf:
    ensure  => file,
    owner   => 'root',
    group   => 'root',
    mode    => '0644',
    content => "# managed by Puppet (${tuning})\nServerLimit ${workers}\n",
    notify  => Service[$svc],
  }
 
  service { $svc:
    ensure => running,
    enable => true,
  }
}

Now the step people skip. --noop (short for no-operation) compiles the catalog and reports what it would change without touching anything, which lets you read back which branch the facts actually chose. Seeing Package[httpd] and /etc/httpd/conf.d/workers.conf in the resource paths is proof the RedHat arm won on this node. If you expected the Debian arm, the bug is in the fact you branched on rather than in the case, and you found that out before a single file changed.

puppet apply --noop --show_diff -e 'include profile::webserver' \
  --modulepath /etc/puppetlabs/code/environments/production/site-modules

Notice: Compiled catalog for web01.example.com in environment production in 0.31 seconds
Notice: /Stage[main]/Profile::Webserver/Package[httpd]/ensure: current_value 'absent', should be 'present' (noop)
Notice: /Stage[main]/Profile::Webserver/File[/etc/httpd/conf.d/workers.conf]/ensure: current_value 'absent', should be 'file' (noop)
Notice: /Stage[main]/Profile::Webserver/Service[httpd]/ensure: current_value 'stopped', should be 'running' (noop)
Notice: Class[Profile::Webserver]: Would have triggered 'refresh' from 3 events
Notice: Stage[main]: Would have triggered 'refresh' from 1 event
Notice: Applied catalog in 0.42 seconds

A Release Major Of 9 Is A String, Not A Number

Facts arrive typed, and the types are rarely the ones you assume. $facts['os']['release']['major'] is the string '9', because release identifiers are labels rather than arithmetic. Ubuntu's major is '22.04', and plenty of platforms put letters in theirs. Compare that string against the bare number 9 with == and Puppet answers false every time, because a String is never equal to an Integer. No error, no warning, the run reports success, your conditional took the other arm. Reach for < on the same pair and you get the opposite treatment: Comparison of: String < Integer, is not possible stops the compile dead. The loud one is the friendly one. Memory sizes and processor counts really are integers, and the JSON view gives you the first hint, because Facter quotes strings and leaves numbers bare.

facter --json os | jq '.os.release'
facter --json processors | jq '.processors.count'

{
  "full": "9.4",
  "major": "9",
  "minor": "4"
}
8

Better than reading quote marks off a screen, ask Puppet what it thinks. type() reports the inferred type of a value, and Puppet infers tightly: String[1, 1] means a string of exactly one character, Integer[8, 8] an integer whose possible range runs from eight to eight. Pass 'generalized' as a second argument when those ranges are noise and you want plain String or Integer back. One run of this per platform, before you write the comparison, retires the whole family of silent wrong-branch bugs.

puppet apply -e '
$major  = $facts["os"]["release"]["major"]
$cores  = $facts["processors"]["count"]
$memory = $facts["memory"]["system"]["total_bytes"]
notice("major  is ${type($major)}")
notice("cores  is ${type($cores)}")
notice("memory is ${type($memory)}")
'

Notice: Scope(Class[main]): major  is String[1, 1]
Notice: Scope(Class[main]): cores  is Integer[8, 8]
Notice: Scope(Class[main]): memory is Integer[16637620224, 16637620224]
Notice: Compiled catalog for web01.example.com in environment production in 0.03 seconds
Notice: Applied catalog in 0.01 seconds

The other trap is absence. Ask for a fact this node does not report and you get undef, which is harmless on its own. Index one level deeper into that undef and the compile dies with an error pointing at the bracket rather than at the missing fact, which is a confusing thing to read at two in the morning. getvar() is the fix. It walks a dotted path and returns whatever default you name the moment any step along the way is missing, so an optional fact stops being a landmine.

# Indexing into a fact this node does not report
puppet apply -e 'notice($facts["nope"]["deeper"])'
 
# The safe form: a dotted path plus a default you chose on purpose
puppet apply -e 'notice(getvar("facts.nope.deeper", "not reported"))'

Error: Evaluation Error: Operator '[]' is not applicable to an Undef Value. (line: 1, column: 22) on node web01.example.com
 
Notice: Scope(Class[main]): not reported
Notice: Compiled catalog for web01.example.com in environment production in 0.02 seconds
Notice: Applied catalog in 0.01 seconds

Facts Facter Does Not Ship With

Core Facter knows what it can find by looking at the machine. It cannot know which datacentre a rack sits in, what your change system calls this host, or which build of your own application is unpacked under /opt. You add those yourself. External facts are the low-effort path, and they are what they sound like: a sticky note left on the fridge for whoever opens it next. Drop a static file into a directory Facter reads and the values in it become facts. The formats are YAML, JSON (both plain-text ways of writing nested data) and a .txt of name=value lines. Running as root on Linux, Facter reads /opt/puppetlabs/facter/facts.d/, /etc/puppetlabs/facter/facts.d/ and /etc/facter/facts.d/. No Ruby, no module, no sync step.

---
datacenter: us-east-1
rack: b14
maintenance_window: sun-0400

Custom facts are Ruby. They live in a module under lib/facter/ (pdk new module scaffolds that layout for you) and reach agents automatically through pluginsync, the step at the start of every agent run that copies module plugin code down to the node. Use them when the value needs real logic: read a file, parse command output, fall back cleanly when the tool is missing. Two things matter in practice. confine restricts the fact to nodes where it could possibly resolve, so you are not shelling out to dnf on a Debian box every thirty minutes. And Facter::Core::Execution.execute with on_fail stops a missing binary or a non-zero exit from taking the whole fact collection down with it.

# Counts pending security errata (the vendor's published security fixes).
# Reaches agents through pluginsync.
Facter.add(:security_updates) do
  # Facter evaluates this itself, so the fact never runs on Debian or Windows.
  confine 'os.family' => 'RedHat'
 
  setcode do
    out = Facter::Core::Execution.execute(
      '/usr/bin/dnf --quiet updateinfo list --security',
      on_fail: '',             # missing dnf, or a non-zero exit => empty string
    )
    out.lines.count { |line| line.match?(/RHSA|ALSA|ELSA/) }
  end
end

Test both kinds without a full agent run. --custom-dir and --external-dir point Facter straight at directories, so you can check a fact from a git checkout on your laptop, or from the control repo on the server, before anything ships to a node.

facter --custom-dir /etc/puppetlabs/code/environments/production/site-modules/profile/lib/facter \
       --external-dir /opt/puppetlabs/facter/facts.d \
       security_updates datacenter

datacenter => us-east-1
security_updates => 3

On a real node the two kinds arrive through different doors. External facts you drop on the box are read locally by Facter on every run. Custom facts, and any external facts shipped inside a module's own facts.d/, have to land in the agent's cache first, and until they do the fact is silently absent and your conditional quietly takes the default arm. --tags none is the cheapest way to push them out: none is not a magic word, it is a tag that no resource carries, so the agent syncs plugins and then applies nothing at all. puppet facts show gives you the Puppet-side view afterwards, custom and external facts included, and it is what replaced the old facter -p.

# Sync module plugins (custom facts) and apply no resources
puppet agent -t --tags none
 
# Prove the fact landed on the node, then read it the way Puppet will
ls /opt/puppetlabs/puppet/cache/lib/facter/
puppet facts show security_updates

Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter/security_updates.rb]/ensure: defined content as '{sha256}0e6f4b1c9a...'
Info: Loading facts
Info: Caching catalog for web01.example.com
Info: Applying configuration version '1753113600'
Notice: Applied catalog in 0.09 seconds
 
security_updates.rb
 
{
  "security_updates": 3
}

One custom fact that shells out is fine. Twenty of them are a tax you pay on every node, on every run, forever. Facter can cache by group: name a set of expensive facts, then give that group a TTL (time to live, meaning how long a stored answer stays usable before Facter goes and looks again). The config file is HOCON, which is JSON with the punctuation relaxed.

# Group your expensive facts, then give the group a lifetime.
fact-groups : {
    site-expensive : [ "security_updates" ]
}
 
facts : {
    # Never resolve these at all on this fleet
    blocklist : [ "EC2", "file system" ],
 
    # Resolve at most once an hour; serve from disk in between
    ttls : [
        { "site-expensive" : 1 hour }
    ]
}

Facts Are Self-Reported. $trusted Is Not

Everything above assumes the node tells the truth. Nothing makes it. Facts are gathered by the agent, on the node, and shipped to the compiler, and the Puppet Server cannot check a single one of them. They are a visitor badge the visitor filled in. Anyone who can write to a facts.d directory can make that machine claim to be anything, and the precedence order runs in the attacker's favour: external facts outrank custom facts, which outrank built-in ones. That ordering is a feature when you override something on purpose. It is an escalation path when the directory permissions are loose. Here is the whole attack.

# On the node, as anyone who can write to the external fact directory:
printf 'role: database\n' > /opt/puppetlabs/facter/facts.d/pwn.yaml
 
# The next run reads it. This is what the compiler is handed
# (run as root here, the way the agent runs):
puppet apply -e '
$role = $facts["role"]
$cert = $trusted["certname"]
$auth = $trusted["authenticated"]
notice("fact role: ${role}")
notice("certname:  ${cert}")
notice("authed as: ${auth}")
'

Notice: Scope(Class[main]): fact role: database
Notice: Scope(Class[main]): certname:  web01.example.com
Notice: Scope(Class[main]): authed as: local
Notice: Compiled catalog for web01.example.com in environment production in 0.03 seconds
Notice: Applied catalog in 0.01 seconds

The role changed. The certificate name did not, and could not. $trusted is assembled by the Puppet Server from the node's signed certificate rather than from anything the agent says about itself, which makes it the ID card instead of the visitor badge. $trusted['certname'] is the subject of that certificate, and a node cannot alter it without getting a new one signed by your certificate authority (the service that issues and signs those certificates, normally the Puppet Server itself). $trusted['extensions'] holds custom certificate extensions baked in at signing time, and that is where a role or a security tier belongs if you plan to gate access on it. Look at $trusted['authenticated'] in that output. It reads local, because puppet apply verified nothing. In an agent run against a real server it reads remote, and only then is the guarantee worth anything.

Certificate extensions are fixed at signing, so they suit facts that never change. For data that moves (owner, cost centre, environment, ticket state) Puppet 8 gives you trusted_external_command. Point the setting at a script, or at a directory of scripts, on the Puppet Server. On each compile the server runs them with the node's certname as the only argument and merges the JSON they print into $trusted['external'], keyed by script filename. The lookup key comes from the signed certificate and the code runs on the server, so the node gets no say in the answer. This is the correct home for a role you branch on, and the natural bridge to your CMDB (configuration management database, the system of record that lists what each host is for).

[server]
# Point at a directory and every executable inside runs, keyed by its filename,
# giving you $trusted['external']['cmdb'] from the script below.
trusted_external_command = /etc/puppetlabs/puppet/trusted_external

#!/bin/bash
# Puppet Server passes exactly one argument: the certname taken from the node's
# signed certificate. Print a JSON object on stdout. A non-zero exit fails the
# compile, so every failure path here has to end in valid JSON.
#
# This runs as the puppet user on the server (not root), once per compile,
# so keep it fast and keep the token readable by that user alone.
set -u
certname="$1"
token_file=/etc/puppetlabs/puppet/cmdb.token   # mode 0400, owned by puppet
 
token=""
[ -r "$token_file" ] && token="$(cat "$token_file")"
 
curl -sf --max-time 5 \
  -H "Authorization: Bearer ${token}" \
  "https://cmdb.internal/v1/nodes/${certname}" \
  || echo '{"role":"unknown","tier":"untrusted"}'

class profile::hardening {
 
  # Cosmetic branching. A self-reported fact is fine here: the worst a liar
  # gets is the wrong package name on their own box.
  $audit_pkg = $facts['os']['family'] ? {
    'RedHat' => 'audit',
    'Debian' => 'auditd',
    default  => fail("no audit package known for ${facts['os']['family']}"),
  }
 
  package { $audit_pkg: ensure => installed }
 
  # An access decision. Never a fact. Resolved server-side from the certname.
  $tier = getvar('trusted.external.cmdb.tier', 'untrusted')
 
  if $tier == 'pci' {
    include profile::hardening::pci   # pulls cardholder-zone data from Hiera
  }
}

Asking The Whole Fleet

Facts steer one node's catalog, and they also pile up. Every agent run uploads its fact set, and PuppetDB (the database the Puppet Server writes facts, catalogs and reports into) keeps the latest set for every node it has seen. That turns any fact, including the ones you wrote yourself, into a question you can answer across thousands of machines in about a second. PQL (Puppet Query Language) against the inventory endpoint is the shortest route. The puppet query command does not come with the agent package: install the puppetdb_cli gem into the agent's Ruby with puppet resource package puppetdb_cli provider=puppet_gem ensure=present, then point it at your server in /etc/puppetlabs/client-tools/puppetdb.conf. What you get back is the difference between believing you patched those hosts and holding the list.

puppet query 'inventory[certname, facts.security_updates] {
                facts.os.family = "RedHat" and facts.security_updates > 0
              }'

[
  {
    "certname": "web01.example.com",
    "facts.security_updates": 3
  },
  {
    "certname": "db03.example.com",
    "facts.security_updates": 11
  }
]

The same query shape verifies that a change landed. Roll the new profile to one canary, wait for its next run, then ask PuppetDB which nodes report the value you expect and how many do not yet. It also catches the lie from the previous section. A node whose role fact says database while its certname sits nowhere near the database pool is either misconfigured or owned, and finding it is one query rather than a spreadsheet.

puppet query 'inventory[certname] { facts.role = "database" and !(certname ~ "^db") }'

[
  {
    "certname": "web01.example.com"
  }
]

Put that query on a schedule. Facts are cheap to collect and cheap to ask about, so the cost of noticing that a host has started claiming a role its certificate never granted is a cron entry and a webhook. The cost of not noticing is that the next compile hands that host whatever the database profile pulls out of Hiera.

Try this

Run facter os.family on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: every custom fact runs on every node, every run. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related