Build & test stages

Compile, test, and fail fast.

A kitchen runs two checks before a plate leaves the pass: is the dish actually cooked, and does it taste right. Your pipeline runs the same two checks on code. A teammate pushes a change to the checkout service, and before any human reads a line of it, two questions have to be answered by a machine: does the code still compile, and do the tests still pass? Those two questions are the build stage and the test stage, the beating heart of every continuous integration (CI) pipeline. CI means every push gets built and tested automatically, with nobody having to remember. The build stage turns source code into something runnable: a compiled binary, a JAR, a package. The test stage runs the automated test suite against it. Get these two stages right and everything downstream, packaging, scanning, deploying, is standing on solid ground.

pipeline {
  agent { label 'linux' }                 // run on a worker agent, never the controller
  options { timestamps() }                 // prefix every log line with a timestamp
  stages {
    stage('Build') {
      // fail fast: a compile error surfaces in seconds, before the slow suite
      steps { sh 'mvn -B -DskipTests package' }
    }
    stage('Test') {
      steps { sh 'mvn -B verify' }          // runs the unit + integration suite
    }
  }
  post {
    always {
      // publish results whether the build passed or failed
      junit '**/target/surefire-reports/*.xml'
    }
  }
}

+ mvn -B verify
[INFO] Scanning for projects...
[INFO] --- surefire:3.2.5:test (default-test) @ checkout ---
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.acme.checkout.CartServiceTest
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
[INFO] Running com.acme.checkout.PricingTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] Results:
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS
[Pipeline] junit
Recording test results
Finished: SUCCESS

One rule decides pass or fail: the exit code

Every command you run hands back a small number when it finishes, the way a delivery driver hands back a signed slip. Zero means it went fine. Anything else means something went wrong. That number is the exit code, and it is the whole contract a sh step lives by. A sh step is the workhorse that runs a shell command on the agent, and it succeeds on exit code 0 and fails on any other code. That is the entire rule. So you never write logic to detect a failed test. You let the test runner do what it already does. mvn -B verify exits non-zero the moment a test fails. (The -B means batch mode, which keeps the log clean and non-interactive; verify is the Maven goal that runs the lifecycle all the way up through compile, test and package.) When a step fails, its stage fails, the pipeline stops, and the build is marked FAILURE. Putting the cheap Build stage ahead of the slow Test stage is the fail-fast principle in one line: a compile error turns the build red in seconds, instead of after a ten-minute suite that was doomed before it started.

Publish the results, do not leave them buried in the log

Scrolling a console log to find the one line that names the broken test is miserable work. The junit step ends it. It reads JUnit XML, a plain-text file format that test runners write to describe what passed and what failed, and turns it into a proper Test Result page: pass and fail counts, a drill-down into any single test, and a trend graph plotting passes and failures across recent builds so a flaky test stands out at a glance. Maven writes that XML for free. The Surefire plugin, which is Maven's unit-test runner, drops one file per test class into target/surefire-reports/. The glob **/target/surefire-reports/*.xml catches all of them, in every module of a multi-module project. Keep junit inside a post block so results get recorded whether the build passed or failed. A failed build is exactly when you most want to see which test broke.

Under the hood, junit parses each XML file, stores what it finds against the build record, and stitches those numbers into the trend. When a first run logs None of the test reports contained any result, the glob almost always missed the files. Maybe the tests never ran because something failed to compile upstream. Maybe the reports landed somewhere else. Maybe the pattern has a typo in it. Check the pattern against the workspace the agent actually used. And note the guardrail: junit fails its own step when it matches zero files, unless you pass allowEmptyResults: true. That is deliberate, so a quietly broken test stage cannot pass itself off as green.

UNSTABLE versus FAILURE, the distinction that trips everyone up

Jenkins shows three everyday build results as coloured balls: SUCCESS (blue or green), FAILURE (red) and UNSTABLE (yellow). The gap between the last two is the most misunderstood thing in CI, and it comes back to that exit code. FAILURE means a step itself errored, because the command exited non-zero: the code would not compile, or Surefire stopped because a test failed. The pipeline halts there. UNSTABLE means every step exited 0 and nothing errored, but a reporting step looked at the results and judged them imperfect. When the XML it reads contains failing tests, the junit step downgrades the build to UNSTABLE rather than failing it. It never sets FAILURE itself. So the same broken test can come out either colour, and the only thing deciding which is whether the test command aborted. By default mvn -B verify aborts on a failing test, which gives you FAILURE. Run mvn -B verify -Dmaven.test.failure.ignore=true and Maven records the failures but exits 0, so the pipeline carries on and junit marks the build UNSTABLE.

+ mvn -B verify
[INFO] Running com.acme.checkout.PricingTest
[ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0
[ERROR]   PricingTest.appliesBulkDiscount:47 expected:<90.00> but was:<100.00>
[INFO]
[INFO] Results:
[ERROR] Tests run: 20, Failures: 1, Errors: 0, Skipped: 0
[INFO] BUILD FAILURE
[Pipeline] }                    // Test stage aborts here (mvn exited 1)
[Pipeline] junit                // still runs — it lives in post { always }
Recording test results
Finished: FAILURE

    stage('Test') {
      // record failures but keep going, so junit grades the result:
      steps { sh 'mvn -B verify -Dmaven.test.failure.ignore=true' }
    }
    // (junit still runs in post { always } and downgrades to UNSTABLE)

[INFO] Tests run: 20, Failures: 1, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS               // mvn exited 0 despite the failing test
[Pipeline] junit
Recording test results             // junit reads the XML, finds 1 failing test
Finished: UNSTABLE                 // junit downgraded the build — mvn never failed

So which do you want? For unit tests, take the default and let it be FAILURE. A broken unit test means the code is wrong, and a red build that blocks the merge is the signal you want. Fail fast, fix it, move on. UNSTABLE earns its keep for softer signals: a flaky end-to-end suite, a coverage threshold, or lint warnings you want visible on the build page without blocking every merge while the team gets them under control. The anti-pattern is reaching for -Dmaven.test.failure.ignore=true only to push a greenish build past a deadline. Nobody watches the yellow ball, and broken tests pile up quietly for weeks. Whichever you pick, keep junit in post { always } so you get the report either way, red or yellow.

Where the build actually runs, and why security cares

Every sh step here executes on an agent: a worker machine, often a container that gets thrown away when the build ends, that the controller hands work to. The controller is the brain of the setup. It schedules builds, stores the configuration and the credential store, and serves the web UI, and it should run zero builds of its own. Treat that split as a security boundary rather than a scaling convenience. mvn -B verify downloads dependencies and runs test code, and when the change arrived as a pull request, that is effectively arbitrary code nobody on your team wrote, executing on the agent. Keeping builds off the controller means a hostile test cannot reach the controller's secrets or its config. If a test genuinely needs a secret, hand it over with withCredentials so Jenkins masks the value in the log, and never print a secret from inside a test, because junit publishes test output and a leaked token would sit in the report for anyone who can open the build. The junit and Surefire plugins are ones you actually use, so keep them installed and current. Plugins nobody uses are attack surface wearing a feature's icon.

Right now this pipeline throws away everything it produced: the packaged JAR, the coverage report, the test XML. Attach those to the build instead and you can pull back exactly what build 412 made, weeks later, without rebuilding a thing. The post block can carry more weight too, notifying the right people and cleaning up the workspace on every outcome, green, yellow or red. That is the next lesson, Artifacts & post actions.

Try this

Work through “Where the build actually runs, and why security cares” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: a test suite that can never fail the build is set dressing. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related