Process debugging: ptrace, gdb and core dumps

Attach, inspect and capture a crash.

When a process hangs or crashes and its logs say nothing useful, you need to read the process itself: what each thread is waiting for, which lock it holds, which pointer was bad. This lesson does that with two small programs you compile yourself, one that deadlocks and one that crashes. You will attach a debugger to a running service within Ubuntu's ptrace restrictions, detach without harming it, capture and open a core dump through apport on Ubuntu and systemd-coredump on RHEL, and see where ltrace helps and where it misleads. The system-call view (strace) was the subject of the strace lesson; this lesson goes inside the process. The tools are not all in a default install: sudo apt install gcc gdb ltrace on Ubuntu Server, sudo dnf install gcc gdb ltrace on RHEL.

ptrace, and who may use it

Debuggers are built on ptrace(2), the system call that lets one process (the tracer) stop another (the tracee), read and write its memory and registers, and resume it. gdb, strace and ltrace all use it. Two consequences follow. The target is stopped while a tracer works on it, so attaching to a busy service pauses it for as long as you take. And ptrace can read anything in the target's memory, including keys and passwords, so the kernel restricts who may use it.

The first check is ownership: an unprivileged tracer must run as the same user as the target. The second is the Yama setting kernel.yama.ptrace_scope from the lesson "System calls and strace": Ubuntu's 1 allows an unprivileged tracer only its own descendants, RHEL's 0 any process of the same user, and CAP_SYS_PTRACE (a sudo) passes either; a hardened host may set 2 (root only) or 3 (no attaching until reboot). Here is what that means for a debugger.

A hung service: attach, read, detach

The first program starts a second thread. Each thread takes one mutex, sleeps a second so the timing never varies, then asks for the mutex the other one holds, a lock-order inversion, the classic deadlock. It is compiled with -g (debug information, so gdb can name variables and source lines) and -O0 (no optimisation, so the code follows the source). Create a directory for the lesson's programs first (mkdir -p ~/dt).

/* dt-worker.c: two threads take the same two locks in opposite order */
#define _GNU_SOURCE
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
 
static pthread_mutex_t stats_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_mutex_t queue_lock = PTHREAD_MUTEX_INITIALIZER;
 
static void flush_queue(void)
{
    pthread_mutex_lock(&queue_lock);
    sleep(1);
    pthread_mutex_lock(&stats_lock);
    pthread_mutex_unlock(&stats_lock);
    pthread_mutex_unlock(&queue_lock);
}
 
static void *flusher(void *arg)
{
    (void)arg;
    for (;;)
        flush_queue();
    return NULL;
}
 
static void collect_stats(void)
{
    pthread_mutex_lock(&stats_lock);
    sleep(1);
    pthread_mutex_lock(&queue_lock);
    pthread_mutex_unlock(&queue_lock);
    pthread_mutex_unlock(&stats_lock);
}
 
int main(void)
{
    pthread_t t;
 
    pthread_create(&t, NULL, flusher, NULL);
    pthread_setname_np(t, "flusher");
    printf("dt-worker %d running\n", getpid());
    fflush(stdout);
    for (;;)
        collect_stats();
}

Build it, then run the worker the way a real daemon runs: as a service under your own account, started by systemd. LimitCORE=infinity (no limit on the size of a core dump) will matter later.

A few seconds later the service is hung. Before reaching for a debugger, look from outside.

ps -L lists threads. Both are asleep (S), use almost no CPU, and wait in the kernel's futex code (futex_do_wait): a futex is the kernel primitive under user-space locks, so both threads are blocked on locks. That rules out a busy loop and points at lock waits, but not at which lock. Now attach, as the account that owns the service.

The same user was refused: the service was started by systemd, so your gdb is not its ancestor, and scope 1 forbids the attach. (gdb's hint names /etc/sysctl.d/10-ptrace.conf, an old location; Ubuntu 26.04 ships the setting in /usr/lib/sysctl.d/55-ptrace.conf.) Do not edit either file: a one-off sudo gets past the check for this attach only.

info threads shows each thread's current frame. The main thread (LWP 257216, LWP being the kernel's name for a thread) waits in futex_wait on queue_lock; flusher waits on stats_lock. gdb resolved the futex addresses to the symbols of the two mutexes because the program has debug information. The last line matters as much: gdb detached, and the process carries on exactly as before.

Normally you would continue with thread apply all bt for full stacks. On this lab machine that fails, and the reason is worth knowing because it is not Ubuntu's doing.

This failure is an artefact of the lab's virtual machine: its virtual CPU (on an Apple M4 host) offers the SME matrix extension without SVE, and gdb 17.1 here (and 16.3 on the Rocky lab) cannot read a live thread's registers on such a CPU, so it stops after frame 0. On x86_64 servers and on ARM servers with SVE the command prints full stacks. When live unwinding fails, or when you must not hold a production process stopped while you think, take a core dump and read it at leisure. gcore PID writes one from a running process through ptrace, and the process stays frozen for as long as writing its whole memory takes, seconds to minutes for a large service; on this lab CPU it fails the same way. A service that is deadlocked will need a restart anyway, so here the core comes from killing it with SIGABRT, whose default action is to dump core.

Read from the core, the stacks are complete. flusher blocked in flush_queue at line 14 asking for stats_lock; the main thread blocked in collect_stats at line 31 asking for queue_lock. The mutex's __owner field (glibc's record of the owning thread) closes the case: stats_lock is held by 257216, the main thread, and queue_lock by 257225, flusher. Each holds what the other wants. ([PAC] marks return addresses signed by ARM pointer authentication; x86_64 shows none.) systemd records the ending as Result=core-dump with signal 6.

Crashes: core dumps on Ubuntu and RHEL

A core dump is a file with the memory and registers of a process at the moment it died from a signal such as SIGSEGV. The kernel decides where it goes from kernel.core_pattern: a file name, or a | followed by a program that receives the core on its standard input. The second program has a bug: a header without a colon leaves a pointer NULL. Save it in ~/dt and build it the same way.

/* dt-parse.c: prints the length of the value in a "Name: value" header */
#include <stdio.h>
#include <string.h>
 
struct header {
    const char *name;
    const char *value;
};
 
static size_t value_len(const struct header *h)
{
    return strlen(h->value);
}
 
int main(int argc, char **argv)
{
    struct header h = { argv[1], NULL };
    char *colon = strchr(argv[1], ':');
 
    /* bug: a line without ':' leaves value NULL */
    if (colon)
        h.value = colon + 1;
    printf("%zu\n", value_len(&h));
    return argc > 2;
}

On Ubuntu the pipe goes to apport, the crash reporter. The shell said (core dumped), yet nothing was kept. The kernel ignores RLIMIT_CORE for a pipe and passes it to the handler as %c, and apport follows its own rules: for a program that does not belong to a package it writes no crash report (the log says so), and it writes a core file only if the core limit allows one. Ubuntu's default soft limit is 0, both for logins and for services. Raise it for your shell and crash again:

The core lands in /var/lib/apport/coredump, readable only by you (mode -r--------), named after the program, your UID, the boot ID and the PID. For programs from a package, apport instead writes a report to /var/crash/*.crash that contains the core; apport-unpack extracts it. Open the core with the program that produced it:

gdb prints the frame where the program died as it loads the core, then bt prints the whole stack. The crash happened in the C library's strlen, called from value_len at line 12, and print *h shows value = 0x0: the NULL pointer. (gdb asks first whether to download debugging symbols from debuginfod.ubuntu.com and, reading no terminal, answers no; the line is left out.) Delete cores once you have what you need.

RHEL sends every core to systemd-coredump, which stores it compressed in /var/lib/systemd/coredump, logs the crash with a backtrace in the journal, and lets you list and open cores with coredumpctl. Like apport, it keeps a core only when the process's core limit allows one, but on RHEL both SSH logins and services get an unlimited limit.

Older entries from earlier runs are left out of the list; their files have since been removed. coredumpctl debug prints the journal record, including a stack trace systemd-coredump made at crash time, then starts gdb on the stored core (here in batch mode with bt). It shows the same NULL pointer path, without the C library's line numbers because its debug symbols are not installed. The file belongs to root, and the + after its mode marks an access control list, which is how coredumpctl could read it for you.

The last command shows the size limits from coredump.conf(5): upstream systemd allows 32G on 64-bit systems, and RHEL lowers both to 1G, so a larger core gets no stack trace (ProcessSizeMax=) and is not kept on disk (ExternalSizeMax=). Since every RHEL service may dump, check these limits, and MaxUse= for all cores together, before a large service crashes. Ubuntu can use systemd-coredump too: systemd-coredump and apport's apport-core-dump-handler both provide core-dump-handler and conflict, so installing the first replaces apport's pipe, and apport then builds its reports from systemd-coredump's records. On RHEL, ptrace_scope 0 also means gdb may attach to your own process without sudo, even one started from another shell:

The attach succeeded and listed both threads; the frame column is empty for the same CPU reason as on the Ubuntu lab.

Library calls: ltrace and its limits

ltrace shows the calls a program makes into shared libraries, which sit closer to the program's intent than system calls do. It works by planting breakpoints in the program's PLT, the table of stubs through which calls to shared libraries pass.

For the lab's own program, the output is right: strchr found the colon, strlen measured six characters, printf wrote two bytes. Now a program from the distribution:

hostname printed web01 and exited with status 0, yet ltrace reports errx(1, ...), which would have ended it with status 1, and setdomainname, which it never calls. ltrace matched the PLT entries of this distribution binary to the wrong names on aarch64. Upstream ltrace went twelve years without a release after 0.7.3 (2013), until 0.8.0 and 0.8.1 in September 2025; both lab systems still ship patched 0.7.91 code (a 2023 git snapshot on Ubuntu 26.04, 0.7.91 on RHEL 10), and that is what produced this output. Treat its output as a hint to confirm, not as evidence. For library calls you can trust, use uprobes: bpftrace can count or time any function in libc.so.6 without stopping the process (see the eBPF tools lesson), and perf probe -x does the same with perf.

Try this

Fix the deadlock and prove the fix. Copy dt-worker.c to dt-worker-fixed.c, change flush_queue so it takes stats_lock first and queue_lock second (the same order as collect_stats), and build it. Then run both versions under strace for ten seconds, counting sleeps: strace -f -c -e trace=clock_nanosleep timeout 10 ~/dt/dt-worker and the same for dt-worker-fixed. strace is allowed without sudo because it starts the program, so it is an ancestor. Predict before you run: the deadlocked version sleeps twice (once per thread) and never again; the fixed one keeps working and sleeps about once a second (9 calls in the lab run's ten seconds; when timeout happens to interrupt a sleep, that call is counted as an error).

Takeaway

Look from outside first (ps -L, wchan), attach with a one-off sudo rather than weakening ptrace_scope, and detach promptly; when you cannot hold a process stopped, work from a core, which means knowing where your platform sends cores and what core limit your services run with.

Related