Stack Overflow Detection and Monitoring in RTOS

Stack overflow remains one of the most common and insidious causes of failure in embedded systems running an RTOS. Unlike desktop applications where the operating system expands the stack dynamically or raises a clear exception, RTOS tasks run with fixed-size stacks allocated at creation time. When a task exceeds its allocated stack, the resulting corruption silently overwrites adjacent memory — often another task’s stack, a global variable, or heap metadata — leading to seemingly random crashes, HardFaults, or data corruption that appear long after the actual overflow occurred. This article explores the mechanisms available for detecting and monitoring stack usage in FreeRTOS and other popular RTOS environments, and presents practical strategies for preventing stack-related failures in production firmware.

Why Stack Overflow Is Dangerous in RTOS

Each task in an RTOS maintains its own stack, with the size specified when the task is created via xTaskCreate() or xTaskCreateStatic(). The stack holds local variables, function call return addresses, and the processor context saved during context switches. On ARM Cortex-M processors, the hardware automatically pushes eight registers (xPSR, PC, LR, R12, and R3–R0) onto the stack on exception entry, and the RTOS saves the remaining callee-saved registers (R4–R11) during the context switch.

The danger lies in what happens when a task’s stack usage exceeds its allocation. The overflow silently writes past the stack boundary into whatever memory lies adjacent. There is no hardware boundary protection by default — the Memory Protection Unit (MPU) can be configured to guard stack regions, but this is not universally enabled. The symptoms are notoriously difficult to diagnose: the system may run for hours before the corrupted memory is accessed, and the crash site often has no apparent connection to the task that actually caused the overflow.

Common contributors to stack overflow in RTOS environments include deeply nested function calls, large local arrays or structures, recursive algorithms, printf() and floating-point formatting (which can consume hundreds of bytes of stack, depending on the libc, toolchain, and format options), and interrupt service routines that use the task stack space.

FreeRTOS Stack Overflow Detection Methods

FreeRTOS provides three runtime stack overflow detection methods, controlled by the configCHECK_FOR_STACK_OVERFLOW configuration constant in FreeRTOSConfig.h. Each method adds a small overhead to the context switch time, making them suitable primarily for development and testing rather than production.

Method 1 — Stack Pointer Check

Set configCHECK_FOR_STACK_OVERFLOW to 1. When a task is swapped out, the kernel checks that the stack pointer remains within the valid stack range. This is the fastest method but can miss overflows that occur between context switches — for example, if a function call pushes deep into the stack but returns before the next context switch.

Method 2 — Canary Pattern Check

Set configCHECK_FOR_STACK_OVERFLOW to 2. At task creation, the entire stack is filled with a known pattern (typically 0xA5). On each context switch, the kernel checks the last 16 bytes of the stack space to verify the pattern remains intact. This catches many overflows that Method 1 misses, at a modest additional cost. It is the recommended setting for development.

Method 3 — ISR Stack Check

Set configCHECK_FOR_STACK_OVERFLOW to 3. Available only on selected ports, this method adds checking for the ISR stack in addition to task stacks. When an ISR stack overflow is detected, a configASSERT is triggered (not the hook function, since the hook is task-specific).

When any method detects an overflow, the kernel calls the stack overflow hook function, which you must implement:

void vApplicationStackOverflowHook(TaskHandle_t xTask,
                                    char *pcTaskName) {
    /* Log the offending task name for debugging.
       Typical actions: trigger breakpoint, log via UART,
       or enter infinite loop for debugger attachment. */
    for (;;) {
        /* Trap here so debugger can inspect */
    }
}

It is critical to understand that there is no way to recover from a stack overflow — the memory has already been corrupted. The hook exists solely for debugging. Some processors fault before the kernel hook fires, so defining configASSERT() and using a fault handler to dump the offending task is also advisable.

Runtime Stack Monitoring with High Water Mark

Beyond the overflow detection hook, FreeRTOS provides uxTaskGetStackHighWaterMark() for continuous monitoring. This function returns the minimum amount of remaining stack space (in words, not bytes) since the task started — the “high water mark” of stack usage. On a 32-bit MCU, a return value of 1 means 4 bytes remain unused.

void monitor_task_stack(void *pvParameters) {
    TaskHandle_t tasks[] = { sensorTaskHandle,
                             commsTaskHandle,
                             controlTaskHandle };
    const char *names[] = { "Sensor", "Comms", "Control" };

    while (1) {
        for (int i = 0; i < 3; i++) {
            UBaseType_t hwm = uxTaskGetStackHighWaterMark(tasks[i]);
            /* Log or assert on low watermark */
            if (hwm < 20) {
                log_warning("%s stack HWM: %u words", names[i], hwm);
            }
        }
        vTaskDelay(pdMS_TO_TICKS(5000));
    }
}

A practical strategy is to periodically log the high water mark for all tasks during development and integration testing, then set final stack sizes to the observed peak usage plus a safety margin of 25–50%. STM32 HAL operations, especially those involving printf() or floating-point arithmetic, often consume significantly more stack than developers expect.

Static Analysis and Sizing Strategies

Rather than guessing stack sizes, modern toolchains offer static analysis tools that compute worst-case stack depth:

• GCC -fstack-usage: Generates a .su file for each compilation unit listing the stack usage of every function. Combined with the call graph, you can compute the worst-case stack depth per task.
• GCC -fcallgraph-info=su: Can emit call-graph information decorated with stack-usage data, which can help estimate worst-case depth when combined with task entry points.
• AbsInt StackAnalyzer: A commercial static-analysis tool that computes worst-case stack usage and can account for whole-program behavior, including task entry points and interrupt effects.

A practical starting point for sizing a task stack is:

task_stack = worst_case_function_depth
           + context/context-switch overhead
           + interrupt-related overhead 
           + safety_margin (25-50%)

On ARM Cortex-M with FreeRTOS, the exact overhead depends on the port, whether floating-point context is in use, and how exceptions interact with the active stack pointer. Treat any byte-count estimate as port-specific and validate it empirically with high-watermark monitoring and, where available, static analysis.

MPU-Based Stack Guarding

For production systems where runtime checking is disabled for performance, the Memory Protection Unit offers hardware-enforced stack boundaries. FreeRTOS supports MPU-aware tasks created with xTaskCreateRestricted():

/* Define stack with required alignment (power of 2 for MPU) */
static StackType_t sensorStack[SENSOR_STACK_SIZE]
    __attribute__((aligned(SENSOR_STACK_SIZE * sizeof(StackType_t))));

TaskParameters_t sensorParams = {
    .pvTaskCode   = sensor_task,
    .pcName       = "Sensor",
    .usStackDepth = SENSOR_STACK_SIZE,
    .pvParameters = NULL,
    .uxPriority   = SENSOR_PRIORITY | portPRIVILEGE_BIT,
    .puxStackBuffer = sensorStack,
    /* MPU region definitions for the task */
    .xRegions = {
        { (void *)0x40002800, 0x100, portMPU_REGION_READ_WRITE },
        { 0, 0, 0 },
        { 0, 0, 0 }
    }
};

xTaskCreateRestricted(&sensorParams, &sensorTaskHandle);

When a task overflows its MPU-guarded stack, the processor immediately triggers a MemManage fault, providing instant and precise detection — no adjacent memory is corrupted. The trade-off is increased memory alignment requirements (MPU regions must be sized and aligned to powers of two) and added implementation complexity. Availability and capability of MPU support are architecture and device-specific; for example, Cortex-M0 has no MPU, while Cortex-M0+ may include an optional MPU depending on the implementation.

Best Practices Summary

The most robust approach combines static analysis to establish initial estimates, runtime high-water-mark monitoring during integration to validate those estimates, and MPU guarding in production for immediate fault detection. Never deploy an RTOS application without verifying stack sizes under worst-case conditions — including maximum interrupt nesting depth, peak recursive call depth, and the largest local variable allocations.

References

• FreeRTOS Stack Overflow Checking Documentation — https://www.freertos.org/Documentation/02-Kernel/02-Kernel-features/09-Memory-management/02-Stack-usage-and-stack-overflow-checking
• FreeRTOS Kernel Book, Chapter 13: Troubleshooting (Stack Overflow Section) — https://github.com/FreeRTOS/FreeRTOS-Kernel-Book/blob/main/ch13.md
• ARM Cortex-M Programming Guide to Memory Protection Unit — https://developer.arm.com/documentation/dui0552/latest
• Microchip FreeRTOS Stack Overflow Checking Guide — https://onlinedocs.microchip.com/oxy/GUID-F3CEAE3B-C3C1-4B92-B031-4E07B8ACCD81-en-US-3/GUID-B846AD58-A80A-432A-B73B-E14DC67ADD19.html
• STM32 FreeRTOS Best Practices (Stack Sizing and Hooks) — https://dev.st.com/stm32cube-docs/mw-freertos/2.0.0/en/docs/markup/freertos_best_practices.html