Fixing FreeRTOS Queue Overrun Data Loss in ISR Contexts

Queue overrun in FreeRTOS ISRs is a silent killer. The ISR fires, calls xQueueSendFromISR, the queue is full, and the function returns pdFAIL. Your data vanishes. No assert, no callback, no trace — just gone. This happens because ISRs cannot block, and FreeRTOS queues have fixed depth. When the producer ISR outpaces the consumer task, the queue fills and subsequent xQueueSendFromISR calls drop data.

The fix isn’t a single setting. It’s a combination of proper queue sizing, overflow detection, and architectural choices that match your data criticality. Let’s walk through the root causes, the FreeRTOS primitives that actually work in ISR context, and the patterns that prevent data loss without sacrificing real-time guarantees.

The Problem: Why ISR Queues Lose Data

FreeRTOS queues are fixed-size circular buffers. Each slot holds either a pointer (4 bytes on 32-bit) or a copy of the queued item. When you create a queue with xQueueCreate(10, sizeof(MyMsg)), you get exactly 10 slots. No more.

In task context, xQueueSend blocks until space exists. In ISR context, xQueueSendFromISR cannot block — it returns pdFAIL immediately if the queue is full. This is by design: an ISR that blocks would stall interrupt handling, break latency guarantees, and potentially deadlock the kernel.

// Task context — blocks until space available
BaseType_t xQueueSend(QueueHandle_t xQueue, const void* pvItemToQueue, TickType_t xTicksToWait);

// ISR context — NEVER blocks, returns immediately
BaseType_t xQueueSendFromISR(QueueHandle_t xQueue, const void* pvItemToQueue, BaseType_t* pxHigherPriorityTaskWoken);

The pxHigherPriorityTaskWoken parameter is the only coordination mechanism. If the send unblocks a higher-priority task waiting on the queue, the kernel sets *pxHigherPriorityTaskWoken = pdTRUE. Your ISR must then call portYIELD_FROM_ISR(pxHigherPriorityTaskWoken) to trigger an immediate context switch.

But if the queue is full, nothing happens. No task is woken. No callback fires. The data is simply not enqueued.

Root Cause Analysis: Three Common Patterns

1. Burst Traffic Exceeds Queue Depth

A DMA completion ISR fires for each buffer. If the consumer task processes one buffer per 100 µs but DMA completes a buffer every 10 µs, a queue of depth 10 fills in 100 µs. The 11th ISR drops data.

// ISR — runs every 10 µs
void DMA_IRQHandler(void) {
    BaseType_t xHigherPriorityTaskWoken = pdFALSE;
    Buffer_t* pBuf = get_completed_buffer();
    
    // Queue depth 10 — fills in 100 µs!
    if (xQueueSendFromISR(xDmaQueue, &pBuf, &xHigherPriorityTaskWoken) != pdPASS) {
        // DATA LOST — no recovery, no notification
        overflow_count++;
    }
    portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
}

2. Consumer Task Starvation

The consumer task has lower priority than other tasks. It gets preempted and never drains the queue. The ISR keeps filling until full.

// Consumer task — priority 2 (low)
void ConsumerTask(void* pvParams) {
    Buffer_t* pBuf;
    while (1) {
        if (xQueueReceive(xDmaQueue, &pBuf, portMAX_DELAY) == pdPASS) {
            process_buffer(pBuf);  // Takes 100 µs
        }
    }
}

// High-priority task (priority 5) hogs CPU
void HighPriorityTask(void* pvParams) {
    while (1) {
        do_heavy_computation();  // Runs for milliseconds
    }
}

3. Priority Inversion on Queue Access

A medium-priority task preempts the consumer while it holds the queue lock internally. The ISR (highest priority) tries to send and spins or fails. This is less common with queues (no priority inheritance) but affects mutex-protected shared data.

Solution 1: Size the Queue for Worst-Case Burst

The queue depth must absorb the maximum burst size at the maximum ISR rate, given the minimum consumer drain rate.

QueueDepth ≥ ceil( (ISR_Rate_Hz / Consumer_Rate_Hz) * BurstFactor )

For a 100 kHz ISR and 10 kHz consumer with 2x burst factor: QueueDepth ≥ 20.

But RAM is finite. A queue of 100 items at 64 bytes each = 6.4 KB. On a Cortex-M with 64 KB RAM, that’s 10%. Scale accordingly.

// Calculate minimum depth at runtime
#define ISR_FREQ_HZ       100000
#define CONSUMER_FREQ_HZ  10000
#define BURST_FACTOR      2
#define MIN_QUEUE_DEPTH   ((ISR_FREQ_HZ / CONSUMER_FREQ_HZ) * BURST_FACTOR)  // = 20

QueueHandle_t xDmaQueue = xQueueCreate(MIN_QUEUE_DEPTH, sizeof(Buffer_t*));
configASSERT(xDmaQueue != NULL);

Rule of thumb: Start with 2× the calculated minimum. Monitor uxQueueSpacesAvailable() in production and adjust.

Solution 2: Detect and Handle Overflow Explicitly

Never ignore the return value of xQueueSendFromISR. At minimum, count overflows. Better: implement a fallback path.

volatile uint32_t ulQueueOverflowCount = 0;
volatile uint32_t ulQueueOverflowTimestamp = 0;

void DMA_IRQHandler(void) {
    BaseType_t xHigherPriorityTaskWoken = pdFALSE;
    Buffer_t* pBuf = get_completed_buffer();
    
    if (xQueueSendFromISR(xDmaQueue, &pBuf, &xHigherPriorityTaskWoken) != pdPASS) {
        // Overflow detected — Option A: Count and timestamp
        ulQueueOverflowCount++;
        ulQueueOverflowTimestamp = xTaskGetTickCountFromISR();
        
        // Option B: Fallback to ring buffer with overwrite (non-critical data)
        ring_buffer_overwrite(&rb_fallback, pBuf);
        
        // Option C: Signal consumer to wake up and drain NOW
        vTaskNotifyGiveFromISR(xConsumerTaskHandle, &xHigherPriorityTaskWoken);
    }
    
    portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
}

The consumer task can check overflow count periodically and log/alert:

void ConsumerTask(void* pvParams) {
    Buffer_t* pBuf;
    uint32_t last_overflow = 0;
    
    while (1) {
        if (xQueueReceive(xDmaQueue, &pBuf, pdMS_TO_TICKS(100)) == pdPASS) {
            process_buffer(pBuf);
        }
        
        // Check for overflow every 100 ms
        if (ulQueueOverflowCount != last_overflow) {
            log_warn("Queue overflow: %lu drops since last check", 
                     ulQueueOverflowCount - last_overflow);
            last_overflow = ulQueueOverflowCount;
        }
    }
}

Solution 3: Use Task Notifications for Lightweight Signaling

When you only need to signal “data ready” without buffering the data itself, task notifications are lighter than queues. They consume ~20 bytes per task (vs. queue overhead) and are faster.

// ISR — signal consumer, don't queue data
void DMA_IRQHandler(void) {
    BaseType_t xHigherPriorityTaskWoken = pdFALSE;
    Buffer_t* pBuf = get_completed_buffer();
    
    // Store buffer in a lock-free ring buffer (separate from FreeRTOS)
    if (!ring_buffer_push(&rb_dma, pBuf)) {
        // Ring buffer full — overwrite oldest
        ring_buffer_overwrite(&rb_dma, pBuf);
        ulRingOverflowCount++;
    }
    
    // Notify consumer — only 32-bit value, no queue allocation
    vTaskNotifyGiveFromISR(xConsumerTaskHandle, &xHigherPriorityTaskWoken);
    portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
}

// Consumer — wait for notification, then drain ring buffer
void ConsumerTask(void* pvParams) {
    while (1) {
        // Wait for notification (blocks, no polling)
        ulTaskNotifyTake(pdTRUE, portMAX_DELAY);
        
        // Drain all available buffers
        Buffer_t* pBuf;
        while (ring_buffer_pop(&rb_dma, &pBuf)) {
            process_buffer(pBuf);
        }
    }
}

When to use task notifications:

• Single producer, single consumer
• Data stored in separate lock-free structure (ring buffer, double buffer)
• You only need “wake up and check” signaling
• RAM is extremely constrained

When to stick with queues:

• Multiple producers or consumers
• Need built-in buffering with copy semantics
• Data size varies per message
• Simpler mental model for team

Solution 4: Double Buffering for Deterministic Zero-Copy

For high-throughput streams (ADC, camera, audio), double or triple buffering eliminates queue contention entirely. The ISR swaps pointers; the consumer processes the “full” buffer while the ISR fills the “empty” one.

typedef struct {
    uint8_t data[BUFFER_SIZE];
    volatile bool ready;
} DoubleBuffer_t;

DoubleBuffer_t buffers[2] = {0};
volatile uint8_t current_write = 0;
volatile uint8_t current_read = 0;

void ADC_IRQHandler(void) {
    // Fill current write buffer
    read_adc_into(&buffers[current_write].data[write_offset]);
    
    if (buffer_full) {
        buffers[current_write].ready = true;
        
        // Swap buffers atomically (single write on 32-bit)
        current_write ^= 1;
        
        // Notify consumer
        BaseType_t xHigherPriorityTaskWoken = pdFALSE;
        vTaskNotifyGiveFromISR(xConsumerTaskHandle, &xHigherPriorityTaskWoken);
        portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
    }
}

void ConsumerTask(void* pvParams) {
    while (1) {
        ulTaskNotifyTake(pdTRUE, portMAX_DELAY);
        
        // Process the ready buffer
        if (buffers[current_read].ready) {
            process_buffer(buffers[current_read].data);
            buffers[current_read].ready = false;
            current_read ^= 1;
        }
    }
}

This pattern:

• Zero queue overhead
• Zero data copies
• Deterministic latency (one buffer time)
• Requires atomic pointer/index swap (single instruction on Cortex-M)

Solution 5: Priority Ceiling Protocol for Shared Data

If the ISR and task share data protected by a mutex, you get priority inversion. FreeRTOS mutexes support priority inheritance, but mutexes cannot be used in ISRs.

The fix: never share mutex-protected data directly with ISRs. Instead, use a queue or notification to pass ownership.

// WRONG — mutex in ISR (will assert/crash)
void ISR_Wrong(void) {
    xSemaphoreTake(mutex, 0);  // INVALID — ISR cannot block
    shared_data = new_value;
    xSemaphoreGive(mutex);
}

// CORRECT — ISR sends command to task that owns the mutex
typedef enum { CMD_UPDATE_VALUE } CmdType_t;

typedef struct {
    CmdType_t type;
    uint32_t value;
} Command_t;

QueueHandle_t xCmdQueue;

void ISR_Correct(void) {
    BaseType_t xHigherPriorityTaskWoken = pdFALSE;
    Command_t cmd = {CMD_UPDATE_VALUE, new_value};
    xQueueSendFromISR(xCmdQueue, &cmd, &xHigherPriorityTaskWoken);
    portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
}

void OwnerTask(void* pvParams) {
    Command_t cmd;
    while (xQueueReceive(xCmdQueue, &cmd, portMAX_DELAY) == pdPASS) {
        xSemaphoreTake(mutex, portMAX_DELAY);
        shared_data = cmd.value;
        xSemaphoreGive(mutex);
    }
}

The task that owns the mutex is the only one that locks it. ISRs send commands. This is the priority ceiling protocol in practice: the task runs at the ceiling priority while holding the mutex, preventing inversion.

ASCII Art: Queue Overflow State Machine

+------------------------------------------------------------------+
|                    QUEUE OVERFLOW STATE MACHINE                   |
+------------------------------------------------------------------+
|                                                                  |
|   ISR FIRES                                                     |
|      |                                                           |
|      v                                                           |
|   +-----------+    Queue Full?    +-----------+                  |
|   | xQueueSend|------------------>|  pdFAIL   |                  |
|   | FromISR() |    (No Space)     |  Returned |                  |
|   +-----------+                   +-----------+                  |
|      |                                   |                        |
|      | pdPASS                             | Overflow              |
|      v                                   v                        |
|   +-----------+                   +-----------+                  |
|   | Data      |                   | HANDLE    |                  |
|   | Enqueued  |                   | OVERFLOW  |                  |
|   +-----------+                   +-----------+                  |
|      |                                   |                        |
|      |                                   +--> Count Overflow     |
|      |                                   |                        |
|      |                                   +--> Fallback Buffer    |
|      |                                   |                        |
|      |                                   +--> Task Notification  |
|      |                                   |                        |
|      |                                   +--> Alert/Log          |
|      |                                   |                        |
|      v                                   v                        |
|   +-------------------------------------------------------+      |
|   |              CONSUMER TASK DRAINS QUEUE                |      |
|   |  (xQueueReceive / ulTaskNotifyTake / Ring Buffer Pop) |      |
|   +-------------------------------------------------------+      |
|                                                                  |
+------------------------------------------------------------------+

Verification: Stress Testing the Queue

Write a test that hammers the queue at maximum ISR rate and verifies zero data loss.

#define TEST_DURATION_MS    5000
#define TEST_ISR_FREQ_HZ    100000
#define TEST_QUEUE_DEPTH    50

static volatile uint32_t test_sent = 0;
static volatile uint32_t test_received = 0;
static volatile uint32_t test_dropped = 0;

// Simulated high-rate ISR (run from high-priority timer task for test)
void TestProducerTask(void* pvParams) {
    TickType_t xNextWake = xTaskGetTickCount();
    const TickType_t xPeriod = pdMS_TO_TICKS(1000 / TEST_ISR_FREQ_HZ);
    
    while (test_running) {
        uint32_t value = test_sent++;
        BaseType_t xHigherPriorityTaskWoken = pdFALSE;
        
        if (xQueueSendFromISR(xTestQueue, &value, &xHigherPriorityTaskWoken) != pdPASS) {
            test_dropped++;
        }
        // Simulate portYIELD_FROM_ISR in test context
        if (xHigherPriorityTaskWoken) taskYIELD();
        
        vTaskDelayUntil(&xNextWake, xPeriod);
    }
    vTaskDelete(NULL);
}

void TestConsumerTask(void* pvParams) {
    uint32_t value;
    while (test_running || uxQueueMessagesWaiting(xTestQueue)) {
        if (xQueueReceive(xTestQueue, &value, pdMS_TO_TICKS(10)) == pdPASS) {
            test_received++;
            // Verify sequence
            configASSERT(value == test_received - 1);
        }
    }
    vTaskDelete(NULL);
}

void RunQueueStressTest(void) {
    test_sent = test_received = test_dropped = 0;
    test_running = true;
    
    xTaskCreate(TestProducerTask, "Prod", 1024, NULL, 5, NULL);
    xTaskCreate(TestConsumerTask, "Cons", 1024, NULL, 4, NULL);
    
    vTaskDelay(pdMS_TO_TICKS(TEST_DURATION_MS));
    test_running = false;
    vTaskDelay(pdMS_TO_TICKS(100));  // Drain remaining
    
    printf("Sent: %lu, Received: %lu, Dropped: %lu\n", 
           test_sent, test_received, test_dropped);
    configASSERT(test_dropped == 0);
    configASSERT(test_sent == test_received);
}

Run this on target hardware. If test_dropped > 0, increase queue depth or optimize consumer.

Summary

Key takeaways:

1. Never ignore xQueueSendFromISR return value — every pdFAIL is lost data
2. Size queues for burst, not average rate — monitor uxQueueSpacesAvailable() in production
3. Task notifications + ring buffers beat queues for high-rate single-producer streams
4. Double buffering eliminates queue contention entirely for streaming data
5. ISRs never take mutexes — send commands to the task that owns the mutex

Related Reading

• FreeRTOS Critical Sections and Synchronization
• FreeRTOS Task Notification Latency vs Queue Performance
• Zero-Copy DMA Patterns on ARM Cortex-M
• Lock-Free Ring Buffers for ISR-to-Task Communication
• Debugging Hard Faults on ARM Cortex-M: Systematic Approach

References

1. FreeRTOS Kernel Documentation — Queue Management: https://www.freertos.org/Documentation/02-Kernel/04-API-references/06-Queues/00-QueueManagement
2. FreeRTOS API Reference — xQueueSendFromISR: https://www.freertos.org/Documentation/02-Kernel/04-API-references/06-Queues/04-xQueueSendFromISR
3. FreeRTOS API Reference — Task Notifications: https://freertos.org/Documentation/02-Kernel/04-API-references/05-Direct-to-task-notifications/00-RTOS-task-notifications
4. Barry, R. “Mastering the FreeRTOS Real Time Kernel” — Chapter 5: Queue Management and Interrupt Safety
5. ARM Cortex-M Programming Guide to Memory Barriers: https://support.arm.com/documentation/100067/0618
6. “Real-Time Systems Design Patterns for Embedded Systems” — Priority Ceiling Protocol, Queue Sizing Strategies