FreeRTOS SMP: Multi-Core Task Affinity & Load Balancing

FreeRTOS SMP extends the single-core scheduler to multiple identical cores sharing memory. The kernel runs as one logical instance: a single shared array of ready lists (pxReadyTasksLists) is used across all cores. Cores concurrently access these lists to find the highest-priority task whose affinity mask allows it to run on that core. This works well for throughput-oriented workloads but introduces non-determinism for hard real-time tasks if tasks are allowed to float between cores.

The SMP scheduler relies on a few key configuration constants:

• configNUMBER_OF_CORES — number of cores (1 = single core, >1 = SMP)
• configUSE_CORE_AFFINITY — enables vTaskCoreAffinitySet()/Get()

Note that task affinity defaults to tskNO_AFFINITY (all cores) when created.

On a dual-core RP2040 (Cortex-M0+) or dual-core ESP32 (Xtensa LX6), the same FreeRTOS image runs on both cores. Core 0 starts first, initializes the kernel, creates tasks, and then calls vTaskStartScheduler(). The port-specific startup code within the scheduler automatically releases Core 1 from reset and starts scheduling on it—you do not call vTaskStartScheduler() on Core 1 manually. Note that FreeRTOS SMP requires identical cores (Symmetric Multiprocessing) and cannot be used on asymmetric chips like the STM32H7 (Cortex-M7 + M4), which require an AMP (Asymmetric Multiprocessing) approach.

Task Affinity API

/* Set affinity mask: bit 0 = core 0, bit 1 = core 1, etc. */
void vTaskCoreAffinitySet(TaskHandle_t xTask, UBaseType_t uxAffinityMask);

/* Get current affinity mask */
UBaseType_t uxTaskCoreAffinityGet(TaskHandle_t xTask);

/* Example: pin a task to core 1 only */
vTaskCoreAffinitySet(xHandle, (1UL << 1));

/* Example: allow task to run on cores 0 and 2 (3-core system) */
vTaskCoreAffinitySet(xHandle, (1UL << 0) | (1UL << 2));

Scheduler Behavior

+-----------------------------------------------------------+
|               SHARED READY LISTS (pxReadyTasksLists)      |
|                                                           |
|  Priority 5: [Task A (Core 0/1)]  [Task F (Core 0 Only)]  |
|  Priority 4: [Task D (Core 0/1)]  [Task E (Core 0/1)]     |
+-----------------------------------------------------------+
         ^                        ^
         |                        |
 +-------+--------+       +-------+--------+
 |     CORE 0     |       |     CORE 1     |
 | Checks highest |       | Checks highest |
 | priority task  |       | priority task  |
 | matching its   |       | matching its   |
 | core mask (0)  |       | core mask (1)  |
 +----------------+       +----------------+

When a core becomes idle or a scheduling event occurs, the core checks the shared pxReadyTasksLists array. It scans from the highest priority downwards, looking for a ready task whose affinity mask has the bit for this core set. The list itself is protected by spinlocks since multiple cores may attempt to access it simultaneously.

Migration Latency Breakdown (Cortex-M33 with caches, 200 MHz)

Note: The RP2040 (Cortex-M0+) has no CPU data caches, so cache maintenance cost is zero on that platform. The table below represents a cached Cortex-M33 system (e.g., RP2350).

Practical Guidelines

1. Default to unpinned — Let the scheduler balance load. Only pin when you have a measured reason.
2. Pin ISR-adjacent tasks — Tasks that service core-specific interrupts (e.g., core 0 handles UART0, core 1 handles UART1) should be pinned to avoid IPI storms.
3. Monitor migrations — Enable configUSE_TRACE_FACILITY and use the traceTASK_SWITCHED_IN hook to detect core changes (compare the current core ID against the task’s previous core). There is no built-in taskMIGRATE event, but tools like Percepio Tracealyzer can reconstruct migration timelines from switch-in events. Frequent migrations indicate poor affinity choices.
4. Cache-aware affinity — On cores with private L1 (Cortex-M7, M33, M55), pin hot loops to a specific core to keep L1 warm and avoid cold-cache performance penalties upon migration.

Example: Pinning a Motor Control Loop to Core 1

TaskHandle_t xMotorTask;

void vMotorControlTask(void *pvParameters) {
    TickType_t xLastWakeTime = xTaskGetTickCount();
    
    for (;;) {
        /* Read core-local ADC on core 1 */
        uint16_t adc_val = adc_read_core1(ADC_CH_CURRENT);
        /* Compute PWM update - cache-hot loop */
        motor_update_pwm(adc_val);
        vTaskDelayUntil(&xLastWakeTime, pdMS_TO_TICKS(1));
    }
}

void main(void) {
    xTaskCreate(vMotorControlTask, "Motor", 1024, NULL, 5, &xMotorTask);
    /* Pin to core 1 (bit 1) */
    vTaskCoreAffinitySet(xMotorTask, (1UL << 1));
    
    /* Core 0 runs comms, logging, UI */
    vTaskStartScheduler();
}

Common Pitfalls

Summary

FreeRTOS SMP brings multi-core support to the familiar FreeRTOS API. The key abstraction is task affinity—a bitmask controlling which cores a task may run on. Unpinned tasks migrate automatically for load balancing; pinned tasks stay put for determinism and cache locality. Migration overhead is ~5-15µs on Cortex-M, acceptable for most control loops but problematic for sub-microsecond deadlines. Profile before pinning, pin with purpose, and trace migrations to verify behavior.

Related Reading

• Rate Monotonic Priority Ceiling Protocol Fixes Priority Inversion — Priority ceiling protocol for multi-core resource sharing
• FreeRTOS Task Notification Latency vs Queue Performance — Lightweight IPC that works across cores
• ARM Cortex-M NVIC Interrupt Latency Optimization — Core-local interrupt handling

References

1. FreeRTOS SMP Reference Manual, FreeRTOS Kernel V11.1.0, “Symmetric Multiprocessing (SMP) Support”
2. Raspberry Pi RP2350 Datasheet, “Dual-core Cortex-M33” — Inter-processor interrupts and mailbox
3. RP2040 Datasheet, Raspberry Pi, “Dual-core Cortex-M0+ Processor” — Hardware spinlock and mailbox for inter-core sync
4. ESP32 Technical Reference Manual, Espressif Systems, “Dual-core Xtensa LX6” — SMP architecture and inter-core hardware synchronization
5. “Real-Time Systems on Multicore Platforms”, Björn Brandenburg, MPI-SWS, 2018 — Schedulability analysis for global vs. partitioned scheduling
6. FreeRTOS Kernel Source, tasks.c — vTaskCoreAffinitySet, prvSelectHighestPriorityTask SMP implementation