Tracealyzer for FreeRTOS
FreeRTOS is a popular real-time operating system (RTOS) for embedded software and often used on 32-bit microcontrollers such as STM32 and ESP32. Developing embedded software can be challenging, partly due to poor observability. Percepio Tracealyzer offers deep observability that lets you solve issues quickly and speed up development.
Solve Issues Quickly
- Visualize task execution
- Spot issues and drill down
- Full system debugging
- CPU load, timing variations
- Analyze memory usage
- Find inefficient code
- Matching design intent?
- Best practices followed?
- Performance requirements?
Deep Observability for FreeRTOS Software
Tracealyzer provides many views on different levels of abstraction. All are connected for a streamlined workflow. This way, you can spot issues in high-level overviews, such as a spike in the CPU load, and then drill down into the details to understand the cause.
The trace view provides a detailed timeline of task scheduling and interrupts, FreeRTOS API calls as well as custom “user events” logged in the application code. This way, you can check if your code executes as intended and learn how to improve the software design. The large set of visual overviews includes for example CPU load, task timing, stack usage and heap memory allocation (i.e. malloc/free). Such overviews provides a profile of the resource usage over time, which helps you see the big picture, spot anomalies and optimize the system.
Tracealyzer supports all recent versions of FreeRTOS and all relevant types of FreeRTOS services, including tasks, queues, semaphores, mutexes, event groups, queue sets, stream buffers, message buffers, task notification and software timers. Some examples are presented below.
Watch how Tracealyzer can help your development.
Trace your Tasks
The core of a FreeRTOS system is the concept of tasks, which are threads scheduled by the FreeRTOS kernel to provide multitasking. Each task has its own stack and a fixed scheduling priority, which is very important for functional correctness and performance. Tracealyzer lets you analyze the behavior and performance of different priority assignments, as well as the stack usage of the tasks. If your stacks are too small, you risk nasty bugs due to stack overflow. If they are too large, you are wasting precious RAM that might be needed elsewhere in your application.
The trace view shows both the task scheduling and calls to FreeRTOS API functions. This allows you to see exactly when tasks are activated, when they actually execute, and why they sometimes don’t execute as intended. You can also see an overview showing which tasks are consuming the processor time, as shown in the “CPU Load Graph”. Detailed statistics is also available, like task execution times and response times.
Trace Kernel API Calls
FreeRTOS offers several APIs for passing data between tasks and for protecting shared resources, such as queues, semaphores and mutexes. These API functions may block the calling task’s execution until another task has performed a matching operation. Such API calls may form a network of dependencies between the tasks that is not apparent in the source code. Tracealyzer can visualize the task interactions, which makes it far easier to understand, debug and optimize FreeRTOS applications. Some examples are provided below.
Queues allow FreeRTOS tasks to communicate by sending and receiving messages. Queues have a fixed number of slots and messages are normally buffered in FIFO order. The sender calls xQueueSend to add a message in the queue. The receiver task calls xQueueReceive to fetch the next message from the queue.
Queue operations might block if trying to send a message to a full queue, or trying to read a message from an empty queue. The screenshot shows how Tracealyzer displays queue usage and blocking. The TX task sends two messages to the queue. This wakes up the RX task (hence the green label) and it receives the two messages. Finally, the task is blocked (red label) by xQueueReceive since the queue is now empty.
The queue forms a dependency between TX and RX, that can be seen in the Communication Flow graph below. The direction of the arrows show the usage, i.e. that TX is sending to the queue and RX is receiving from it. Double-clicking on any of the nodes shows the corresponding events.
Blocking may also result in a timeout, depending on the timeout argument for xQueueReceive and xQueueSend. These are displayed as orange labels in Tracealyzer. Timeouts are possible on many FreeRTOS API calls, not just for queues, and are really important to keep track of. Some might be intentional, while other may indicate serious errors.
Queue usage + blocking
Semaphores allow for waking up tasks on a particular event. A semaphore can be regarded as a signal, which is sent by calling xSemaphoreGive and received by calling xSemaphoreTake. An example screenshot is shown below. Note that the RX task is blocked by a previous call to xSemaphoreTake and only wakes up after TX has called xSemaphoreGive.
Semaphores may also be used to protect critical sections in the code, i.e. mutual exclusion. However, when using a regular semaphore for this purpose there is a risk for priority inversion, meaning that high-priority tasks are delayed by lower-priority tasks. Tracealyzer makes it easy to spot issues like this, for example using the “Actor Instance Graph” showing the task response times.
FreeRTOS offers Mutex objects to allow critical sections without the risk for priority inversion. Mutexes are used in almost the same way as semaphores and use the same API functions, xSemaphoreTake and xSemaphoreGive. However, mutexes are typically locked (taken) and released (given) in sequence and by the same task, as shown below, to provide mutual exclusion.
Mutex objects implement the priority inheritance protocol to avoid priority inversion. The below example shows how Mutexes appear in Tracealyzer. The blue labels show the priority inheritance, where the priority of the holding task is raised (inherited) to the same level as the waiting task to avoid unsuitable preemptions.
This trace view, combined with a large set of visual overviews, makes it easy to understand the real-time behavior of your FreeRTOS system. You can verify that task priorities are suitable and that the system works as designed. If something seems to be wrong, you can isolate and debug the real-time behavior without halting the system, especially in combination with application logging (see below). You can also profile the system to ensure it runs in an efficient manner, so you get the most out of your hardware.
You may log custom events and data in your application code and display it in Tracealyzer, together with the FreeRTOS kernel trace. This provides visibility into the application code at runtime, for easier debugging and analysis. This is especially useful for real-time algorithms, such as signal processing and control loops.
In the above example, variables in a PID control loop have been logged as user events (yellow labels). The loop variables can then be plotted over time, in parallel with the software execution, to analyse latency and jitter together with the task execution..
State transitions can also be logged and displayed in a logic analyzer view where multiple state variables are shown over time. The result can be shown within the trace view (as shown on the left) or summarized as a state graph (on the right), making it easy to spot incorrect behavior.
Learn more in the videos and in our RTOS Debug Portal, with many tutorials and articles about Tracealyzer.
How it works
You do not need any particular hardware to use Tracealyzer, just a suitable development board. You can capture snapshots of the latest activity from a trace buffer in RAM, or stream the data to host using for example Ethernet or a debug probe like SEGGER J-Link, IAR I-jet or Keil ULINK. This way, you can monitor your system over long periods of time and capture any issues.
Tracealyzer for FreeRTOS relies on the Percepio TraceRecorder library, available on Github under the Apache 2.0 license. This is integrated in your project within a few minutes by following the Getting Started guide for FreeRTOS. Also make sure to read the Tracealyzer user manual to learn how to connect and record traces (see the Help menu).
Tracealyzer works with most embedded processors (e.g. Arm devices and others) and essentially any C/C++ compiler. Percepio provides plugins and integrations for common development tools, including Eclipse/GDB, STM32CubeIDE, Keil MDK, IAR and Lauterbach.
Problems that otherwise would take days to solve are obvious with this tool and just a quick fix.