Sr. FAE at Percepio
Not that long-ago, developers thought that a high-level language was not really needed when building embedded applications. Writing a good UART driver in assembler was done in only a day or two, so why bother with C?
Besides, even if you wrote in C, who could trust that the generated code did exactly what was intended?
However, faster processors and better compilers, together with more complex system demands made higher abstraction necessary, and thus C/C++ became the standard development language. At the same time, debugging tools also evolved to support the new abstraction level, by allowing for source-level debugging. This did not replace the low level views of assembly code and register contents, but the source view became the standard debugging perspective. After all, it is quite natural to debug on the same abstraction level as you develop.
Today the use of an RTOS is increasingly common, as they often greatly facilitate development of the increasingly complex systems we design today. When introducing an RTOS, we raise the abstraction level once again, much like when introducing a high-level language.
An RTOS is a very special piece of software. It controls the execution of the other software modules, as seemingly parallel tasks. Moreover, it may suspend the running task at any time and switch to another task, depending on their priority and operations.
How do you debug a system where an RTOS decides what part of the software to run at any given time? Placing a breakpoint in a function will work fine, but what will be executed next time the system takes a step? The next line in the same function? An ISR? Part of the RTOS Kernel?
How do you decide which priorities to use and how do you detect if this decision is wrong? And how do you know if the system executes the way you intended?
For these and many other questions we have developed Tracealyzer for µC/OS-III, a software tracing tool with RTOS awareness and powerful visualization. This works as a complement to traditional debugging tools, providing a timeline at the RTOS level that greatly facilitates debugging, validation, and performance optimization. In short: better insight into RTOS-based firmware, enabling higher quality software in shorter time.
The software consists of two parts, the trace recorder library that is added to the embedded RTOS firmware, and Tracealyzer, the analysis and visualization software.
Tracealyzer provides more than 25 different views showing the actual behavior of your system at the RTOS level, i.e., the real-time execution of the tasks, the amount of CPU time used by each task over the timeline, and all events from RTOS service calls.
The Trace Recorder
The recorder library is the target-side component of Tracealyzer. The recorder has two different modes: in snapshot mode it keeps the recorded data in an internal ring-buffer, while in streaming mode the data is transferred continuously to the host PC. The default setup is the snapshot mode. It is highly optimized for memory efficiency and typically only uses 4 bytes per event. This allows for keeping a decent trace even if your MCU only has a few KB of RAM available.
The other method, to continually send the data to the debug PC is called Streaming mode. Here any method can be used that is available on the system, but examples are shipped using TCP/IP, USB CDC (simulating a fast serial port) and for SEGGER J-Link.
The first step is to add the recorder library to your system. The recorder source code is included with the Tracealyzer application, and a detailed integration guide is found in the user manual. However, the steps needed are the same as for any source code: add the source code to the build project, and update the include paths so that the recorder header files are found by the compiler. You need to configure a few mandatory settings in trcConfig.h, but those are well documented and pretty straight forward, like what processor family that is used.
In my example, I have started with a simple example flashing a LED using the µC/OS-III RTOS and Simplicity Studio as IDE. My target system is a STK3700 board (EFM32™ Giant Gecko Starter Kit) equipped with a EFM32GG9901024 device.
This first step is quite easy using Simplicity Studio, just copy the library files to a project directory, all source code files are then automatically added to the build.
Next thing is that the location of header files must be added to the compiler settings in the project. For snapshot mode, you only need to add the two first directories (include and config). For streaming mode, you also need to specify a stream port directory (where trcStreamingPort.h is found).
NOTE: This guide is intended for Tracealyzer v3.1.0 or later. The recorder library is different in older versions.
Then we open trcConfig.h, select ARM Cortex-M as hardware port and also add “#include “em_system.h” to provide access the ARM CMSIS functions.
Now the system should build correctly, but the recorder is not yet active. In os_cfg.h the symbol OS_CFG_TRACE_EN must be defined as DEF_ENABLED for the trace recorder to be included.
To initialize the trace recorder, in you main routine, add a call to vTraceEnable. Depending on the recorder mode I give the function different arguments. Note that in snapshot mode, we must call vTraceEnable before OSInit, while streaming mode requires the opposite order. This difference is not ideal, but necessary and can easily be handled with some pre-processor directives.
#if TRC_CFG_RECORDER_MODE == TRC_RECORDER_MODE_SNAPSHOT
OSInit(&err); /* Init uC/OS-III.*/
#if TRC_CFG_RECORDER_MODE == TRC_RECORDER_MODE_STREAMING
In streaming mode, the vTraceEnable(TRC_START_AWAIT_HOST) call will block the system until the “Start Recording” button is pressed in the Tracealyzer GUI.
If using parameter TRC_INIT instead, the call does not block but only initializes the recorder, it does not activate it. The tracing is activated when the user clicks the “Start Recording” button the Tracealyzer GUI.
NOTE: In the project I started from, I noted a statistics function that used the timestamping feature in µC/OS. This was found to interfere with the trace recorder timestamping, and was therefore disabled by setting OS_CFG_TS_EN in OS_Cfg.h to DEF_DISABLED.
Saving a trace snapshot
In snapshot mode, the data is kept in a RAM buffer. This must be uploaded to host for analysis and visualization. As the STK3700 board has an integrated SEGGER J-Link debugger (supported by Tracealyzer) this is very easy.
Assuming the target is running, in Tracealyzer select J-Link -> Read Trace (Snapshot). This will read the data from target memory, analyze it and then display the visualizations. The first time, you need to specify the device and the approximate memory range of the RAM buffer. The location of the buffer is found automatically, so the exact address is not required.
Continuous trace streaming
To use the recorder in streaming mode, open trcConfig.h and change the value of TRC_CFG_RECORDER_MODE to TRC_RECORDER_MODE_STREAMING.
Next, decide which “stream port” to use, i.e., how to transfer the data. Tracealyzer comes with several predefined stream ports, which are just a header file with a few basic I/O definitions. They are found in the directory “streamports” and include support for SEGGER J-Link as well as examples for TCP/IP and USB CDC. Select which stream port to use by setting the compilers include paths accordingly – it should identify the directory where the right trcStreamingPort.h file is located.
The last thing to consider is how to initialize the trace. If you want to start the system normally and but allow the Tracealyzer GUI to activate the tracing, use vTraceEnable(TRC_INIT).
If the initial startup sequence is important, use vTraceEnable(TRC_START_AWAIT_HOST) instead. This blocks the system until the tracing is activated from Tracealyzer, and thereby captures the execution from this call and forwards.
Finally, vTraceEnable(TRC_START) starts the tracing directly, and initializes the recorder if not already done. In streaming mode, this is mainly intended for streaming to device storage, like a flash file system. It works also for target->host streaming, but it is important that the host-side receiver and the streaming interface is ready to receive the data before calling this function, otherwise you may get blocking or lost data.
After building and downloading the system to the board, start execution. Open Tracealyzer -> File Connect to Target System -> Press “Start Recording” when you wish, and after a while stop the recording. Note the diagram over CPU-Usage while collecting the trace data.
The project with Trace Recorder can be easily downloaded from us, just contact email@example.com for a link.