Tracealyzer for On Time RTOS-32


Tracealyzer for On Time RTOS-32 is a runtime diagnostics tool for embedded software systems based on On Time RTOS-32. Tracealyzer gives developers an unprecedented insight into the runtime behavior, which allows for reduced troubleshooting time and improved software quality, performance and reliablity. Complex software problems which otherwise may require many hours or days to solve, can with Tracealyzer be understood quickly, often in a tenth of the time otherwise required. This saves you many hours of troubleshooting time, which quickly returns the investment cost of this tool. Moreover, the increased software quality resulting from using Tracealyzer can reduce the risk of defective software releases, causing damaged customer relations.

The insight provided by Tracealyzer also allows you to find opportunities for optimizing your software. You might have unnecessary resource conflicts in your software, which are "low hanging fruits" for optimization and where a minor change can give a significant improvement in real-time responsiveness and user-perceived performance. By using Tracealyzer, software developers can reduce their troubleshooting time and thereby get more time for developing new valuable features. This means a general increase in development efficency and a better ability to deliver high-quality embedded software within budget.

Tracealyzer has been developed in partnership with the RTOS developers at On Time Informatik GmbH, based on Percepio's leading Tracealyzer technology, developed since 2004.

Tracealyzer provides more than 20 interlinked views of the runtime behavior, including task scheduling and timing, interrupts, interaction between tasks as well as user events generated from your application. Tracealyzer can be used side-by-side with a traditional debugger and complements the debugger view with a higher level perspective, ideal for understanding the complex errors where a debuggers perspective is too narrow.

Tracealyzer is more than a just a viewer. It contains several advanced analyses developed since 2004, that helps you faster comprehend the trace data. For instance, it connects related events, which allows you to follow messages between tasks and to find the event that triggers a particular task instance. Morover, it provides various higher level views such as the Communication Flow graph and the CPU Load graph, which make it easier to find anomalies in a trace.

Tracealyzer does not depend on additional trace hardware, which means that it can be used in deployed systems to capture rare errors which otherwise are hard to reproduce.

System Overview

The Tracealyzer solution consists of two parts:

The PC application Tracealyzer runs on Microsoft Windows and Linux, the latter using the Mono runtime environment for .NET applications.

For information on the On Time RTOS-32 kernel trace function, please refer to the RTOS-32 documentation. See "Kernel Tracer" in the "RTKernel-32" section of "RTKernel-32 Programming Manual" and "RTKernel-32 Reference Manual".

Using Tracealyzer

Tracealyzer provides several graphical views which gives different perspectives of the runtime behavior, based on a recording of task scheduling and On Time RTOS-32 kernel service calls. You may also choose to include interrupt service routines (ISRs) as well as application events by adding own calls to the trace recorder library. Application events, or user events, contains custom data.

This section is intended to give a quick overview of the features and how this tool can be used. More detailed information on the various features are available in the later sections.

The main trace view provides all recorded information on a vertical time line. This view is complemented by over 20 additional views providing high level overviews or focused views from different perspectives. The task scheduling is presented using color coded rectangles, where the color helps to identify the actor. By actor we mean a thread of execution - a task or interrupt. The actor colors are taken from a color scheme based on the actor priority. The default color scheme is the natural light spectrum, going from red (high priority) to blue (low priority), and with lighter shades blue for the lowest priority tasks and finally white for the idle task. The exact colors used depends on the number of actors in the trace.

The execution of tasks and interrupts can be visualized using three visualization modes, which you quickly can switch between to get best clarity for each situation. The modes are:

Kernel events such as calls to kernel services and user events are displayed as text labels on the right side of the scheduling trace. The labels are color coded depending on the status of the operation.

Clicking on an actor, system call or user event shows information in the Actor Information display, as illustrated above. This is a tree structure containing a lot of information, both general statistics of the actor and information about the selected instance. Some nodes can be double-clicked to navigate the trace view accordingly. You can follow the execution of the selected actor by pressing "Previous Instance" and "Next Instance" buttons below the Actor Information display.

Double-clicking on an actor, system call or user event opens a focused view showing a list of all related events, the object history view. This shows all system calls on a selected kernel object, i.e., a message queue, semaphore or mutex. A similar view is available for actors, showing all instances of the actor. Double-clicking on a user event shows the user event log, showing a list of all user events recorded.

Additional Views

Below follows a few examples of other views and dialogs which complements the trace view. There are over 20 views in total. All views are connected to the main trace window or another relevant view, which allows you to switch perspectives without losing focus.

To locate a particular task or interrupt in the trace view, open the Finder (Shortcut Key "Ctrl-F"). In the Finder dialog, you can also specify filters on timing properties such as response time, in order to quickly find the extreme cases.

The Object History View allows you to track a particular kernel object. You can follow a buffered message from send to receive, or a semaphore signal. You can see the ownership of mutexes. Double clicking in this view highlights the event in the main trace view.

The CPU Load Graph displays the CPU usage per task and interrupt over time. This can be zoomed separately of the main trace view, and can also be used to navigate the trace view by double clicking in the graph.

Example Uses

Some of the possible uses of Tracealyzer are, to:

Tracealyzer gives you better possibilities for finding out what is really going on in your system. For instance, if a task never gets to execute you can see why this is the case. Perhaps higher priority tasks are using all CPU time, so that the scheduler never lets the lower priority tasks execute. This is revealed by the CPU Load Graph.

Another reason for a "missing task" might be that the triggering message/semaphore signal is never sent, or sent to the wrong destination, or that the timeout is too short so that the send fails. The Communication Flow graph shows an overview diagram of all IPC events, which can reveal if a message is sent to the wrong message queue. The Finder can be used to find individual IPC events, e.g., a message sent from a particular task to a particular message queue, and has a filter which can be used to quickly find operations which failed due to timeouts.

If the system performance is too slow, or if you simply want to measure the performance, you can study the statistics report for a performance overview - task response times and average CPU usage. If the system performance is not satisfying, the Actor Instance graph reveals the instances with high response times. The trace view shows you the reason - it could be that the actual execution time of the task is too long, but could also be caused by interference from other tasks or interrupts. This allows you to focus your optimization efforts on the true cause, which can save you a lot of time.


Tracealyzer requires:

For detailed instructions on how to use the Trace Exporter, please refer to the comments in the source code files and the readme.txt file. The directory of the Trace Exporter code can be easily found through the Help menu option Trace Exporter for On Time RTOS-32. This directory includes the readme.txt file, which also can be in the application Start Menu folder.

Copyright Percepio AB 2014, all rights reserved.