Getting Started With Tracealyzer for Linux
1) Setting up LTTng
First of all, you will need to install LTTng on your target device. The steps to install the packages will vary depending on your setup, and the instructions are outside the scope of this document. Please contact us if you need assistance with this part.
Make sure you have a user that is part of the tracing group, unless you are tracing as root.
You will need to ensure that lttng-sessiond is running as root when you want to trace.
If you build from source, double check the output from building the lttng-modules package as it may look like it succeeds but still fail.
When you have installed LTTng and made sure your user is part of the tracing group and lttng-sessiond is running as root, you should be able to create a simple trace using something like this:
lttng create
lttng enable-event -k sched_*
lttng add-context -k -t pid
lttng add-context -k -t ppid
lttng start
# Run some test command
echo “Hello world” | wc
lttng stop
lttng destroy
This should give you a trace in ~/lttng-traces if everything is setup correctly. You should be able to use the babeltrace tool to verify that it contains various scheduling events.
2) What this version of Tracealyzer supports from an LTTng trace
This version of Tracealyzer handles the following events.
- Scheduling events (sched_* in the Kernel domain)
- Signal events (signal_* in the Kernel domain)
- IRQ and SoftIRQ events (irq_* in the Kernel domain)
- Syscalls (
--syscall -a in the kernel domain)
Any other events will be presented as raw events using traditional Tracealyzer event labels. They will also be accessible from the event log.
For the best experience, we recommend enabling the pid and ppid context fields on the kernel channel containing the scheduling events. When tracing userspace events (UST), we recommend to also enable the vtid context. This will help Tracealyzer present events correctly under some special circumstances, in particular involving migrations in multicore systems.
3) Using Tracealyzer
If you are new to Tracealyzer, we recommend visiting the Getting Started with Tracealyzer page at percepio.com. You will also receive a series of onboarding emails over the next few weeks. The rest of this document will focus on the particulars of the new Linux support.
The first thing to do is to make sure Raw Events are enabled in settings, on the Performance page. For installations on new systems, this will default to enabled.
The next step is to acquire an LTTng trace from a target system. If you already have traces, you should be able to drag these into Tracealyzer, or open them from the File menu. You can also connect to your target using SSH from Tracealyzer.
Target Library
You can setup target profiles for easy access. Use the New button to create a new profile. From there you can setup the address, username and password or private key file. Once a profile has been created, the Connect button will allow you to connect to your target. A tab will open to the right with three main parts. A trace session control panel, a command log and a command details box. Click the Start New Session button to start a new session. By default, a TzTrace session has been defined which is preconfigured to include scheduling, syscalls, signals and UST events. Sessions and channels can be edited through the new Sessions And Channels Editor tool.
Quick Connect options
An example of a connection string would be
host=raspberrypi;user=pi;pass=raspberry;
The complete set of Quick Connect options are
- host – Hostname or address of the target
- port – SSH port (defaults to 22)
- myhost – Address to the host machine as seen from the target. This is can usually be left unspecified, but can be explicitly specified if the target is unable to lookup the hostname of the host computer.
- user – The user to connect as
- pass – The password of the user
- keyFile – Key file to be used with public key authentication. This is highly experimental. Note the uppercase F.
- cmdname – The name of the lttng command. Defaults to lttng, but can be overridden in case a prefix is used or similar.
Once a connection string has been entered, you should be able to connect using the button titled “Connect”. This requires your host machine to have the lttng-relayd tool installed, commonly installed from the lttng-tools package of your distribution. If you are running Tracealyzer on a Windows host, you need to use Windows Services For Linux and make sure lttng-relayd is installed and accessible in that environment.
View Features
Some new Linux-related features you may be interested in.
The Signals And Syscalls explorer is a tool that can help you analyze how a process interacts with the kernel through syscalls. For example, it will let you track which files are being accessed. It will try to help you track file descriptors (there are some limitations) and it will help you identify failing calls. It can show you a list of syscalls per thread, process or process tree. In addition, it can show you how signals are generated and delivered.
The syscalls plot will let you plot the duration of syscalls. It support two modes, execution time and response time. The former will only account for time when the thread is active while the former will include time when the thread is switched out (i.e. blocked).
The communication flow view has three new Linux-specific modes that can be accessed through its View menu. It supports a process tree mode, a signals mode and a file descriptors mode. The latter can help you track how processes are communicating over pipes etc.
The main trace view has a new experimental field called Actor Tree. This is under heavy development, but the intention is to give you a better view of how processes and threads are spawned.
If you select a raw event in the main trace view or in the event log and right click it, you can add numeric fields to a quick plot. This can be used with any event type, including your own user space tracing events.
4) Known Issues
The LTTng Machine Interface (--mi) has very limited support for snapshots. In particular, it does not include the output path. For this reason, snapshot support is very limited. We are looking at workarounds.
Traces with uneven buffer coverage may confuse the tool. This is frequently the case with snapshot traces, and for this reason snapshot traces are not recommended. By uneven buffer coverage, we are referring to scenarios where different channels or event streams cover different time periods. If you have 10 minutes of scheduling data for one core, but only 10 seconds of scheduling data for another core, not necessarily aligned at the start or end of the trace, the combined scheduling will simply not make sense. The coverage can be analyzed through the interval set “External Data / Intervals / Buffer Coverage / (all)”.
We recommend using the latest version of LTTng on both target and host. In particular, we have seen problems where using an older version (2.10) of lttng-relayd on host side causes traces to be created without metadata, rending them unusable.
You can report issues with Tracealyzer, and send general feedback, using the builtin report function in Tracealyzer – it’s located in the far right end of the menu bar.