Kernel markers were a static kernel instrumentation support mechanism for Linux kernel source code, allowing special tools such as LTTngTracepoints and Markers , LTTng's Tracing Wiki or SystemTapUsing Markers, SystemTap Wiki to trace information exposed by these probe points. Kernel markers were declared in the kernel code by one-liners of the form:

trace_mark(name, format_string, ...);

Where name is the marker's unique name, and format_string describes the remaining arguments' types.

A marker can be on or off depending on whether a probe is connected to it or not. Code which wants to hook into a trace point first calls:

int marker_probe_register(const char *name, const char *format_string, marker_probe_func *probe, void *pdata);

to register its probe callback with the marker point (pdata is a private data value that the code wants to pass to the probe). Later, the probe is turned on and off using:

int marker_arm(const char *name); int marker_disarm(const char *name);

Using markers has a negligible overhead thanks in part to Immediate Values,Mathieu Desnoyers, [patch 08/10] Immediate Value - Documentation another support mechanism that embeds switches in the code that can be dynamically turned on and off, without using a memory reference and thus saving cache lines.Mathieu Desnoyers, [patch 7/8] Immediate Values - Documentation

The initial motivation to create this static instrumentation infrastructure was the large performance overhead induced by the predating dynamic instrumentation mechanism Kprobe mechanism, which depends on breakpoints. Static instrumentation can also more easily survive source code changes because the markers are in the source code.

Kernel Markers consisted essentially of a C preprocessing macro which added, in the instrumented function, a branch over a function call. By doing so, neither the stack setup nor the function call are executed when instrumentation is not enabled. By identifying the branch executing stack setup and function call as unlikely (using the gcc built-in expect()), a hint is given to the compiler to position the tracing instructions away from cache lines involved in standard kernel execution.Mathieu Desnoyers, Low-Impact Operating System Tracing, Ph. D. Dissertation, Département de génie informatique et génie logiciel, École Polytechnique de Montréal, Université de Montréal, December 2009

Two Kernel Markers drawbacks were identified which led to its replacement by Tracepoints: Type verification was limited to scalar types because the API is based on format strings. This could be problematic if pointers must be dereferenced by the tracer code. The Markers "hide" the instrumentation in the source code, keeping no global registry of the instrumentation. This makes namespace conventions and tracking of instrumentation modification difficult unless the whole kernel tree is monitored.

A patch-set implementing them was merged into version 2.6.24,Linux 2.6.24 Changelog, Linux Kernel Newbies which was released on January 24, 2008. To address issues regarding kernel markers, Mathieu Desnoyers, their original author, implemented a simpler and more type-safe version of static probe points named Tracepoints. A patch-set implementing Tracepoints was merged into version 2.6.28,Linux 2.6.28 Changelog, Linux Kernel Newbies which was released on December 25, 2008. Starting then, kernel markers were slowly removed from kernel sources and eventually fully removed in Linux kernel 2.6.32,Linux 2.6.32 Changelog, Linux Kernel NewbiesChristoph Hellwig, fc537766 tracing: Remove markers, Torvalds' Linux git tree which was released on December 3, 2009.

See also Kernel debugger

References

External links Jonathan Corbet, Kernel markers, LWN.net, 2007 Mathieu Desnoyers, , Linux kernel documentation, 2008 Jonathan Corbet, Tracing: no shortage of options, LWN.net, 2008