This section lists a number of native tools available on Windows, Linux, and Oracle Solaris operating systems that are useful for troubleshooting or monitoring purposes. A brief description is provided for each tool. For further details, refer to the operating system documentation (or man pages for Oracle Solaris and Linux operating systems).
The format of log files and output from command-line utilities depends on the release. For example, if you develop a script that relies on the format of the fatal error log, then the same script may not work if the format of the log file changes in a future release.
You can also search for Windows-specific debug support on MSDN developer network.
The following sections describe troubleshooting techniques and improvements to a few native operating system tools.
Oracle Solaris 10 operating system includes the DTrace tool, which allows dynamic tracing of the operating system kernel and user-level programs. This tool supports scripting at system-call entry and exit, at user-mode function entry and exit, and at many other probe points. The scripts are written in the D programming language, which is a C-like language with safe pointer semantics. These scripts can help you in troubleshooting problems or solving performance issues.
The dtrace command is a generic front end to the DTrace tool. This command provides a simple interface to invoke the D language, to retrieve buffered trace data, and to access a set of basic routines to format and print traced data.
You can write your own customized DTrace scripts, using the D language, or download and use one or more of the many scripts that are already available on various sites.
The probes are delivered and instrumented by kernel modules called providers. The types of tracing offered by the probe providers include user instruction tracing, function boundary tracing, kernel lock instrumentation, profile interrupt, system call tracing, and many more. If you write your own scripts, you use the D language to enable the probes; this language also allows conditional tracing and output formatting.
You can use the dtrace -l command to explore the set of providers and probes that are available on your Oracle Solaris operating system.
The DTraceToolkit is a collection of useful documented scripts developed by the Open Oracle Solaris DTrace community. For more information about the DTraceToolkit, see the DTraceToolkit.
For more information about dynamic tracing in general, see the Solaris Dynamic Tracing Guide.
The Java HotSpot VM contains two built-in probe providers: hotspot and hotspot_jni. These providers deliver probes that can be used to monitor the internal state and activities of the VM, as well as the Java application that is running.
The JVM probe providers can be categorized as follows:
VM lifecycle: VM initialization begin and end, and VM shutdown
Thread lifecycle: thread start and stop, thread name, thread ID, and so on
Class-loading: Java class loading and unloading
Garbage collection: start and stop of garbage collection, systemwide or by memory pool
Method compilation: method compilation begin and end, and method loading and unloading
Monitor probes: wait events, notification events, contended monitor entry and exit
Application tracking: method entry and return, allocation of a Java object
In order to call from native code to Java code, the native code must make a call through the JNI interface. The hotspot_jni provider manages DTrace probes at the entry point and return point for each of the methods that the JNI interface provides for invoking Java code and examining the state of the VM.
At probe points, you can print the stack trace of the current thread using the ustack built-in function. This function prints Java method names in addition to C/C++ native function names. Example 2-38 is a simple D script that prints a full stack trace whenever a thread calls the read system call.
Example 2-38 Stack Trace at Probe Points with DTrace
#!/usr/sbin/dtrace -s
syscall::read:entry
/pid == $1 && tid == 1/ {
ustack(50, 0x2000);
}
The script in Example 2-38 is stored in a file named read.d and is run by specifying the PID of the Java process that is traced as shown in Example 2-39.
If your Java application generated a lot of I/O or had some unexpected latency, then the DTrace tool and its ustack() action can help you diagnose the problem.
The pmap utility was improved in Oracle Solaris 10 operating system to print stack segments with the text [stack]. This text helps you to locate the stack easily.
Example 2-40 shows the stack trace with improved pmap tool.
Example 2-40 Stack Trace with Improved pmap Tool
19846: /net/myserver/export1/user/j2sdk6/bin/java -Djava.endorsed.d 00010000 72K r-x-- /export/disk09/jdk/6/rc/b63/binaries/solsparc/bin/java 00030000 16K rwx-- /export/disk09/jdk/6/rc/b63/binaries/solsparc/bin/java 00034000 32544K rwx-- [ heap ] D1378000 32K rwx-R [ stack tid=44 ] D1478000 32K rwx-R [ stack tid=43 ] D1578000 32K rwx-R [ stack tid=42 ] D1678000 32K rwx-R [ stack tid=41 ] D1778000 32K rwx-R [ stack tid=40 ] D1878000 32K rwx-R [ stack tid=39 ] D1974000 48K rwx-R [ stack tid=38 ] D1A78000 32K rwx-R [ stack tid=37 ] D1B78000 32K rwx-R [ stack tid=36 ] [.. more lines removed here to reduce output ..] FF370000 8K r-x-- /usr/lib/libsched.so.1 FF380000 8K r-x-- /platform/sun4u-us3/lib/libc_psr.so.1 FF390000 16K r-x-- /lib/libthread.so.1 FF3A4000 8K rwx-- /lib/libthread.so.1 FF3B0000 8K r-x-- /lib/libdl.so.1 FF3C0000 168K r-x-- /lib/ld.so.1 FF3F8000 8K rwx-- /lib/ld.so.1 FF3FA000 8K rwx-- /lib/ld.so.1 FFB80000 24K ----- [ anon ] FFBF0000 64K rwx-- [ stack ] total 167224K
Prior to Oracle Solaris 10 operating system, the pstack utility did not support Java. It printed hexadecimal addresses for both interpreted and compiled Java methods.
Starting with Oracle Solaris 10 operating system, the pstack command-line tool prints mixed mode stack traces (Java and C/C++ frames) from a core file or a live process. The tool prints Java method names for interpreted, compiled, and inlined Java methods.
