OS4X create log dumps

What are OS4X dumps?

OS4X log dumps are binary files containing deep information about the system, like SQL statements, their results, network traffic, OFTP status information etc. Every OS4X program generates OS4X log messages, which are organized by the daemon "os4xdebugd" (OS4X debug daemon).

Requirements

In order to get an OS4X debug dump file, the daemon must be running. Verify this by checking the daemon's running state. The daemon must be running at least for three seconds in order to gather all needed information from running daemons.

Normal behaviour

All OS4X programs send their log information to the OS4X debug daemon. The daemon uses a ring buffer to save the logged messages, so with a limited amount of entries, the daemon will overwrite old entries when the buffer is full. The daemon process size depends on the amount of entries saved in memory. The more entries are configured, the bigger the process grows. A normal state should be about 2-5 MB in memory.

Generating a dump

When a critical situation occurs, the daemon should be initialized to dump its content to an OS4X debug dump file. This can be in two ways:

via web GUI
via command line

Dumping log via web GUI

1. Select "Programs" -> "Daemons" in the administrative web interface:

2.1 In the new panel, click on the button "Dump log", which is activated when the daemon is running:

If the daemon is not running, it makes no sense to start it and dump for an ancient event: the event must occur when the daemon "os4xdebugd" is running.

2.2 If the previous step fails, it may help to dump a log to the directory "/tmp", since the OS4X's configurable temporary directory may be not available. Use the button "Dump to /tmp" in order to accomplish this task.

Dumping log via command line

Dumping log via command line is quite easy: call the daemon binary with the parameter "-d":

root@os4xbox:~# /opt/os4x/bin/os4xdebugd -d
initiating dump...OK

Alternatively, you can send the daemon the signal 3 (SIGQUIT) in order to write an actual debug dump to OS4X's temporary directory.

If the previous step fails, it may help to dump a log to the directory "/tmp", since the OS4X's configurable temporary directory may be not available. Use the parameter "-D":

root@os4xbox:~# /opt/os4x/bin/os4xdebugd -D
initiating dump to /tmp...OK

Retrieving the result file

The created file (either in OS4X's configurable temporary directory or in "/tmp") will be in the named form of:

os4x-logfile-<YYYYmmdd>-<HHmmss>.log

i.e.:

os4x-logfile-20120926-182223.log

The file can be compressed very good, a compression ratio of 90% is nearly in all times possible. Send this generated file to the OS4X support in order to get more information.

Customizing

The debug daemon behaviour can be easily changed with some parameters, which can handle most of the requirements requested.

Event script

The configurable event script "Debug daemon log script" can be used for post-processing on the generated file. The event script itself gets the absolute filename as only parameter, so you can i.e.

compress the file
mail it to a recipient for support handling
archive the file / delete it after successful handling
...

Configuration

The TCP/IP listening port of the debug daemon is configurable at "Configuration" -> "TCP/IP" -> "TCP/IP port of OS4X debug daemon".
The OS4X debug daemon supports a "continous debug write" mode which writes a logfile (and executes the event script) every time the ring buffer is full. Use with care, enable only when requested. This feature could fill up your filesystem quite fast: "Configuration" -> "Logging" -> "Enable continous write of OS4X debug daemon output?".
The hidden, via web GUI not editable database configuration value "os4xdebugd_max_bufferlength" defines the amount of memory entries of the ring buffer. You can raise the value in order to get bigger timeframes for analysis. The default value is 10000.