This page describes how to enable debug logging in the C SDK and debug application crashes and potential bugs. Basic C SDK logging can be enabled using environment variables, and may also be controlled from the SDK API itself.
The default logger can also be enabled programmatically via the LCB_CNTL_CONLOGGER_LEVEL setting or the console_log_level option in the connection string.
If you wish to control where the logs are output or what information is displayed (beyond basic verbosity filtering) you can install a logging callback via the LCB_CNTL_LOGGER setting.
If you wish to change the actual destination of the logs (for example, log to a file) you may set the console_log_file directive in the connection string.
The output format is subject to change. It is intended for human consumption and is not designed to be parseable.
The current thread/process identifier. On Linux this is also the process ID for single-threaded programs, further helping distinguish between multiple forks of an application.
The following section applies to Linux, Mac OS X, and other Unix-like systems.
Stack traces and core dumps may be required if your application is hanging or crashing. In this case knowing in which location the library (or application) is misbehaving becomes useful.
To get a stack trace you need to have GDB installed on your machine and be familiar with its use. If you cannot run your application from within GDB, you can attach GDB to an existing process.
To generate a useful stack trace, you need to run your application with a debug build of the Couchbase C library because the released binaries are optimized. During optimization the compiler might rearrange the code, which can result in misleading source file and line number information in the stack trace.
Ensure debugging symbols for the C client are installed. If you are using a binary distribution, use your package manager to install the libcouchbase-dbg or libcouchbase-debuginfo package. If you are using a version of the C library built from source, ensure that ‑‑enable‑debug was passed to the configure script.
You should also build your application with debug symbols, though not strictly required.
If your application crashes within the C library, examine the stack trace information. The C library is asynchronous in its core, so almost everything happens from within the lcb_wait() function. Tracing actual state changes and events within the library is only possible by using logging.
Reading symbols for shared libraries .
Valgrind can help you detect issues in your application (or the library itself) related to accessing invalid (already freed or unallocated) memory or uninitialized memory regions. This information is useful when debugging application behavior before a crash because crashes are often caused by entering a normally impossible state induced by a previous write or read to an invalid memory location.
You can install Valgrind through your package manager.
Valgrind will slow down execution time significantly, often by a factor of 20x. To lessen this impact on an application that performs lots of load, temporarily reduce the amount of CPU it uses.
Valgrind typically does not work well with dynamic language interpreters such as Python, Ruby, or Perl due to the way these languages optimize memory allocation. To effectively use Valgrind with these languages, you often need to use a special debug build of the interpreter. To find out more about using Valgrind with interpreters, refer to the documentation for the language you are using.
The following example shows how to run an application with Valgrind. In the example, the operation appears to time out, but the real cause of the problem is uninitialized memory access that causes the client to incorrectly read the response from the server.
==29887== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
Core dumps might be generated when an application crashes (though on many systems core file generation is off by default). The core dump contains a memory dump of your application that can be inspected on a different system. This lets you inspect the state of the library to determine what the error was at the time of the crash.
Core dumps can only be analyzed if the binaries that generated them are accessible and have debugging symbols. Binaries in this sense includes the client library, the application, and any other shared libraries loaded by the application. This means that the platform (and distribution) that generated the core dump must be available and loadable (or at least reproducible) when analyzing the core dump.
A core file contains the memory contents of your application. Anyone who can read your core file can access potentially sensitive data that your application was processing at the time of the crash. If possible, have your application operate on sample data.
Invoke ulimit -c unlimited in the same shell that your application will run (or a parent thereof) prior to the invocation of that application. The ulimit command instructs the kernel to generate core dump files for all subsequent processes that terminate abnormally and are executed within the current shell or any child shell.
Ensure you are either using a binary installation of the client library (which was installed from an official Couchbase repository) or are using a source build with debug symbols enabled.
When the application crashes, a core file is created. The exact location and name of the file is dependent on the specific kernel configuration. For more information about where core files are located, see t his Stack Overflow discussion on how core location may be configured.
Compress the core file using gzip or a similar utility before filing an issue or sending this to Couchbase support. Compressing the file makes the upload and download times quicker for analyzing the core file.
Disable core dumps once completed. Core dumps are disabled by default on most Linux distributions because they may potentially write sensitive information to disk and many core dump files can quickly fill up a file system.