Porting Central
Skip to main content

Porting Central

Application Factory - Porting: AS/400 Debug Heap - Version 0.856

AS/400 Debug Heap - Version 0.856

Many of the Unix or Windows applications being ported to the AS/400 today have memory leaks. These (perhaps small) leaks are simply unacceptable when trying to get the applications to run for days, weeks or months without restart on a system that provides more mission critical services.

In order to help debug these problems, it would be very useful to have a heap tool that allowed tracing, dumping, and other memory check operations.

Questions, send mail: mailto:rchthrds@us.ibm.com

Download the source files.

Note: Links like this point to more information and open in a new window.

TOC

Expectation

Its the expectation of this document that you're familiar with ILE concepts relating to memory management, activation groups, and application development.

In addition, understanding environment variables APIs, creating and changing of IFS files, and various Windows or Unix type APIs will be a benefit.

Description

The default behavior of the Debug Heap is to trace each memory allocation and deallocation made by an AS/400 program to an IFS or spool file. When the activation group (application) terminates, the heap contents will be dumped. You can examine the allocation trace or the dump file to see which allocations were made or which allocations still exist in the heap at the time of termination.

You can dump the heap or the call stack of your program manually at any time by calling one of the dump functions. A trace of the call stack at the time of each allocation/deallocation can also be turned on so that you know exactly which code makes the memory allocations and deallocations (the 'Stack-on' option).

When the heap detects storage corruption or invalid operations on the storage (like freeing the storage twice, over runs off of the end of the storage or the beginning of the storage) or other problems, the heap will throw an escape message (CPF9898) and trace the call stack at the time of the failure. You can use this call stack to identify problem areas.

Capabilities of the Debug Heap

  • Heap storage functions and operators malloc(), free(), new, new[], delete, delete[] are remapped to debug/tracing versions.
  • Allocation/Deallocation Tracing. With optional memory initialization (to 'N' which stands for New) on allocation and memory reset (to 'D' which stands for Deleted) when deallocation.
  • Various manipulation and validation of the storage contents that can help cause exceptions when the storage is accessed. This helps provide early detection of storage problems.
    Examples are: An eyecatcher that is validated before and after the storage allocation, filling the storage with invalid pointers (if requested), a 'fence' at the beginning and end of the storage. These things detect bounds overruns by your application, and use of uninitialized pointers in the storage.
  • The ability to use a single OS/400 object for each allocation. This immediately causes MCH3402 exceptions if you attempt to access the storage after it has been free'd.
  • The ability to trace the current function call stack at the time of the memory allocations/deallocations or at the time that a user specifies
  • The ability to dump the entire heap for visual examination (dump can occur manually by the application or automatically at application end)
  • The ability to work with the default MI (ILE) heap or a stand alone heap in the applications current activation group
  • The ability to format and dump arbitrary storage easily to an IFS or spool file
  • Flexible options setting via INI files or an environment variable. Optionally disable caching of these options for very dynamic option settings depending on the user's requirements.

Using the Debug Heap

  1. Change your modules, including the header file Qp0wHeap.H in every module that you want to trace allocations in. Remember that allocations made in a module that included Qp0wHeap.H cannot be free'd by a module that didn't include Qp0wHeap.H or the reverse. (The heaps are different).
  2. Recompile your modules and program.
    • You can define the preprocessor value QP0W_NO_DEBUG_HEAP to remove all definitions of the Qp0wHeap.H header file. For example if you're doing a release build, defining QP0W_NO_DEBUG_HEAP effectively makes the Qp0wHeap.H header file a no-op.
    • The Qp0wHeap.H file defines the preprocessor value QP0W_DEBUG_HEAP_ACTIVE that you can use to insert optional debug code to dump additional data or take some other action when compiling for a Debug Heap application.
    • You can define the preprocessor value QP0W_NO_DEBUG_HEAP_REMAPPING to turn off the automatic remapping of malloc/free and the C++ global allocation operators. Using this option then requires that you explicitly code to the Qp0wDBH* allocation functions if you want to use them. (This is very useful if you only want the Qp0wDBHDumpHeap() or the Qp0wDBHDumpStack() functions. Pay attention to Debug Heap option 'Mi-defaultheap-on' if you want to use Qp0wDBHDumpHeap() alone though. Using the M option will allow the debug heap to dump the default MI heap, which is likely where your data will be.
  3. Bind your program to the QP0WHEAP service program. The location of the service program depends on the steps you've taken in the Installing the Debug Heap section.
  4. Set up any non default options you want. Its recommended that you skip this step and use the default options the first time you use the debug heap.
    To set non default options:
    • Create a INI file in IFS that contains configuration options for the debug heap. The INI files that are referenced (in order) are Qp0wHeap.ini in the current directory, home directory, and then the root directory. I.e. "./Qp0wHeap.ini", "/home/<USER>/Qp0wHeap.ini", and finally, "/Qp0wHeap.ini". If any of these INI files are found, the others are not used.
      An example ini file (Qp0wHeap.ini) can be found in the source file zip.
      If the environment variable QP0W_DEBUG_HEAP environment variable is set, no INI files are used.
    • Set the environment variable QP0W_DEBUG_HEAP if you desire any non-default options. Use the commands ADDENVVAR, CHGENVVAR or WRKENVVAR to set the environment variable. You can use WRKENVVAR LEVEL(*SYS) on some releases to set a system wide environment variable. It may ease setting options. If the environment variable QP0W_DEBUG_HEAP environment variable is set, no INI files are used.
  5. Run your program or application.
  6. By default, for every activation group that contains code which uses the debug heap, there are 2 output files created. See Output files for information about which output files are created when.
    • The first one is the allocation trace file. It is created immedately (at the time of the first storage allocation), and remains open until the activation group ends.
      This file contains traces for all storage allocations and deallocations (and optionally, the current call stack at the time of the memory requests - See option 'Stack-on' to turn on the dump of the call stack at the time of the memory request, and option 'Allocations-off' to disable the tracing of the memory allocations completely).
    • The second file is the storage dump file created at the time the activation group terminates and is used to dump the contents of the heap at that time. (See option 'Dump-off' to disable the automatic dump of the heap, or use the function Qp0wDBHDumpHeap() so that the application can manually perform a heap dump at some arbitrary point during the run of your application).
    • Note: Code that runs in the default activation group, will create only the allocation trace file.
      In the default activation group, that file file will not be closed until the process terminates. The automatic heap contents dump file will not be created, because the default activation group does not allow activation group termination hooks. Its recommended that the application use the Qp0wDBHDumpHeap() function manually, or temporarily use a different activation group for debug heap testing.
  7. When the program or application ends (For some servers that never end, you may have to end the application manually), examine the contents of the heap storage dump file that gets created. See Output files for information about which output files are created when.
    Note:In the heap storage dump file, all entries contain an Allocation ID# that represents the allocation numerically that created the storage and corresponds to the number in an entry in the stack trace. Included also is the size of the user storage that was requested. Use the allocation ID and the size to track down which code allocated the storage.
    To see what kind of leaks you might have, refer back to the allocation trace file to find out where that storage was allocated. (The 'Stack-on' option helps track this down much more easily, but dumps LOTS of trace and stack data). See Output files for information about which output files are created when.

Functions provided by the Debug Heap

Qp0wDBHDumpHeap()
Performs a dump of all of the contents of the current Debug Heap (whether default MI heap or a uniquely created debug specific heap) to a file.
Qp0wDBHDumpStack()
Dumps the current call stack of the calling thread to a file. (Creates a unique file containing just this dump information).
Qp0wDBHDumpStorage()
Dumps the storage specified to a file. If the heap storage dump file has already been used, continues the storage dump in the same file.
Qp0wDBHMalloc(), Qp0wDBHFree(), Qp0wDBHRealloc(), Qp0wDBHCalloc()
Replacements for malloc(), free(), realloc(), and calloc() that allocate/deallocate storage from the Debug Heap, possibly cutting tracepoints indicating the storage and who the caller is. If the uniquely created debug specific heap is used, all bytes allocated are set to 'N', all bytes free'd are set to 'D'.
operator new, operator new[], operator delete, operator delete[]
Replacements for the C++ global operators of the same name. These operators are simply inline calls to the operators Qp0wDBHGlobalNew(), Qp0wDBHGlobalVecNew(), Qp0wDBHGlobalDelete(), Qp0wDBHGlobalVecDelete(). These operators are functionally the same as the Qp0wDBHMalloc and Qp0wDBHFree in their trace behavior. If the uniquely created debug specific heap is used, all bytes allocated are set to 'N', all bytes free'd are set to 'D'.
Qp0wDBHStrdup()
Replacement for the strdup() functions that uses malloc to allocate space for and create a copy of the null terminated string specified.

Available Options

Remember... By default, you shouldn't need any of these options. The first one you'll probably want to add after you've discovered a leak using all the default options is option 'Stack-on' to show the call stack of each memory allocation as it occurs. Then, you can go back and correlate the memory allocations with the point in the code at which the problem occurred (using the call stack) output.

Another useful option is the 'Object-per-allocation-on' option which indicates that an OS/400 Object should be used for each allocation. This option causes the application to immediately take MCH3402 exception (pointer to destroyed object failure) when it accesses storage that was already free'd. (Unfortunately, the entire heap cannot be dumped at activation group end, or by using Qp0wDBHDumpHeap() if you use option 'Object-per-allocation-on').

Options to the Debug Heap are passed via an environment variable or INI files stored in IFS

  • INI files that are referenced are Qp0wHeap.ini in the current directory, home directory, and then the root directory. If any one of these files is found, the other files are not accessed.

    I recommend you map a drive to your AS/400 system and use your editor to create/edit the files. If the file is tagged with the correct CCSID, the file can contain ASCII data. Use WRKLNK, option 8 to display attributes of your file and ensure that the 'Code page' field is set correctly depending one what data is in it (ASCII is typically 819 or 437, ENGLISH EBCDIC 37).

    If the QP0W_DEBUG_HEAP environment variable is set, it is used first, and no INI files are used. Not setting the QP0W_DEBUG_HEAP environment variable and having no INI files found, would indicate that the default behavior be used for all options.

    Again, the INI file order used by the debug heap are: "./Qp0wHeap.ini", "/home/<USER>/Qp0wHeap.ini", and finally, "/Qp0wHeap.ini". If any of these INI files are found, the others are not used.

    For example: If I want to turn off the automatic dumping of the heap contents and turn on the tracing of the stack information for memory allocation operations, I would create /home/kulack/Qp0wHeap.ini file with the following contents (comments are not required): 
       # Turn off dump at application end
   Dump-off
   # Turn on stack trace at each allocation/deallocation request
   Stack-on
  • The main options environment variable is QP0W_DEBUG_HEAP.

    If the QP0W_DEBUG_HEAP environment variable is set, it is used first, and no INI files are used.

    Unlike the INI file, the environment variable uses only uppercase characters as options flags. Any lowercase and other characters are ignored. Not setting the QP0W_DEBUG_HEAP environment variable and having no INI files found, would indicate that the default behavior be used for all options.

    For example: If I want to turn off the automatic dumping of the heap contents and turn on the tracing of the stack information for memory allocation operations, I would set the environment variable to "QP0W_DEBUG_HEAP=DS" or I could use "QP0W_DEBUG_HEAP=Dump-off Stack-on" for easier readability, and to match the INI file settings.

See the Configuration Options Table.

Installing the Debug Heap

If you want to install the Debug Heap, use the following steps:

If you haven't downloaded the source files yet, do it now.

  • Create a save file CRTSAVF LIB/QP0WHEAP on your target system.
  • Clear the save file using the CLRSAVF LIB/QP0WHEAP command.
  • Use your browser to save the following files to your hard disk.
    • qp0wheap.savf420 (V4R2) - Savefile (Binary file) can be restored on all subsequent releases
    • Qp0wHeap.H - Debug Heap header file (Ascii file). Use this file if you're only compiling applications to use the debug heap.
    • Using FTP, put the files that you jsut downloaded onto the AS/400. For the save files, do a binary mode FTP, and put the data into the save file that you just created above: 
      put qp0wheap.savfxxx lib/qp0wheap.qp0wheap
    • Restore the service program qp0wheap from the savefile, using the command RSTOBJ OBJ(QP0WHEAP) SAVLIB(QSYS) DEV(*SAVF) SAVF(LIB/QP0WHEAP) MBROPT(*ALL) ALWOBJDIF(*ALL) RSTLIB(LIB).
    • Copy or use ASCII mode FTP to send the Qp0wHeap.H header file to a location that you can use to compile your application.

Debug Heap Source Code

If you'd like to attempt to make changes or fix bugs in the debug heap, please feed them back to us. Here's the source, it was compiled with an internal C++ compiler, and you may have to make changes in order to get it to compile on your system. You should use VA C++ for OS/400 or the native C++ compiler to compile this code.

  • Qp0wHeap.C - C++ Source for the debug heap. (Ascii file)
  • Qp0wHeap.qsrvsrc - Service program export file. Describes data and procedure exports from Qp0wHeap service program (Qp0wHeap.C).
  • Qp0wSHeap.H - Internal header for the debug heap. (Ascii file)
  • Qp0wHeap.ini - Example ini file used to set debug heap options. (Ascii file)
  • There are some example programs that directly use the debug heap:
    • testheap.C - (Ascii file)
    • testheap2.C - (Ascii file)
    • testheap3.C - (Ascii file)
    • testheap4.C - (Ascii file)

Output files

Beginning with version 0.85 of the debug heap, much more emphasis has been placed on IFS as a better mechanism for output and configuration files.

By default, all output files are now created in IFS instead of in a spooled file. The environment variable QP0W_DEBUG_HEAP_NOIFS can be used to get back to the old spooled file behavior. The old behavior may be removed in a subsequent version.

Output files always go to the applications current directory, and are named depending on the type of output file.

Filename Description
Qp0wHeap.storage.%j.%a.%c.txt Dump of storage by the user or dump of the entire heap due to user request or application end.
Qp0wHeap.alloc.%j.%a.%c.txt The allocation trace that contains each allocation and free request. Additionally, contains integrated stack tracing if the 'Stack-on' option is used.
Qp0wHeap.stack.%j.%a.%c.txt Any call stack dumps requested explicitly by the user calling Qp0wDBHDumpStack().

The format specifiers in the above file names are:

  • %j - The fully qualified jobname using - instead of slashes.
  • %a - A unique-ifier to identify which instance of the debug heap in this job is being used. The number is the activation group mark and changes within a job as different activation groups load a version of the debug heap.
  • %c - An id number (counter) identifying the count of the number of files of this type that have been opened for this job. Incrememts from 0.

Restrictions

  • Storage allocated from the Debug Heap should not be free'd by the C/C++ runtime functions for dynamic storage allocation (i.e. don't mix debug heap storage with malloc/free/new and delete)
  • You must not change the options that control how storage is allocated on the fly. If options 'Mi-defaultheap-on' or 'Object-per-allocation-on' are used on the fly, the debug heap could try to storage in a way that it wasn't allocated in. Errors will result.
  • Code running in the default activation group will not automatically dump the heap contents when the activation group terminates. The application should directly use the Qp0wDBHDumpHeap() API to dump the heap at the point you feel is most useful, or you can change the application temporarily to run in a different activation group.
  • You might be able to run code prior to the heap initializing or after it gets destroyed. (Like with C++ Ctors/Dtors, atexit() functions, or activation group termination functions.) If you do, it'll likely blow up because the heap won't be present and completely initialized.
    Note: The first attempt to fix this occurred in version 0.856, it was easy and looks pretty promising. We simply skip debug heap cleanup when the activation group goes away.
  • If you implement your own global new and delete operators the Debug Heap operators may conflict with them. You might have to modify the Qp0wHeap.H header file to address this on your own.
  • Don't use CPA threads and the debug heap.

FAQ - Frequently Asked Debug Heap Questions

  1. What can I do to tell you how great this tool is?
    Send me a note at mailto:rchthrds@us.ibm.com. With a couple of simple pats on the back, I'll be more willing to add new functions... (Its that 'flys with honey' thing... 8-)
  2. So what's the easiest way to use this tool to find bugs?
    Follow these steps:
    • Compile all your source modules with debug heap header file Qp0wHeap.H included.
    • Run your application with the default options (i.e. Just run it, no INI files present or setup required). If you want to however, you can get extra exception conditions and information about bad heap operations by setting the INI files or QP0W_DEBUG_HEAP environment variable to include the 'Object-per-allocation-on' option.

      If you use the QP0W_DEBUG_HEAP environment variable, be sure that you either use ADDENVVAR or CHGENVVAR in the same job that you'll run your application in OR, use WRKENVVAR *SYS to WRKENVVAR LEVEL(*SYS) to add the 'Object-per-allocation-on' option.
    • When your application completes, is ended, or has an exception thrown to it from the heap look at the files created. There is a 'heap storage dump file' (named Qp0wHeap.storage.<full-jobname>.<AG>.<counter>.txt See Output files for information about which output files are created when) that is created when your program or activation group ends (if your code is NOT in the default activation group).
    • If there are still allocated blocks of storage in the heap storage dump file, you have memory leaks of some sort in your application. Continue with the following steps.
    • Signoff, signon and restart your job using the 'Stack-on' option.
    • Stack information is automatically dumped when you commit some sort of felony (like corrupting the heap or freeing some storage twice), but for simple leaks, you'll have to use the 'Stack-on' option and track down the offending code yourself.
    • When the application completes or is ended, look in the heap storage dump file again. Each allocated block shown in that storage dump file has an allocation-ID and a size field in its header. Correlate the allocation-ID and the size field in the allocated blocks shown in the heap storage dump file with the trace entries in the newly created allocation trace file.
    • The call stack shown that has the same allocation ID in the trace should be the code that allocated storage that was not free'd.
      For example, the following allocated block in the heap storage dump has an allocation ID number of 182.
      By looking back at the allocation trace file and searching for #182, we find the line that corresponds to the allocation request. The pointer in the stack trace is the pointer that was returned to the user. In a stack trace entry you will also have a stack trace, that stack trace is not shown here. 
        #182  Malloc  p   =80000000 00000000 e7f5af5f 1b0019c0  s=34
  etc...
  3. Is the debug heap thread safe?
    Yes, by default, the debug heap is native thread safe.
  4. Does this work for system state code?
    Yes, the debug heap works for system state code. If you're using it on an older release (prior to v4r2), debug heap may have to be recompiled and changed to *INHERIT state. Give me a call and I can do this for you. Its quite easy, but will require you to recompile and rebind your source if you want to use the debug heap.
  5. Does this work with CPA threads?
    NO.
  6. Why don't the allocation numbers increase monotonically? For example, the allocation trace file might look like the snippet below.
    The debug heap does allocations internally every time it creates a new output file or dumps the call stack. Sometimes these allocations are free'd immediately (stack dump), and sometimes they are not free'd until later. When dumping the heap, the heap will show a single line of information about these allocations, but will not actually dump the storage. Don't worry about them. 
    AS/400 Debug Heap file - Fred Kulack, version 0.85.
Output date: Tue Mar 14 14:40:11 2000

#2  Malloc   p=80000000 00000000 c82362b6 fe001020    s=100
#4  Realloc  pold=80000000 00000000 c82362b6 fe001020
             pnew=80000000 00000000 c82362b6 fe001180 s=200
    Free     p=80000000 00000000 c82362b6 fe001180
#5  Calloc   p=80000000 00000000 c82362b6 fe0012a0    s=100

To-Do List

The following recommendations came from various individuals. Most of these things I haven't gotten around to doing. If you code some of them, let me know, and I'll integrate the souce changes back here.

  • Add a __FILE__ and __LINE__ macro to record the file and line of the actual call of malloc and free. Keep it inside the actual buffer?
  • Keep better track of allocations with no free's and then dump them?
    Actually this is already done: This is what the default dumping of the heap contents at activation group termination does. Can we improve on it any?
  • Some perl tools might be useful to parse the trace spool file. We'll have to do a little testing and see what kind of info is needed. (Obviously, if the trace contains the call stack entry, we just look for any malloc that has no matching free, and dump out the call stack information of the malloc). Anything else?

Version List: changes and known bugs

  • Known Bugs
    • None. (Yeah, sure...)
  • Version 0.856
    • Changed cleanup processing so that it really doesn't cleanup. This allows static C++ objects to use the heap during application termination. The heap dumping and other activity still occurs during cleanup. To get the old cleanup behavior back, set the environment variable QP0W_DEBUG_DO_AG_CLEANUP to 'Y' or 'y'.
  • Version 0.855
    • Minor bug in message displayed in joblog that it didn't correctly reflect output directories updated as a result of the log-file-location option.
  • Version 0.854
    • Added the log-file-location option.
  • Version 0.853
    • The mi-default-heap option caused the storage returned not to be set to 'N'.
  • Version 0.852
    • The free() function doesn't accept a null pointer.
    • Added support for the 'Timestamps-on' option that allows the the trace output records to have timestamps on them.
    • Added a call to Qp0sEnableSignals() so that the heap could correctly run in 'legacy' jobs that don't access/use signals. The Qp0wGetJobID() function relies on that initialization.
  • Version 0.85 (Now defaults to IFS version)
    • Off by one bug in the allocation ID numbers in the tracepoints.
    • Default output files now reside in IFS. Set the environment variable QP0W_DEBUG_HEAP_NOIFS to disable the IFS behavior. This version contains a transition to IFS model where focus on IFS is emphasized, and focus on the QSYS.lib file system is de-emphasized. This is for spooled files versus IFS files, and for initialization versus options.
    • Removed any reference to CPA threads. The debug heap no longer works with CPA threads because of the IFS support that was added (references to _C_IFS_xxx functions for C file I/O.
    • Removed all the change flags because there were getting to be so many of them that didn't reflect reality, they were more painful than helpful.
    • Added support for creating the output files in IFS.
    • Added support for an INI file used so that the options can be specified there. If the INI file is found, then the environment variable is ignored.
    • The IFS filenames are more unique, and the INI files allow more descriptive options. The following two to-do items are taken care of.
      • We should add an eyecatcher parameter to the Qp0wDBHDumpHeap() or Qp0wDBHDumpStack() spool files so that the individual files created by user code can be distinguished from others. We should also add eyecatchers to the spool files created automatically by the Debug Heap.
      • Uppercase options characters that are invalid should be flagged as such instead of being ignored. This will help when adding new options in the future.
  • Version 0.81
    • Changed so that when dumping stack frames, we correctly tolerated the condition that there were more stack frames than 25 (default is now set to 64), also enabled the QP0W_DEBUG_HEAP_STACKSIZE setting to work correctly, and bumped the maximum to 1024.
    • Added the version number to the code.
  • Version 0.8
    • Changed to handle MCH6801 exceptions when trying to retrieve OS/400 stack frame information. Causes less noise in joblogs.
  • Version 0.7
    • Changed the dumping slightly so that header information that the user really isn't concerned about isn't dumped. Also, storage used internally by the tool itself is not dumped. Also clarified some of the dump messages.
  • Version 0.6
    • Added the ability to use one OS/400 object per allocation. Helps detect accesses to free'd storage by causing MCH3402 exceptions, occasionally allows the application to take exceptions when walking off the front or back of an allocation. For walking off the end, an exception only uccurs when a page boundary is crossed. The system allocates storage in multiples of pages.
    • Added additional eyecatchers and more validation of free requests. The Debug heap now sends CPF9898 exception when an invalid free() request or a free() request when the application has corrupted data occurs. This will detect from a single byte to 16 byte overflows from the application.
    • Added the ability to have the heap initialize the storage allocated with invalid pointers so that any use of pointer fields within that storage results in the application taking an MCH3402 exception.
  • Version 0.5
    • !!Changed function names and exports!! so they wouldn't conflict with real Qp0w component code. (Added the DBH designator on every exported function.
    • Allow use from system and user state.
    • Added a header with eyecatcher, size and allocation number tracking to all heap allocations. When the entire heap is dumped, each allocation should now have a header on the front of it and be 16 bytes larger than you expected it to be.
    • For Realloc... Checked the size of the storage that was allocated and only copied that much of the storage.
    • Validated that the delete operator can take a NULL pointer.
    • Fixed... Realloc with a NULL pointer should act the same as malloc(). Fixed it so that it didn't always return NULL always, because it took an exception.
  • Version 0.4
    • Provided strdup, realloc
  • Version 0.3
    • Added thread safety support and new service program for CPA threads.
    • Added thread IDs to output spool files when using the QP0WHEAPC (CPA) debug heap code.
  • Version 0.2
    • Misc. output format changes for better readability, script processing, or more consistent formatting.
      • Remove the MATHSAT (Materialize Heap Space Attributes) MI template from the dump its redundant, and its there twice... 8-)
      • Rearrange the call stack output so the procedure name is the last field. This is the field that has the dynamic length. If its last on the line, we can allocate more space for it, but still have the output more readable by rearranging the columns. This will allow a small 5250 display to display the majority of the information in most cases. Now, the program is always cut of when using a narrow display.
      • In the traceoutput, columnize the output so that the pointers and the sizes are vertically in line with one another. It'll make it visually easier for the eye to detect the difference between an allocation and a deallocation, due to the size value (s=xxx) sticking out at the end of the trace.
      • Leave the unknown procedure names blank instead of " - unknown - " in the call stack dumping.
      • Remove the "Qp0wFDumpStack__FP4FILE" entry from the call stack dump. Its ALWAYS the same, and is nor required by the user. The HeapStackTrace should probably be left in, because if the user manually calls Qp0wDBHDumpStack(), that function won't be present.
    • Added environment variable QP0W_DEBUG_HEAP_STACKSIZE to allow changing the maximum stack size for dumping. Default is 64, valid range is 64 to 1024.
  • Initial release version 0.1

Example Output

Allocation trace file with no stack information (default)

When allocations are traced, the pointer to the allocated storage and the size is traced, as well as a unique number identifying the allocation. Note that the unique ID starts at 3 and skips numbers in this example. This is because internal allocations are done when the Debug Heap dumps the threads stack or the entire heap. You can ignore these gaps in the allocation ids. See the heap storage dump spool file to make a correspondence between the allocation ID and the actual storage.

When deallocations are done, they similarly have the 16 byte pointer traced. The s is the size in bytes, and the p is the pointer value returned by the allocation function or free'd by the deallocation call. On functions such as realloc and strdup, both the input and the output pointers are traced, pold is the input pointer, and pnew is the output pointer. Realloc of course free's that input pointer, while strdup does not.

Note that in this example there are 3 functions which allocate storage (malloc, realloc, calloc), and only 2 that free it (realloc, and free). It might be a bug... 8-).

AS/400 Debug Heap file - Fred Kulack, version 0.85.
Output date: Tue Mar 14 14:40:11 2000

#2 Malloc  p=80000000 00000000 c82362b6 fe001020    s=100
#4 Realloc pold=80000000 00000000 c82362b6 fe001020
           pnew=80000000 00000000 c82362b6 fe001180 s=200
   Free    p=80000000 00000000 c82362b6 fe001180
#5 Calloc  p=80000000 00000000 c82362b6 fe0012a0    s=100

Allocation trace file showing a corrupted free

The following example shows a really buggy testcase doing something that you hopefully can catch in your applications. Its doing a free after corrupting a storage fence (i.e. allocate 200 bytes, and write to the 201st byte). After the stack dump for the invalid memory operation is created in this allocation trace file, an AS/400 exception is also thrown to the user.

Trace spool file with stack information (Option 'Stack-on')

So you figured out that you may have a leak (using the heap dump spool file).

You can use the program, library, module, procedure and statement number of the stack dump information to help determine which code does the wrong thing.

For example if you try to do a free twice, you'll likely get a heap exception. Qp0wDBHFree() will catch that exception and show you this call stack information. With the stack information, you can find out who else (earlier in the trace file) may have free'd the storage. Note that the heap allocation ID numbers grow much faster in this output, because the debug heap allocates internal storage to dump the stack. Those storage allocations aren't shown in the trace.

Example.

Heap Storage Dump spool file

All heap dump trace records contain an allocation ID# and the size of the storage in bytes that the caller requested. This information can be used to correlate the block of allocated storage to the stack trace records.

For example: The only block remaining in the heap storage dump has an allocation ID# 8 and is for 100 bytes. If we look at the prior trace example that includes stack information above, we immediately see that allocation #8 for 100 bytes was done by function badFunction__Fv() in module TEST, at statement #31. That is the allocation that was never free'd.