Converting a Custom Memory Allocator

Overview

When an application has its own custom memory allocator, it can still run ADI without modifying the source code to check the large memory blocks allocated by the standard memory allocator. If the application also wants to check the small memory chunks allocated by its own custom memory allocator, the application source code needs to be modified. The change of the source code is relative minor in most cases. This document discusses what changes are needed for ADI checking.

Requirements

An application program needs to meet the following requirements for SPARC M7 ADI code checking.
  1. The application binary must be built in 64-bit mode.
  2. The application needs to enable ADI on the target memory area.
  3. The allocated memory needs to be 64-byte aligned and its size must be multiple of 64.
  4. The allocated area should be set a version number with the pointer value being adjusted with the corresponding version number.
  5. Complex pointer manipulation should be avoided, but simple pointer operations can still work.
Let's explore these in a little more detail.

1. The application binary must be built in 64-bit mode

ADI requires the memory pointer in 64 bits, because it embeds the version number in the high order bits of the pointer. Therefore the application must be built in 64-bit mode. Add the compilation flag -m64 when you compile your application with Oracle Solaris Studio C or C++ compilers. All the linked libraries must be in 64 bit mode too.

2. The application needs to enable ADI on the target memory area

The program needs to call memcntl(address, size, MC_ENABLE_ADI, NULL, 0,0) to enable ADI on the memory area starting at the spcified address with the specified size. Please note that both address and size must be PAGESIZE (8k) aligned. Example:
large_block_ptr = (large_block*) memalign(8192, 64 * 1024);
if (memcntl(large_block_ptr, 64 * 1024, MC_ENABLE_ADI,NULL,0,0)!=0) {
    error("memcntl call for enabling ADI failed");
}

3. The allocated memory needs to be 64-byte aligned and its size must be multiple of 64

ADI is implemented in hardware with the granularity of a 64-byte cache line. Therefore any allocated memory chunk needs to be 64-byte aligned and its size must be multiple of 64. Example:
object_ptr = (my_object*) my_malloc(sizeof(my_object));
needs to be changed to:
adjusted_size = (sizeof(my_object) + 63) & ~63; // adjust to multiple of 64
object_ptr = (my_object*) my_malloc(64,adjusted_size); 
// my_malloc() needs to make sure the allocated object address is 64 byte aligned

4. The allocated area should be set a version number with the pointer value being adjusted with the corresponding version number.

When a memory chunk is allocated, a version number should be assigned to the area. It is important to have different version numbers for the adjacent areas. ADI supports version numbers from 0 to 13. 0 is reserved for system to use, the application can use the numbers from 1 to 13. Example (continue from the above example):
adjusted_object_ptr = (my_object*) adi_set_version(object_ptr, adjusted_size, new_version_number);
if (adjusted_object_ptr == NULL) report_error();

5. Complex pointer manipulation should be avoided, but simple pointer operations can still work.

In general, avoid mixing the pointers with possible different version numbers in a expression except comparison. But most popular pointer operations such as array element access by adding pointer and index value still work. Example:
adjusted_array_ptr = (my_array*) adi_set_version(array_ptr, adjusted_array_size, my version_number);
if (adjusted_array_ptr == NULL) report_error();
(adjusted_array_ptr + 2)->value = 100; // set the third array element structure value field to 100.

Catching ADI Signal

When the application program encounters an ADI error, the process is sent a segmentation fault signal. By default, this kills the process and produces a core file. However the application can create a signal handler that catches the signal to customize the ADI error handling behavior. Below is an example of how to catch the signal and print the instruction address when an ADI version mismatch occurs Example:
// setup the signal handler at the beginning of application process.

#include <signal.h>

struct sigaction sa;
sa.sa_sigaction = sig_handler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = SA_SIGINFO;
if (sigaction(SIGSEGV, &sa, NULL) == (int)SIG_ERR) {
    printf("ERROR : sigset failed on SIGSEGV\n");
    exit(1);
}

/* signal handler which prints the pc of version mismatch */
void sig_handler(int sig, siginfo_t *info, void *uap)
{
  if (info->si_signo != SIGSEGV)
    return;

  caddr_t instr_addr;
  ucontext_t *ucp = (ucontext_t*)uap;
 
  if (info->si_code == SEGV_ADIPERR) {  // precise error
    instr_addr = si_pc;

  } else if (info->si_code == SEGV_ADIDERR) { // disruptive error
    instr_addr = info->si_addr;
  } else {
    return;
  }

  fprintf(stderr, "Version mismatch detected at PC 0x%lx\n", instr_addr);
}

A custom memory allocator example program with ADI changes in pseudo code is described in the next page.