[C API] SVS custom allocator api support

[klmckeig]

---

Problem

SVS allocates from its own private C++ heap. PostgreSQL cannot observe, account for, or bound those
allocations. PostgreSQL memory cap has no effect on Vamana index builds because the SVS graph
structure is invisible to PG memory contexts.

A custom allocator hook in the SVS C API would let the PG extension intercept every SVS allocation
with a byte counter, turning the governance model from a pre-flight estimate into real-time
enforcement.

---

Existing SVS Allocator Infrastructure

SVS's C++ layer already has a well-designed allocator abstraction — this proposal asks to surface it
through the C API.

svs::lib::Allocator<T> (include/svs/lib/memory.h) — default STL-compatible allocator, uses
::operator new(n, std::align_val_t(alignof(T))). This is what svs::c_runtime::MaybeBlockedAlloc
defaults to when no custom allocator is provided.

svs::HugepageAllocator<T> (include/svs/core/allocator.h) — allocates via mmap with
hugepage support (1GB → 2MB → 4KB fallback). Used as the default allocator for graph adjacency
lists: SimpleGraph<Idx, HugepageAllocator<Idx>>.

svs::AllocatorInterface / AllocatorHandle<T> (include/svs/core/allocator.h) — a
type-erased runtime-polymorphic allocator base class already present in the C++ layer. Has
allocate(n), deallocate(ptr, n), clone(), and rebind_*() virtual methods. This is the
natural C++ target for the C API wrapper to call into.

svs_threadpool_interface (include/svs/c_api/svs_c.h, lines 76–90) — the existing custom
thread pool C API. The allocator API should follow exactly the same vtable + self-pointer pattern.

---

Proposed C API

The allocator API is modelled on svs_threadpool_interface — the existing pattern SVS already uses
for caller-supplied implementations.

svs_c.h additions

/* -----------------------------------------------------------------------
 * Custom allocator interface
 *
 * All function pointers MUST be thread-safe — SVS build and search paths
 * call them concurrently from multiple threads.
 *
 * `self` is the opaque caller-owned state pointer passed through unchanged
 * on every call. Use it to carry a byte counter, a lock, or a pool handle.
 *
 * `size` is passed to free/aligned_free so callers do not need to
 * maintain a side-table of allocation sizes (standard free(ptr) does not
 * communicate size back to the caller).
 * ----------------------------------------------------------------------- */

// clang-format off
struct svs_allocator_interface_ops {
    void *(*malloc)       (void *self, size_t size);
    void  (*free)         (void *self, void *ptr,  size_t size);
    void *(*realloc)      (void *self, void *ptr,  size_t old_size, size_t new_size);

    /* aligned_alloc is mandatory — SVS uses SIMD types requiring alignment > 8.
     * HugepageAllocator uses operator new(n, align_val_t), CacheAlignedAllocator
     * uses 64-byte alignment for thread-local storage. Both must be intercepted. */
    void *(*aligned_alloc)(void *self, size_t alignment, size_t size);
    void  (*aligned_free) (void *self, void *ptr, size_t alignment, size_t size);
};
// clang-format on

struct svs_allocator_interface {
    struct svs_allocator_interface_ops ops;
    void *self;
};

typedef struct svs_allocator_interface *svs_allocator_i;

New builder function

/**
 * Attach a custom allocator to a builder.
 *
 * Must be called after svs_index_builder_create() and before svs_index_build()
 * or svs_index_build_dynamic().
 *
 * SVS does not copy the allocator_interface struct — the caller must keep it
 * alive for the lifetime of any index built or loaded with this builder.
 *
 * Passing NULL restores the default allocator (svs::lib::Allocator /
 * HugepageAllocator depending on the data structure).
 *
 * @param builder   Builder handle to configure.
 * @param allocator Caller-supplied allocator interface, or NULL to reset.
 * @param out_err   Error output handle.
 */
SVS_API void svs_index_builder_set_allocator(
    svs_index_builder_h          builder,
    svs_allocator_i              allocator,   /* NULL → reset to default */
    svs_error_h                  out_err
);

/**
 * Attach a custom allocator to a loaded index (for search-path allocations —
 * thread-local candidate buffers, search history pools, etc.).
 *
 * Must be called before the first search on this index handle.
 *
 * @param index     Index handle to configure.
 * @param allocator Caller-supplied allocator interface, or NULL to reset.
 * @param out_err   Error output handle.
 */
SVS_API void svs_index_set_allocator(
    svs_index_h                  index,
    svs_allocator_i              allocator,
    svs_error_h                  out_err
);

---

Required SVS C++ Changes

The C API additions require threading the allocator through the existing C++ dispatch path:

IndexBuilder::build() / build_dynamic() (bindings/c/src/index_builder.hpp) must forward
the allocator to dispatch_vamana_index_build() and dispatch_dynamic_vamana_index_build().

dispatch_vamana_index_build() (bindings/c/src/dispatcher_vamana.hpp) must accept an
optional AllocatorHandle and pass it down to graph construction:

svs::Vamana dispatch_vamana_index_build(
    const svs::index::vamana::VamanaBuildParameters& build_params,
    svs::data::ConstSimpleDataView<float> data,
    const Storage* storage,
    svs::DistanceType distance_type,
    svs::threads::ThreadPoolHandle pool,
    std::optional<svs::AllocatorHandle<std::byte>> allocator = std::nullopt  // NEW
);

SimpleGraph construction (include/svs/core/graph.h, default_graph()) currently defaults
to HugepageAllocator<Idx>. With the custom allocator, the C API wrapper builds a
svs::AllocatorHandle backed by the caller's vtable and passes it through.

The C → C++ bridge in svs_c.cpp wraps the C vtable in a thin AllocatorInterface subclass:

class CAllocatorBridge : public svs::AllocatorInterface {
    svs_allocator_i iface_;
public:
    CAllocatorBridge(svs_allocator_i iface) : iface_(iface) {}
    void* allocate(size_t n) override {
        return iface_->ops.malloc(iface_->self, n);
    }
    void deallocate(void* ptr, size_t n) override {
        iface_->ops.free(iface_->self, ptr, n);
    }
    // clone(), rebind_float(), rebind_float16() forward to same vtable
};

---

Why size Must Be Passed to free

Standard C free(ptr) does not communicate size back to the caller. Without size in the free
callback, the counter can only go up — you'd need to maintain a side-table of ptr → size mappings
to decrement it on free, which requires a lock and a hash map inside every free call.

Passing size to free and aligned_free makes the counter trivially correct without any
bookkeeping. SVS already knows the allocation size at the deallocation call site (it tracks it to
call sized operator delete), so this is no extra burden on SVS.

---

Why aligned_alloc Cannot Be Omitted

SVS uses two distinct aligned allocation paths that would bypass a malloc-only hook:

1. svs::lib::Allocator<T> — uses ::operator new(n, std::align_val_t(alignof(T))) for any
   type with alignment > __STDCPP_DEFAULT_NEW_ALIGNMENT__ (16 bytes on x86). SIMD types
   (AVX-512) have 64-byte alignment requirements.

2. CacheAlignedAllocator<T, 64> — used by SequentialTLS for thread-local build buffers.
   Uses ::operator new(bytes, std::align_val_t{64}) directly.

Both must be intercepted to get an accurate byte count.

---

Open Questions for SVS Team

1. NULL return handling: SVS must handle NULL returns from the allocator by propagating
   SVS_ERROR_OUT_OF_MEMORY rather than dereferencing or abort()ing. Is this already guaranteed
   by the C++ exception model, or does it require explicit if (!ptr) throw std::bad_alloc()
   guards at each allocation site?

2. realloc usage: Does SVS ever call realloc? If yes, the bridge must implement it. If no,
   we can assert-fail on that vtable slot to catch regressions.

3. Allocator lifetime for loaded indexes: svs_index_set_allocator is proposed for search-path
   allocations on already-loaded indexes. Are search-path allocations per-query (freed at end of
   search) or retained (thread-local buffers that persist)? This affects whether the counter
   accurately reflects live usage or cumulative usage.

4. AllocatorHandle rebind: The existing AllocatorInterface has rebind_float() and
   rebind_float16() virtual methods. The C bridge needs to implement these. Should they return a
   new CAllocatorBridge backed by the same vtable, or does SVS expect a different allocator for
   different element types?

---

[klmckeig]

PostgreSQL-Side implementation plan for reference

Once SVS exposes this API, the PG extension implements a counter-backed allocator:

/* vamanabuild.c */

typedef struct VamanaBuildAllocatorCtx {
    pg_atomic_uint64  bytes_committed;
    uint64            limit;    /* maintenance_work_mem in bytes */
} VamanaBuildAllocatorCtx;

static void *
vamana_alloc_malloc(void *self, size_t size)
{
    VamanaBuildAllocatorCtx *ctx = self;
    uint64 prev = pg_atomic_fetch_add_u64(&ctx->bytes_committed, size);
    if (prev + size > ctx->limit) {
        pg_atomic_fetch_sub_u64(&ctx->bytes_committed, size);
        return NULL;  /* SVS must propagate NULL as SVS_ERROR_OUT_OF_MEMORY */
    }
    return malloc(size);
}

static void
vamana_alloc_free(void *self, void *ptr, size_t size)
{
    VamanaBuildAllocatorCtx *ctx = self;
    pg_atomic_fetch_sub_u64(&ctx->bytes_committed, size);
    free(ptr);
}

static void *
vamana_alloc_aligned(void *self, size_t alignment, size_t size)
{
    VamanaBuildAllocatorCtx *ctx = self;
    uint64 prev = pg_atomic_fetch_add_u64(&ctx->bytes_committed, size);
    if (prev + size > ctx->limit) {
        pg_atomic_fetch_sub_u64(&ctx->bytes_committed, size);
        return NULL;
    }
    void *ptr = NULL;
    posix_memalign(&ptr, alignment, size);
    return ptr;
}

static void
vamana_alloc_aligned_free(void *self, void *ptr, size_t alignment, size_t size)
{
    VamanaBuildAllocatorCtx *ctx = self;
    pg_atomic_fetch_sub_u64(&ctx->bytes_committed, size);
    free(ptr);
}

/* Wire up and attach before build */
static svs_allocator_interface_ops vamana_alloc_ops = {
    .malloc        = vamana_alloc_malloc,
    .free          = vamana_alloc_free,
    .realloc       = NULL,   /* SVS must not call realloc, or we map it */
    .aligned_alloc = vamana_alloc_aligned,
    .aligned_free  = vamana_alloc_aligned_free,
};

This converts the governance from cooperative pre-flight estimate to enforced runtime limit:
if SVS attempts to exceed maintenance_work_mem, malloc returns NULL, SVS propagates
SVS_ERROR_OUT_OF_MEMORY, and the PG extension catches it as ERRCODE_OUT_OF_MEMORY. No OOM kill
possible.