fix(#261): support heap (address-less) ByteBuf in ByteBufProxy.in() - #291
Open
bernardladenthin wants to merge 1 commit into
Open
fix(#261): support heap (address-less) ByteBuf in ByteBufProxy.in()#291bernardladenthin wants to merge 1 commit into
bernardladenthin wants to merge 1 commit into
Conversation
…y.in() ByteBufProxy.in() called buffer.memoryAddress() unconditionally, which throws UnsupportedOperationException for any ByteBuf without a native address (hasMemoryAddress()==false) — i.e. any heap buffer. Netty 4.2's adaptive allocator can hand back heap-backed AdaptiveByteBuf instances (e.g. heapBuffer()), so apps on 4.2 hit this; but it is not 4.2-specific — a plain PooledHeapByteBuf fails identically on 4.1. Fix: when the buffer has no memory address, copy its readable bytes into native scratch (MemoryManager.allocateDirect) and point the MDB_val there, returning the Pointer so lmdbjava keeps the scratch reachable for the native call. This mirrors the existing ByteArrayProxy.in() copy path. Direct buffers keep the zero-copy fast path unchanged (in()'s null return). The reserve variant in(buffer, size, ptr) cannot support heap buffers (out() repoints the caller's buffer at LMDB memory via a PooledUnsafeDirectByteBuf field swap, which needs a direct buffer), so it now throws a clear LmdbException instead of an opaque UnsupportedOperationException. Note: this fixes the concrete, reproducible failure in lmdbjava#261. It deliberately does NOT touch allocate()'s exact-class check or out()'s zero-copy field swap; the report's claim that allocate() receives an AdaptiveByteBuf is not reproducible (PROXY_NETTY pins PooledByteBufAllocator, which still yields PooledUnsafeDirectByteBuf on 4.2, and the full suite passes on 4.2). Tests: ByteBufHeapBufferTest (heap value; heap key+value round-trip). Full suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01GdiZ3ABYsVHXBRNCkEizpE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the concrete, reproducible failure behind #261.
Root cause (verified)
ByteBufProxy.in()calledbuffer.memoryAddress()unconditionally. That throwsUnsupportedOperationExceptionfor anyByteBufwithhasMemoryAddress() == false— i.e. any heap buffer:Netty 4.2's adaptive allocator can hand back heap-backed
AdaptiveByteBufinstances (e.g.heapBuffer()), so 4.2 apps trip this — but it is not 4.2-specific: a plainPooledHeapByteBuffails identically on 4.1. So the fix targets the underlying issue and applies on both.Fix
When the buffer has no memory address, copy its readable bytes into native scratch (
MemoryManager.allocateDirect) and point theMDB_valthere, returning thePointerso lmdbjava keeps the scratch reachable for the native call — the same patternByteArrayProxy.in()already uses. Direct buffers keep the zero-copy fast path unchanged.The reserve variant
in(buffer, size, ptr)can't support heap buffers (itsout()repoints the caller's buffer at LMDB memory via aPooledUnsafeDirectByteBuffield swap, which requires a direct buffer), so it now throws a clearLmdbExceptioninstead of an opaqueUnsupportedOperationException.What this does NOT do (deliberately)
It does not touch
allocate()'s exact-class check orout()'s zero-copy field swap. During investigation I could not reproduce the report's stated mechanism — "allocate() getsAdaptivePoolingAllocator$AdaptiveByteBuf":PROXY_NETTYpinsPooledByteBufAllocator, which still returnsPooledUnsafeDirectByteBufon 4.2, and the full suite passes on 4.2 (verified locally, 918/918). So that part of the writeup appears inaccurate; I've asked on the issue for a concrete stack trace in case there's a different failure to address separately.Tests
ByteBufHeapBufferTest— heap value round-trip, and heap key+value round-trip (bothputand lookup). Full suite green;fmt-maven-plugin:checkclean. No change to the direct-buffer fast path.Refs #261.