fix: use-after-free in unpackb() ExtraData path for non-contiguous input - #722
Open
ThomasWaldmann wants to merge 1 commit into
Open
fix: use-after-free in unpackb() ExtraData path for non-contiguous input#722ThomasWaldmann wants to merge 1 commit into
ThomasWaldmann wants to merge 1 commit into
Conversation
For non-contiguous input, get_data_from_buffer() makes a temporary contiguous copy and buf points into it. unpackb() released the buffer view (freeing that copy) in the finally block, and only afterwards read buf+off to build the ExtraData payload, so ExtraData.extra was filled from freed memory. Copy the extra data out before releasing the view. Fixes msgpack#720
Contributor
Author
|
^ made with Claude Opus 5, please review carefully. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #720.
Problem
For non-contiguous input,
get_data_from_buffer()releases the original view and makes a temporary contiguous copy, sobufpoints into memory owned byview:https://github.com/msgpack/msgpack-python/blob/main/msgpack/_unpacker.pyx#L129-L134
unpackb()then releasesviewin itsfinallyblock — freeing that copy — and only afterwards readsbuf+offto build theExtraDatapayload.ExtraData.extrais therefore filled from freed memory.This is the same class of bug as #677, on a code path that fix did not cover.
Reproducer (from #720)
Reproduced on macOS 15 / arm64 / CPython 3.14 with 1.2.1 and with current
main. Running the same reproducer under the macOS allocator's free-poisoning makes the use-after-free unambiguous:so arbitrary heap contents can end up in
ExtraData.extra, not merely a shifted copy of the input. The reporter's ASan output (memcpy-param-overlap) is the same event seen from the other side:PyBytes_FromStringAndSizere-allocates the just-freed block and copies it onto itself.Unpacker.feed()uses the same helper but copies viaappend_buffer()before releasing, so it is unaffected.fallback.pyis also unaffected.Fix
Copy the extra data out before releasing the view. Kept minimal:
unpack_data()still runs outside thetry, so only thePyBytes_FromStringAndSizecall moved.Test
test_unpack_noncontiguous_memoryview_extra_dataintest/test_memoryview.py, next to the #677 test. Verified it fails onmain(At index 4 diff: b'\x00' != b'a') and passes with the fix.Full suite passes for both implementations — 135 passed (Cython), 133 passed / 2 skipped (
MSGPACK_PUREPYTHON=1) — and also withMallocScribble=1.Note that whether the corruption is observable depends on allocator reuse, so the test uses a payload/extra size combination that reliably triggers it on CPython's pymalloc; the assertion itself is unconditionally correct.