gh-157605: Accept empty Python frame stack in sampling frame chain parser - #157657
Open
aidaodedjl wants to merge 2 commits into
Open
aidaodedjl wants to merge 2 commits into
aidaodedjl wants to merge 2 commits into
Conversation
…ain parser A native thread can retain a Python thread state while waiting in C code with no Python functions executing. The frame chain of that thread state holds only the base_frame sentinel, and process_frame_chain() treated the resulting NULL first frame as a broken chain, failing the entire sampling call and discarding valid stacks from all other threads. Only raise when the unparseable first frame is not the base_frame sentinel; otherwise stop the walk and accept the empty stack. Walking stops at the sentinel, so last_frame_visited still equals base_frame_addr and complete-stack validation in the frame cache is preserved. Tests: a self-contained regression test in test_external_inspection.py compiles a small C helper at runtime (skipping when no compiler is available), starts a native thread that retains a Python thread state with an empty frame stack via ctypes, and asserts that same-process sampling succeeds while other Python threads remain visible, with and without frame caching.
sysconfig's install-style include paths do not necessarily exist for build-tree interpreters: in an out-of-tree build the test's compile step could not find Python.h (it lives in the source tree's Include directory while pyconfig.h lives in the build directory), so the helper failed to compile and the CI test run failed. Gather candidate include directories from INCLUDEPY, sysconfig paths, the executable directory and srcdir, and keep the ones that actually contain Python.h or pyconfig.h. Skip cleanly when either header cannot be found.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A native thread can retain a Python thread state while waiting in C code with no Python functions executing (e.g. after
PyGILState_Ensure()+PyEval_SaveThread()). The frame chain of that thread state holds only thebase_framesentinel.process_frame_chain()treated the resulting NULL first frame as a broken chain, raisedRuntimeError: Failed to parse initial frame in chain, and failed the entire sampling call — discarding valid stacks from all other threads.With this change, the unparseable-first-frame error is raised only when the first frame is not the
base_framesentinel. When it is the sentinel, the walk stops and the empty Python stack is accepted, so sampling returns results for the remaining threads.The walk stops at the sentinel, so
last_frame_visitedstill equalsbase_frame_addrand the complete-stack validation in the frame cache keeps working for the empty-stack case.Reproduction / evidence
pin=1(native thread retaining an empty-stack thread state) fails 40/40 samples withRuntimeError: Failed to parse initial frame in chain, with and without frame caching;pin=0baseline succeeds 20/20.pin x cache_framescombinations succeed 20/20 with the target thread's Python frames visible.Test
test_external_inspection.pygains a self-contained regression test (TestEmptyPythonStackSampling):ctypes);pin x cache_framescombinations;CCis not set) and fails loudly if the helper itself fails to compile.On this machine (macOS, 3.16 debug build) the test reproduces the failure before the fix (
{'failure': 5}) and passes after (1.1 s).