Add tools/memory_snapshot.py to measure startup memory footprint - #15358

Motivation

The recent startup-performance work (#15310, #15333, and the open #15340 / #15353) is well served by tools/importtime_average.py, but that tool only answers "what is slow to import?". There is currently no tooling at all for memory — no tracemalloc, memory_profiler, objgraph, or pympler anywhere in the repo.

Those are genuinely different questions. A module can be slow to import but cheap to keep around (compiles once, retains little), or fast to import but expensive (unmarshals megabytes of tables). More importantly, caches built during InteractiveShell construction cost memory without appearing in -X importtime at all — the Pdb frame-pinning lru_cache fixed in #15310 is exactly that shape of bug, and import-time profiling could never have surfaced it.

What this adds

A single self-contained script, deliberately mirroring the style and CLI conventions of tools/importtime_average.py.

Each measurement runs in a fresh subprocess. This is the central design point: imports are sticky, so once prompt_toolkit is in sys.modules, asking what wcwidth costs tells you nothing. A bare-interpreter baseline is measured and subtracted so the reported numbers are attributable.

Two metrics per scenario:

• RSS — peak resident set via resource.getrusage, i.e. what the OS actually charges, including the unmarshalled .pyc code objects that dominate startup. ru_maxrss is KiB on Linux but bytes on macOS; the script handles the difference rather than silently reporting numbers 1024× wrong.
• module count — sys.modules growth. A good proxy for import-graph breadth and far less noisy than RSS.

Defaults cover import IPython, the terminal shell, the ipykernel shell, and the heavy third-party dependencies. -c / -m measure arbitrary statements or modules. --tracemalloc attributes allocations to source lines when RSS alone is not enough to locate the culprit. --json emits raw data for scripting. Scenarios that fail — typically an optional dependency that is not installed — are reported as unavailable rather than aborting the run, so the tool is useful in a minimal environment.

Sample output

bare interpreter: 12.4 MiB (subtracted below)
median of 3 run(s), each in a fresh subprocess

  import IPython                         6.6 MiB    151 modules
  terminal shell                        26.9 MiB    457 modules
  kernel shell                          27.1 MiB    381 modules
  prompt_toolkit                        14.3 MiB    264 modules
  jupyter_client                        20.7 MiB    270 modules
  jedi                                  12.4 MiB    177 modules
  wcwidth                                3.7 MiB     53 modules
  traitlets                              0.8 MiB     38 modules
  pygments                               0.0 MiB      1 modules
  zmq                                    1.8 MiB     44 modules
  tornado                                0.0 MiB     10 modules
  asyncio                                6.3 MiB     79 modules

Measurements are independent and therefore overlap (jupyter_client and IPython both pay for traitlets); the script says so explicitly rather than inviting readers to sum the rows.

Observations from running it

Recorded here because they bear on where further optimisation effort is and is not worth spending:

• The terminal and kernel paths cost about the same but for entirely different reasons. ZMQInteractiveShell subclasses InteractiveShell, not TerminalInteractiveShell, so prompt_toolkit, wcwidth and jedi are never imported under ipykernel — verified directly. The terminal's dominant cost is prompt_toolkit; the kernel's is jupyter_client.
• IPython's own code is a minority of its footprint in both paths — roughly a quarter to a third, with the balance in third-party dependencies. This is worth knowing before anyone proposes rewriting IPython modules as native extensions for memory reasons: the addressable slice is small, and extension modules bring their own resident cost.
• jupyter_client's import graph pulls in email (13 modules), dateutil (11) and colorama (6). Those look incidental rather than necessary, but they are upstream of this repository.
• wcwidth costs ~3.7 MiB and declares a __lazy_modules__ list, but on Python < 3.15 its __init__.py eagerly performs 18 from .x import y imports anyway, so the lazy machinery does not currently take effect on supported versions. Also upstream.
• IPython/terminal/ptutils.py imports wcwidth solely to test wcwidth(ch) == 0 at two lines, which is a combining-mark test that unicodedata.combining() (already imported in that file) answers directly. Not proposed here, because prompt_toolkit.utils imports wcwidth unconditionally and ptutils imports prompt_toolkit in the same file — so the change would save exactly zero memory and is a readability matter only.

This PR adds no runtime code and changes no behaviour; it is tooling only.

Generated by Claude Code