Skip to content

Register IPython's built-in magics lazily - #15340

Open
Carreau wants to merge 1 commit into
mainfrom
claude/lazy-magic-registration-kogq8r
Open

Register IPython's built-in magics lazily#15340
Carreau wants to merge 1 commit into
mainfrom
claude/lazy-magic-registration-kogq8r

Conversation

@Carreau

@Carreau Carreau commented Jul 30, 2026

Copy link
Copy Markdown
Member

Starting a shell imported all fifteen modules under IPython.core.magics
and instantiated every Magics class in them, even though a session
typically uses a handful of magics at most. Most of that cost is not the
class bodies but what they drag in: urllib and ast for %edit, cProfile,
pstats and timeit for %timeit, plus the eighteen @magic_arguments()
decorations that each build an argparse parser and format its help at
import time.

Only the magic names are now known up front, from hand-maintained
tables in IPython.core.magics._table, and the module implementing a magic
is imported the first time that magic is looked up. Registering a name
installs a LazyMagic placeholder: listing magics, completing them and
membership tests only read the table's keys and stay cheap, while
calling a magic -- or touching any attribute of one -- imports the
module, registers the real Magics instance in its place and delegates.
The same mechanism is available to third parties through
MagicsManager.register_lazy_class(), which, unlike the existing
lazy_magics config, does not require packaging the magics as an
extension.

This takes roughly 30 ms off import IPython and 20 ms off
import-plus-shell-startup.

Keeping the tables honest is tests/test_magic_table.py: it imports every
magics module, instantiates every Magics subclass, and fails -- printing
the corrected table -- if anything is missing, stale or attributed to
the wrong class. It also checks that starting a shell still imports no
magics module at all.

Places that need the whole picture rather than one magic keep working:
%config and its completions load everything first, so the list of
configurable classes is unchanged, and registry["ExecutionMagics"]-style
lookups load the class on the miss. init_magics no longer runs %colors
purely to trigger its observer, since that alone would import the basic
magics on every startup.

Co-Authored-By: Claude Opus 5 [email protected]
Claude-Session: https://claude.ai/code/session_01BqGDoRyv11kHsBvvFQD4T4

Starting a shell imported all fifteen modules under IPython.core.magics
and instantiated every Magics class in them, even though a session
typically uses a handful of magics at most. Most of that cost is not the
class bodies but what they drag in: urllib and ast for %edit, cProfile,
pstats and timeit for %timeit, plus the eighteen @magic_arguments()
decorations that each build an argparse parser and format its help at
import time.

Only the magic *names* are now known up front, from hand-maintained
tables in IPython.core.magics._table, and the module implementing a magic
is imported the first time that magic is looked up. Registering a name
installs a LazyMagic placeholder: listing magics, completing them and
membership tests only read the table's keys and stay cheap, while
calling a magic -- or touching any attribute of one -- imports the
module, registers the real Magics instance in its place and delegates.
The same mechanism is available to third parties through
MagicsManager.register_lazy_class(), which, unlike the existing
lazy_magics config, does not require packaging the magics as an
extension.

This takes roughly 30 ms off `import IPython` and 20 ms off
import-plus-shell-startup.

Keeping the tables honest is tests/test_magic_table.py: it imports every
magics module, instantiates every Magics subclass, and fails -- printing
the corrected table -- if anything is missing, stale or attributed to
the wrong class. It also checks that starting a shell still imports no
magics module at all.

Places that need the whole picture rather than one magic keep working:
%config and its completions load everything first, so the list of
configurable classes is unchanged, and registry["ExecutionMagics"]-style
lookups load the class on the miss. init_magics no longer runs %colors
purely to trigger its observer, since that alone would import the basic
magics on every startup.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BqGDoRyv11kHsBvvFQD4T4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants