Major release: the legacy worker API is removed; contexts (py_context, py:call/3) are the only execution path. See Removed below for the full list of removed functions.

Added

• isolated context mode - py_context:new(#{mode => isolated}) runs
  CPython in a child OS process per context, with the same call/eval/exec,
  callback, erlang.send/whereis, worker-loop and pool API as the embedded
  modes. It is the first mode with a hard bound: py_context:interrupt/1
  stops a blocking C call (a signal in the child) and SIGKILL is the
  backstop after kill_after ms; py_context:kill/1 kills at once. rlimits
  (as, cpu, nofile) and a cgroup v2 directory bound the child; a
  segfault in a C extension returns {error, {child_exited, {signal, 11}}}
  and the node survives. The child restarts on crash within a budget
  (restart, max_restarts, restart_period); py_context:child_info/1
  reports its OS pid. Children are reaped by the VM and exit when the BEAM
  dies (socket EOF watchdog, PR_SET_PDEATHSIG on Linux, PROC_PDEATHSIG_CTL
  on FreeBSD). cgroup is refused outside Linux; rlimits apply everywhere:
  as is kernel-enforced on Linux and FreeBSD and enforced by an RSS
  watchdog in the child on macOS ({child_exited, {memory_limit, Bytes}}).
  Validated on macOS (arm64) and FreeBSD 14.3 (OTP 28, Python 3.11).
• py_context:pass_fd/2 - hands a file descriptor to an isolated child
  over the control socket (SCM_RIGHTS), so erlang.server.serve works out
  of process: Erlang binds once, N killable children accept.
• Pure-Python ETF codec (priv/_erlang_impl/_etf.py) with the type
  mapping of py_convert.c; the child needs no C extension. Integers beyond
  64 bits round-trip exactly in isolated mode.
• Shared memory - py_shm:new/1,2, write/3, read/3, binary/3
  (no copy), close/1: fixed-size regions over
  iommap (optional dependency) that any
  context mode maps as erlang.SharedMemory (buffer protocol, numpy
  friendly). py_buffer:new(#{shared => true}) is a streaming buffer over
  such a region with ring backpressure, usable as wsgi.input in isolated
  contexts. Handles are plain terms and travel inside any argument or result;
  py_shm:read_only/1 and new(Size, #{writable => false}) hand Python a
  read-only mapping. py_buffer:write/3 takes a timeout for the case where
  the ring is full and nobody reads (default 30 s).
• py:python_executable/0, py:kill/1, py_nif:os_kill/2.
• py_isolated is a gen_statem (states idle, {busy, Id}, looping,
  stopping_loop, {restarting, Reason}): sys:get_state/1 and
  sys:trace/2 work on isolated contexts, requests arriving during a
  restart are served by the new child, and py_context:kill/1 returns once
  the new child is up.
• Timeouts on an isolated context cancel their own request only (queued
  requests are dropped, the executing one is interrupted); the kill backstop
  is bound to that request, so a busy shared context is never killed because
  another caller gave up. Soak-tested: callback storms, interrupt/kill
  storms, loop churn, 60 s mixed workload with resource counters checked.
• Guide: docs/isolated.md, with what each of the three modes guarantees.

Changed

• The NIF side of every context request goes through one dispatcher
  (ctx_dispatch, ctx_dispatch_async in c_src/py_nif.c) instead of a
  per-request copy of the enqueue-and-wait loop; the execute functions are
  ctx_execute_* and the thread functions ctx_thread_main_*, since both
  serve worker and owngil contexts. Creating a process-local env and
  applying imports or paths run on the context thread in worker mode too;
  the scheduler-side copies of those paths are gone.
• The NIF function table is assembled from one PY_*_NIFS macro per area,
  defined at the end of the file that owns the NIFs.
• py_context keeps the API and the reply protocol; the process body for
  embedded modes moved to py_context_embedded. py delegates streaming,
  virtual environments and shared dicts to py_stream, py_venv and
  py_shared_dict. The public API is unchanged.

Documentation

• make check-code-map (also run by CI) verifies that every source file is
  in docs/code-map.md, every Erlang module has a moduledoc and a row in
  the Modules table of test/coverage_audit.md.

Removed

• The legacy worker API (py_nif:worker_new/0,1, worker_call, worker_eval,
  worker_exec, worker_next, worker_destroy, import_module/2,
  get_attr/3, set_callback_handler/2, send_callback_response/2,
  resume_callback/2) and the single executor thread behind it, the
  async_worker_*/async_call/async_gather/async_stream NIFs that only
  returned deprecated, the unused worker pool (pool_* NIFs), the
  cancel_reader/writer aliases, and the unreachable inline executor
  branches of the context NIFs. Contexts (py_context, py:call/3) are the
  only execution path. py:memory_stats/0 and py:gc/0,1 now run on the
  calling scheduler under the GIL.

Fixed

• pthread_timedjoin_np was called without _GNU_SOURCE, an implicit
  declaration on Linux that newer compilers reject.
• Callback pipes waited with select(), which is undefined for a file
  descriptor above 1024: in a VM with many open files a thread callback
  could time out with "Failed to spawn thread handler". The waits use
  poll(), the handler ready-wait no longer holds the GIL, and
  py_thread_handler logs a failed ready signal.

Worker loops: run an ErlangEventLoop forever inside a context and drive it from Erlang. This is the runtime API for gunicorn-style Python workers inside the VM (Erlang binds the socket once, N owngil contexts accept on their copy of the fd, Erlang supervises).

Added

• py_context:start_loop/1,2, stop_loop/1,2, loop_ref/1, submit/4,5, submit_await/4,5,6; {py_loop_exit, Ctx, Result} to the owner; {error, loop_running} for call/eval/exec while a loop runs; preload option on py_context:new/1.
• erlang.server: serve(listen_fd, factory, udp=False), adopt(fd, factory), stop_serving.
• Coroutine injection into owngil loops from any Erlang process; task start failures reported to the caller instead of dropped.

Fixed

• owngil contexts had no ErlangEventLoop; context_get_event_loop returned the main loop and re-pointed its worker.
• owngil calls went through a blocking dispatch on a dirty scheduler with a 30 s cap; now the async queue.
• fds closed while still in the BEAM poll set under connection churn (erts_poll / stealing control reports).
• READ re-arm from the loop thread crashed in enif_select.
• ErlangEventLoop recycled call_soon/call_at handles: cancelling one after it ran (as asyncio.sleep does) cancelled the callback that reused it.
• Queued tasks behind a running loop could be left waiting for the next wakeup.

Docs: docs/workers.md. Bench: examples/bench_worker_loop.erl.

Full changelog: v4.0.0...v4.1.0

Closes four gaps that blocked running Python you do not control: no way to stop
running code, no per-context memory bound, no async-generator streaming, and a
lossy callback result encoding.

Breaking

A string returned from an Erlang callback ("abc", a list of integers) now
reaches Python as [97, 98, 99] rather than 'abc', the same conversion call
arguments have always used. Return a binary (<<"abc">>) for a Python str.
Nothing raises on upgrade, so check what your registered callbacks return before
deploying.

Interrupt running Python

py:interrupt/1 raises KeyboardInterrupt in the thread executing a context;
the in-flight call returns {error, interrupted}. py_context:call/eval/exec
interrupt automatically when their timeout expires, so {error, timeout} now
stops the Python code instead of abandoning the reply while the thread kept
burning CPU and the context stayed wedged.

Works in worker and owngil mode, and is callable while the context process
is blocked in a NIF. Async exceptions land at bytecode boundaries, so code
blocked in a C call is interrupted once that call returns. See docs/interrupts.md.

Per-context memory caps

py_context:new(#{mode => owngil, memory_limit => Bytes}) caps memory per
context, accounted from obmalloc arenas and enforced with MemoryError. Opt-in
via {enable_memory_limits, true}, since the allocator is hooked before Python
starts.

Allocations over 512 bytes bypass obmalloc and are not counted, granularity is
one 1 MB arena, and worker mode returns {error, memory_limit_requires_owngil}
because those contexts share the main interpreter. See docs/memory.md.

Async generator streaming

py:stream_start/3,4 drives async generators on a private event loop. The docs
had claimed this since 3.0.0 with no code behind it. py:stream/4 with kwargs
and py:stream_eval/1,2 stay sync-only, now stated explicitly.

Callback results as external term format

Results from an Erlang callback cross as term_to_binary and are decoded by the
same converter used for call arguments, replacing a Python repr string parsed
with ast.literal_eval. Fixes binaries containing backslashes, quotes, newlines
or tabs (which produced an unparseable literal and were silently delivered as the
raw repr text), [] arriving as '', float precision loss, and the base64
round-trip for pids and refs, which now cross as native Pid/Ref objects.

Callback round-trips get faster as a result: a nested map goes from 104 to 46 us,
an 8 KB binary from 75 to 48 us. Hot paths are unchanged.

Full changelog: v3.1.1...v4.0.0

Changed

• Lower minimum OTP to 27 - minimum_otp_vsn is now 27. The OTP 28/29 support work was source-compatible with 27 (the try ... catch cleanups build fine there), so the floor was raised further than needed. CI now also builds and runs the full Common Test suite on OTP 27 across Python 3.12/3.13/3.14.

Full Changelog: 3.1.0...v3.1.1

Fixed

• NIF robustness hardening - make_py_error no longer passes a NULL message/type
  to enif_make_string/enif_make_atom when a Python exception's text isn't
  UTF-8-encodable; binary_to_string rejects names/code containing an embedded NUL
  (which would silently truncate a module/function/attr/code string) rather than
  truncating; a leaked split method object in the reactor buffer is released; and a
  stray debug fprintf on the normal worker send path is removed.

Security

• No shell for venv/installer commands - py:ensure_venv and dependency
  installation now run the executables via open_port({spawn_executable, ...}) with an
  argument list instead of building a shell string for os:cmd. Venv paths, requirement
  files, and extras are passed literally, so shell metacharacters can't be injected. For
  uv, VIRTUAL_ENV is passed via the port {env, ...} option rather than a shell prefix.
• Bounded shared state + safe stream/log builders - py_state gained an optional
  max_state_entries cap (default infinity, unchanged behavior) enforced with atomic
  admission so Python-driven state_set can't exhaust node memory, and its size counter
  is protected from corruption. The py:stream and logging helpers that build Python
  source now strictly validate module/function/kwarg names as identifiers (rejecting
  injection at positions where quoting is meaningless) and escape string-literal values
  including control characters.
• Validated event-loop fd handles - The asyncio reader/writer integration no longer
  hands Python a raw fd_resource pointer as an integer key. Each handle is an opaque id
  validated against a registry on every use, so a stale, duplicate, or fabricated id is a
  safe no-op (or clean error) instead of a double-free or arbitrary-pointer dereference
  that crashed the node. fd_read/fd_write also moved to dirty IO schedulers.
• OWN_GIL worker robustness (Python 3.14+) - A per-request allocation failure in
  a subinterpreter worker no longer breaks (and permanently kills) the worker command
  loop; it returns an error and keeps serving. The owngil_* dispatch NIFs now run on
  dirty IO schedulers and use non-blocking, deadline-bounded pipe reads and writes, so a
  stalled or dead worker can't wedge a scheduler forever. The internal SuspensionRequired
  exception is now looked up per-interpreter (like ProcessError), avoiding cross-
  interpreter object use under OWN_GIL.
• Callback suspend/resume lifetime hardening - The worker resource is now kept
  alive for the lifetime of a suspended callback (it could previously be GC'd mid-
  suspension, causing a use-after-free on resume). A resume frees any prior result
  before storing a new one (no leak/double-replay on a duplicate resume), the
  pending-callback thread-local is cleared at the worker request boundary, and the
  callback-response pipe writes run on dirty schedulers with non-blocking, deadline-
  bounded writes so a stalled reader or large payload can't wedge a scheduler or
  desync the framed protocol.
• Zero-copy buffer pinning - py_buffer no longer relocates (and frees) its
  storage while a Python memoryview points into it. A write that would grow the
  buffer while a view is held now returns an error instead of dangling the view into
  freed memory (a use-after-free that crashed the whole node).
• Bounded recursion in type conversion - The Erlang<->Python converters now cap
  nesting depth, so a deeply nested term (or Python structure) returns a clean error
  instead of overflowing the C stack and crashing the whole node.
• NULL-checked tuple allocation - Argument-tuple allocations in the call/eval paths
  are checked before use, and the Python->Erlang map conversion is bounded against
  mid-iteration dict mutation, closing two ways an allocation failure or re-entrant
  __str__ could corrupt memory.
• Safe term decoding at the NIF boundary - All enif_binary_to_term calls now
  pass ERL_NIF_BIN2TERM_SAFE, preventing attacker-influenced data (notably a Python
  "__etf__:<base64>" callback result) from minting new, non-GC'd atoms and exhausting
  the atom table. Local-node pids/refs and already-existing atoms still round-trip
  unchanged; only brand-new atoms, remote-node pids/refs, and external funs in
  Python-supplied payloads are now rejected.

Changed

• Support Erlang/OTP 28 and 29 - Validated builds and the full Common Test
  suite on OTP 28 and 29. Minimum supported OTP is now 28 (minimum_otp_vsn).
  CI tests OTP 28 and 29 across Python 3.12/3.13/3.14.
• Replaced deprecated catch Expr cleanup calls with try ... catch ... end
  to silence the new OTP 29 default warning; behavior is unchanged.

3.0.0 (2026-05-03)

Breaking Changes

• Simplified execution model - Only two public execution modes: worker and owngil

  • worker: Dedicated pthread per context with stable thread affinity (default)
  • owngil: Dedicated pthread + subinterpreter with own GIL (Python 3.14+)
  • Removed multi_executor and free_threaded from public API
  • Internal capability detection still tracks Python features

• Removed py:num_executors/0 - Contexts now use per-context worker threads
  instead of a shared executor pool. This function is no longer needed.

• py:execution_mode/0 returns worker | owngil - Based on the context_mode
  application configuration. Previously returned internal capabilities like
  free_threaded, subinterp, or multi_executor.

• Removed py:async_stream/3,4 - Streaming async generators was never
  implemented behind the API and always returned {error, stream_not_implemented}.
  Use py:stream_start/3,4 for sync generators; async-generator support may
  return in a later release.

• Removed num_executors / num_async_workers configuration - Both keys
  were no-ops after the v3.0 worker rework. Configure context count via
  num_contexts and the rate-limit ceiling via max_concurrent.

• Strict context-mode validation at the NIF boundary - py_nif:context_create/1
  now returns {error, {invalid_mode, Atom}} for anything other than worker | owngil.
  Previously, callers that bypassed py_context (notably py_reactor_context)
  silently mapped any unknown atom — including legacy auto and subinterp —
  to worker mode. Code that relied on that loophole must pass worker (or
  owngil) explicitly.

Fixed

• py:async_call/3,4 + py:async_await/1,2 round-trip - Previously the
  await receive matched {py_response, _, _} while the event loop sent
  {async_result, _, _}, causing every async call to silently time out.
  Async calls now go directly through py_event_loop:create_task and
  py_event_loop:await.

• py:async_gather/1,2 actually executes - Reimplemented as concurrent
  async_call submission with sequential async_await. Returns
  {ok, [Result1, ...]} on success or {error, {gather_failed, [{Idx, Reason}, ...]}}
  if any call fails. The previous implementation returned gather_not_implemented.

• Thread-callback flakes (issue #63) - Six layered defects in the
  erlang.call/erlang.async_call plumbing could deliver wrong values to
  the wrong caller under load. Reads now loop on partial/EINTR with a
  monotonic deadline; sync writes use a single length-prefixed frame on a
  dirty I/O scheduler with deadlined non-blocking writes; the sync wire
  carries the originating callback id and the receiver discards mismatched
  frames; the async pipe has one writer process per fd with an
  atomics-bounded mailbox (?ASYNC_WRITER_MAX_QUEUE = 10000) and a
  resumable nonblocking parser on the read end; workers that fail to
  resync are unlinked from the pool, freed, and bounded by
  MAX_POISONED_WORKERS = 64.

Documentation

• Audited every fenced code block in README.md and docs/*.md for
  current-API references. Fixed Py_GIL_OWN to PyInterpreterConfig_OWN_GIL
  in docs/scalability.md, corrected the multi_executor fallback claim
  in docs/migration.md, and repaired a broken SharedDict example in
  docs/shared-dict.md.
• New test/coverage_audit.md maps every public py:* and erlang.* API
  to its test suite. Added cases for py:cast/4, py:async_gather/2, and
  py:dup_fd/1 so each documented API has a regression test.
• New scripts/lint_doc_snippets.escript (driven by make lint-docs and
  CI) statically validates every Erlang py:Fn(/N) call and parses every
  Python block in the docs. Snippets that intentionally show removed APIs
  or REPL output opt out via <!-- skip-lint -->.

Changed

• Per-context worker threads - Each context now gets its own dedicated pthread
  that handles all Python operations. This provides stable thread affinity for
  numpy/torch/tensorflow compatibility without needing a shared executor pool.

• Async NIF dispatch - Context operations use async NIFs with message passing
  instead of blocking dirty schedulers. This improves concurrency under load.

• Request queue per context - Replaced single-slot request pattern with proper
  request queues that support multiple concurrent callers.

• No global asyncio policy install on Python 3.14+. asyncio.set_event_loop_policy
  was deprecated in 3.14 and is removed in 3.16. The Erlang integration's run path
  already uses loop_factory= (erlang.run/1, asyncio.Runner) so the global
  policy was only a convenience for bare asyncio.run() inside py:exec. We now
  skip the install on 3.14+ to avoid the deprecation warning. On 3.14+ use
  erlang.run(main) or asyncio.Runner(loop_factory=erlang.new_event_loop)
  explicitly. Behavior on Python 3.9–3.13 is unchanged. erlang.install() raises
  RuntimeError on 3.14+ (still emits a DeprecationWarning and works on 3.12–3.13).

Removed

• Multi-executor pool (g_executors[], multi_executor_start/stop)
• context_dispatch_call/eval/exec functions (dead code)
• References to PY_MODE_MULTI_EXECUTOR in context operations
• py_async_pool legacy gen_server (unused after async API rewire)
• priv/_erlang_impl/_ssl.py (SSLTransport, create_ssl_transport) had no
  importer and was never wired into the asyncio event loop. Removed.
• Internal py_util exports send_response/3, normalize_timeout/1, and
  normalize_timeout/2 had no callers anywhere. Removed. The module is
  marked @private; no external API changes.
• Explicit py:subinterp_* handle API removed. py:subinterp_create/0,
  subinterp_destroy/1, subinterp_call/4,5, subinterp_eval/2,3,
  subinterp_exec/2, subinterp_cast/4, subinterp_async_call/4,
  subinterp_await/1,2, and subinterp_pool_* are all gone. Use
  py_context:new(#{mode => owngil}) instead — it gives the same
  parallelism with OTP supervision and automatic cleanup.
  py:subinterp_supported/0 (capability probe) and py:parallel/1
  (which routes through the context API) stay.
• Internal py_execution_mode_t collapsed from 3 values to 2 (free_threaded
  / gil); py_nif:execution_mode/0 returns free_threaded | gil instead
  of the old free_threaded | subinterp | multi_executor.
• examples/reactor_owngil_example.erl deleted (called nonexistent
  py:subinterp_reactor_* functions; pre-existing breakage).

Changes

• Add executor affinity for numpy/torch thread safety
• Add shared-dict guide to hex docs

Removed

• ASGI/WSGI Support - The py_asgi and py_wsgi modules have been removed
  • py_asgi:run/4,5 - ASGI application runner
  • py_wsgi:run/3,4 - WSGI application runner
  • For web framework integration, use py:call with event loop contexts or the Channel API
  • See Migration Guide for alternatives

Added

• SharedDict - Process-scoped shared dictionaries for cross-process state
  • py:shared_dict_new/0 - Create a new SharedDict
  • py:shared_dict_get/2,3 - Get value with optional default
  • py:shared_dict_set/3 - Set key-value pair
  • py:shared_dict_del/2 - Delete a key
  • py:shared_dict_keys/1 - List all keys
  • py:shared_dict_destroy/1 - Explicit cleanup
  • Python access via erlang.SharedDict with dict-like interface
  • Mutex-protected for concurrent access (~300k ops/sec)
  • Pickle serialization for complex types
  • See SharedDict documentation for details

Added

• OWN_GIL Mode - True parallel Python execution with Python 3.14+ subinterpreters. Each subinterpreter runs with its own GIL in a dedicated thread, enabling true parallelism for CPU-bound workloads.

• Process-Bound Python Environments - Per-Erlang-process Python namespaces with isolated globals/locals that persist across calls.

• Event Loop Pool - py_event_loop_pool distributes async tasks with scheduler-affinity routing.

• ByteChannel API - Raw byte streaming without term serialization. Ideal for HTTP bodies, file streaming, binary protocols.

• PyBuffer API - Zero-copy buffer for WSGI input streams with file-like interface.

• True streaming API - py:stream_start/3,4 and py:stream_cancel/1 for event-driven streaming from Python generators.

• erlang.whereis(name) - Lookup registered Erlang PIDs from Python.

• erlang.schedule_inline(callback) - Inline continuation scheduling.

• py:spawn_call/3,4,5 - Fire-and-forget with result delivery.

• Explicit bytes conversion - {bytes, Binary} tuple for round-trip safety.

• Import caching API - py:import/1,2, py:add_import/1,2, py:add_path/1.

• Per-interpreter preload code - Execute code in new interpreters with inherited globals.

Fixed

• Channel notification for create_task
• Channel waiter race condition
• Event loop isolation and resource safety
• Python 3.14 venv activation
• OWN_GIL safety fixes (mutex leak, deadlock prevention, env validation)

Changed

• py:cast is now fire-and-forget (use py:spawn_call for results)
• OWN_GIL requires Python 3.14+
• Removed auto-started io pool
• Removed py_event_router
• Config-based initialization for imports/paths

Performance

• Direct NIF channel operations (up to 1760x speedup)
• nif_process_ready_tasks optimization (~15% improvement)

See CHANGELOG.md for full details.

Added

• Async Task API - uvloop-inspired task submission from Erlang

  • py_event_loop:run/3,4 - Blocking run of async Python functions
  • py_event_loop:create_task/3,4 - Non-blocking task submission with reference
  • py_event_loop:await/1,2 - Wait for task result with timeout
  • py_event_loop:spawn_task/3,4 - Fire-and-forget task execution
  • Thread-safe submission via enif_send (works from dirty schedulers)
  • See Async Task API docs

• erlang.spawn_task(coro) - Spawn async tasks from sync and async contexts

  • Works where asyncio.get_running_loop() fails
  • Returns asyncio.Task for optional await/cancel

• Explicit Scheduling API - Control dirty scheduler release from Python

  • erlang.schedule(callback, *args) - Release scheduler, continue via Erlang callback
  • erlang.schedule_py(module, func, args, kwargs) - Release scheduler, continue in Python
  • erlang.consume_time_slice(percent) - Check if NIF time slice exhausted
  • ScheduleMarker type for cooperative long-running tasks

• Distributed Python Execution - Run Python across Erlang nodes

  • Documentation and Docker-based demo
  • See Distributed Execution docs

Changed

• Event Loop Performance
  • Growable pending queue (256 to 16384)
  • Snapshot-detach pattern to reduce mutex contention
  • Callable cache (64 slots) avoids PyImport/GetAttr per task
  • Task wakeup coalescing

Fixed

• ensure_venv always installs deps, even if venv exists
• erlang.sleep() timing in sync context
• time() returns fresh value when loop not running
• Handle pooling bugs in ErlangEventLoop
• Task wakeup race causing batch task stalls