Skip to content

Stop RebufferedBytesIO from hanging at EOF (fixes #988) - #1140

Open
binggao1230 wants to merge 1 commit into
construct:masterfrom
binggao1230:fix-988-rebuffered-eof-hang
Open

binggao1230 wants to merge 1 commit into
construct:masterfrom
binggao1230:fix-988-rebuffered-eof-hang

Conversation

@binggao1230

Copy link
Copy Markdown

Problem

RebufferedBytesIO.read (and write) poll the underlying substream in a loop. They treated two distinct outcomes identically:

  • None — a non-blocking stream that has no data yet
  • b"" — the stream has reached end of file

Both hit if not newdata: sleep(0); continue, so a substream that signals EOF by returning empty bytes caused construct to spin forever instead of stopping. This is the hang reported in #988.

from construct.lib import RebufferedBytesIO
import io

rb = RebufferedBytesIO(io.BytesIO(b"abc"))
rb.read(3)   # b"abc"
rb.read(1)   # hangs forever before this fix

Fix

Distinguish the two cases:

  • newdata is None → still keep polling (sleep(0); continue), preserving non-blocking behaviour.
  • empty bytes → break out of the loop.

After breaking, read falls through to its existing could not read enough bytes IOError, and write stops trying to backfill. The same fix is applied to both methods (the write loop had the same pattern and could also spin).

Tests

Added test_rebuffered_reading_past_eof which asserts an over-read now raises IOError rather than hanging. Existing test_bitstream.py tests still pass.

RebufferedBytesIO.read/write polled the substream in a loop and treated
both None (non-blocking would-block) and empty bytes (EOF) the same way:
sleep(0) then keep looping. A stream that returned b'' at end of file
therefore spun forever instead of stopping.

Distinguish the two cases: keep polling on None, but break on empty bytes
so read() falls through to its 'could not read enough bytes' IOError and
write() stops trying to backfill. Fixes construct#988.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant