gh-103089: Colorize only the beginning of very long lines in IDLE - #157647

The Tk text widget itself handles million-character lines fine (with wrap='none', as in the editor), but adding a tag to a line takes time proportional to the number of tags already in the line: Tk walks the line's segment list for the index and then CleanupLine() scans the whole line. So colorizing a line with k tokens costs O(k²) — about 2.5 s for 7,000 tokens, and about half an hour for each of the ~200,000-token lines of the file from the issue. In addition, the colorizer resolved every position as chunk_head+Nc, through all segments of the previous lines of the chunk, and rescanned the whole line with the regular expression on every pass.

Now only the first 2000 characters of a line are colorized (tokens starting after that are skipped, and the scan jumps to the next line), positions are relative to the line start, and tags are added from the end to the start, which is about 1.5 times faster in Tk.

The file from the issue now opens in about 8 s instead of hours. Most of the remaining time is IndentSearcher tokenizing the whole 15 MB file in search of the first indented block; that is proportional to the file size, not the line length, and can be limited separately.

🤖 Generated with Claude Code

• Issue: IDLE editor crashes opening a file with million+ character lines #103089