Summary
Add support for single non-ASCII Unicode code points in CSV dialect character attributes, matching CPython behavior.
Rationale
csv dialect options such as delimiter, quotechar, and escapechar accept exactly one Unicode character in CPython. RustPython currently stores these attributes as u8 in crates/stdlib/src/csv.rs, so valid one-character values outside the byte range (for example, €) are rejected.
Affected area
crates/stdlib/src/csv.rs
- CSV dialect parsing, validation, reader, and writer configuration
Required changes
Replace the byte-only representation and parsing path for dialect characters with a representation that preserves a full Unicode code point, then propagate that support through the CSV reader and writer implementation. Keep existing validation that rejects empty and multi-character values, while reporting conversion/representation failures accurately.
Acceptance criteria
delimiter, quotechar, and escapechar accept a single non-ASCII Unicode code point such as €.
- Empty and multi-character strings remain rejected according to CPython-compatible behavior.
- Reader and writer behavior correctly handles the supported Unicode dialect characters.
- Relevant RustPython CSV tests cover successful non-ASCII dialect-character usage and invalid lengths.
Backlinks
Requested by @hyoinandout.
Summary
Add support for single non-ASCII Unicode code points in CSV dialect character attributes, matching CPython behavior.
Rationale
csvdialect options such asdelimiter,quotechar, andescapecharaccept exactly one Unicode character in CPython. RustPython currently stores these attributes asu8incrates/stdlib/src/csv.rs, so valid one-character values outside the byte range (for example,€) are rejected.Affected area
crates/stdlib/src/csv.rsRequired changes
Replace the byte-only representation and parsing path for dialect characters with a representation that preserves a full Unicode code point, then propagate that support through the CSV reader and writer implementation. Keep existing validation that rejects empty and multi-character values, while reporting conversion/representation failures accurately.
Acceptance criteria
delimiter,quotechar, andescapecharaccept a single non-ASCII Unicode code point such as€.Backlinks
Requested by @hyoinandout.