Skip to content

csv: support non-ASCII Unicode dialect characters #8408

Description

@coderabbitai

Summary

Add support for single non-ASCII Unicode code points in CSV dialect character attributes, matching CPython behavior.

Rationale

csv dialect options such as delimiter, quotechar, and escapechar accept exactly one Unicode character in CPython. RustPython currently stores these attributes as u8 in crates/stdlib/src/csv.rs, so valid one-character values outside the byte range (for example, ) are rejected.

Affected area

  • crates/stdlib/src/csv.rs
  • CSV dialect parsing, validation, reader, and writer configuration

Required changes

Replace the byte-only representation and parsing path for dialect characters with a representation that preserves a full Unicode code point, then propagate that support through the CSV reader and writer implementation. Keep existing validation that rejects empty and multi-character values, while reporting conversion/representation failures accurately.

Acceptance criteria

  • delimiter, quotechar, and escapechar accept a single non-ASCII Unicode code point such as .
  • Empty and multi-character strings remain rejected according to CPython-compatible behavior.
  • Reader and writer behavior correctly handles the supported Unicode dialect characters.
  • Relevant RustPython CSV tests cover successful non-ASCII dialect-character usage and invalid lengths.

Backlinks

Requested by @hyoinandout.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions