Skip to content

host_env: os.replace for Windows - #8212

Merged
youknowone merged 1 commit into
RustPython:mainfrom
joshuamegnauth54:host_env-correct-rename-replace
Jul 10, 2026
Merged

host_env: os.replace for Windows#8212
youknowone merged 1 commit into
RustPython:mainfrom
joshuamegnauth54:host_env-correct-rename-replace

Conversation

@joshuamegnauth54

@joshuamegnauth54 joshuamegnauth54 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

rename and replace are the same for Unixes but different for Windows in Python. POSIX's rename atomically replaces its target; there is no replace in POSIX. renameat2 with RENAME_NOREPLACE atomically checks if a file exists and renames if it doesn't, but Python's rename and replace predate renameat2.

For Windows, rename acts like the RENAME_NOREPLACE flag while replace functions like rename. I implemented both using the Windows API. Both implementations follow CPython's code by using MoveFileExW. There are modern Windows APIs that may be worth using in the future. Rust's standard library prefers the modern APIs but falls back to MoveFileExW for deprecated Windows versions.

Summary

  • Implement os.replace for Windows.

Summary by CodeRabbit

  • New Features

    • Added broader cross-platform support for directory-aware file operations, including rename, replace, and directory creation on Unix-like, WASI, and Windows environments.
    • Improved handling of source and destination directory references for rename-style operations.
  • Bug Fixes

    • Fixed platform differences so file and directory operations behave more consistently across supported operating systems.
    • Updated directory creation and rename-related parameter handling for better compatibility.

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR consolidates POSIX/WASI filesystem helpers (make_dir, rename, replace) into a new shared posix_unix_like module, re-exported by posix.rs and posix_wasi.rs. The legacy host_env::os::rename wrapper is removed. Windows gains rename/replace via MoveFileExW, and mode_t is renamed to RawMode. The vm stdlib's DirFd becomes keyword-typed (via a new DirFdKeyword trait and SrcDirFd/DstDirFd/DefaultDirFd markers), and os.rename/replace are rewired to route through the new host-side APIs with proper src/dst dir-fd handling.

Changes

Rename and filesystem helper unification across platforms

Layer / File(s) Summary
Shared POSIX/WASI helpers
crates/host_env/src/lib.rs, crates/host_env/src/posix_unix_like.rs, crates/host_env/src/posix.rs, crates/host_env/src/posix_wasi.rs
New posix_unix_like module provides make_dir, rename, replace, and RawMode; it is exported from lib.rs and re-exported via wildcard from posix.rs and posix_wasi.rs, replacing their local make_dir implementations.
Remove legacy host_env rename
crates/host_env/src/os.rs
The platform-gated rename wrapper and its Unix-only AsFd import are deleted from os.rs.
Windows rename and replace
crates/host_env/src/posix_windows.rs
mode_t is renamed to RawMode; new rename and replace wrappers dispatch to a shared rename_impl using MoveFileExW, returning I/O errors on failure.
Keyword-aware DirFd
crates/vm/src/stdlib/os.rs
DirFd gains a KW: DirFdKeyword generic with PhantomData, backed by new DefaultDirFd/SrcDirFd/DstDirFd markers; from_args reads the keyword name dynamically; mkdir's mode type changes to RawMode.
Rename and replace routing
crates/vm/src/stdlib/os.rs
RENAME_DIR_FD gate widens to include WASI; RenameArgs uses typed SrcDirFd/DstDirFd fields; rename/replace extract dir-fds conditionally and call host_env::posix::rename/replace, including both filenames in errors.

Estimated code review effort: 4 (Complex) | ~50 minutes

Possibly related PRs

Suggested reviewers: ShaharNaveh, youknowone, coolreader18

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately captures the main change: adding Windows support for os.replace.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/host_env/src/posix_windows.rs`:
- Around line 34-36: The unused-variable lint suppression in posix_windows.rs is
using the wrong cfg predicate, so it applies in the wrong builds and can leave
expectations unmet. Update the cfg_attr on the parameters in the Windows POSIX
path helpers to use debug_assertions instead of debug_assert, and make the same
correction for the matching attributes around the later parameter block so the
`expect(unused_variables)` only applies in non-debug builds.
- Around line 60-66: The rename_impl helper has an inconsistent signature and
uses the wrong wide-string conversion for Windows path handling. Update
rename_impl so both from and to use impl AsRef<Path>, and switch the path
conversion to WideCString::from_os_str for each input before invoking
MoveFileExW. Keep the fix localized in rename_impl in posix_windows.rs and
ensure the resulting wide strings are passed through correctly.

In `@crates/vm/src/stdlib/os.rs`:
- Around line 81-84: The generic DirFd parsing currently accepts rename-specific
keywords, which causes the first flattened DirFd in RenameArgs to consume
dst_dir_fd or src_dir_fd incorrectly. Update the DirFd argument handling in
os.rs so it only reads dir_fd, and move src_dir_fd/dst_dir_fd parsing into the
RenameArgs-specific path that owns those two flattened fields. Use the DirFd and
RenameArgs parsing logic to ensure single-fd APIs reject rename-only keywords
and rename calls map each keyword to the correct field.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 31a0e703-2ab5-4cc1-8e15-f21cc06d3e8a

📥 Commits

Reviewing files that changed from the base of the PR and between 045a6d5 and 698bea5.

📒 Files selected for processing (7)
  • crates/host_env/src/lib.rs
  • crates/host_env/src/os.rs
  • crates/host_env/src/posix.rs
  • crates/host_env/src/posix_unix_wasi.rs
  • crates/host_env/src/posix_wasi.rs
  • crates/host_env/src/posix_windows.rs
  • crates/vm/src/stdlib/os.rs
💤 Files with no reviewable changes (1)
  • crates/host_env/src/os.rs

Comment thread crates/host_env/src/posix_windows.rs Outdated
Comment thread crates/host_env/src/posix_windows.rs Outdated
Comment thread crates/vm/src/stdlib/os.rs Outdated
@joshuamegnauth54
joshuamegnauth54 force-pushed the host_env-correct-rename-replace branch from 698bea5 to 41d2d58 Compare July 6, 2026 01:45
@joshuamegnauth54
joshuamegnauth54 marked this pull request as draft July 6, 2026 01:51
@joshuamegnauth54
joshuamegnauth54 marked this pull request as ready for review July 7, 2026 02:23
@joshuamegnauth54
joshuamegnauth54 force-pushed the host_env-correct-rename-replace branch from 41d2d58 to 99a6517 Compare July 7, 2026 02:23
@joshuamegnauth54
joshuamegnauth54 marked this pull request as draft July 7, 2026 02:26

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/vm/src/stdlib/os.rs (1)

1414-1468: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the shared rename/replace path handling.

rename and replace now duplicate conversion, dir-fd extraction, and OSError construction; only the function name and host operation differ.

♻️ Proposed refactor
+    fn rename_or_replace<'fd>(
+        function: &'static str,
+        args: RenameArgs<'fd>,
+        vm: &VirtualMachine,
+        op: impl FnOnce(
+            &OsPath,
+            Option<crt_fd::Borrowed<'fd>>,
+            &OsPath,
+            Option<crt_fd::Borrowed<'fd>>,
+        ) -> io::Result<()>,
+    ) -> PyResult<()> {
+        let src = PathConverter::new()
+            .function(function)
+            .argument("src")
+            .try_path(args.src, vm)?;
+        let dst = PathConverter::new()
+            .function(function)
+            .argument("dst")
+            .try_path(args.dst, vm)?;
+
+        #[cfg(any(unix, target_os = "wasi"))]
+        let src_dir_fd = args.src_dir_fd.get_opt();
+        #[cfg(not(any(unix, target_os = "wasi")))]
+        let src_dir_fd = None;
+
+        #[cfg(any(unix, target_os = "wasi"))]
+        let dst_dir_fd = args.dst_dir_fd.get_opt();
+        #[cfg(not(any(unix, target_os = "wasi")))]
+        let dst_dir_fd = None;
+
+        op(&src, src_dir_fd, &dst, dst_dir_fd).map_err(|err| {
+            let builder = err.to_os_error_builder(vm);
+            let builder = builder.filename(src.filename(vm));
+            let builder = builder.filename2(dst.filename(vm));
+            builder.build(vm).upcast()
+        })
+    }
+
     #[pyfunction]
     fn rename(args: RenameArgs<'_>, vm: &VirtualMachine) -> PyResult<()> {
-        let src = PathConverter::new()
-            .function("rename")
-            .argument("src")
-            .try_path(args.src, vm)?;
-        let dst = PathConverter::new()
-            .function("rename")
-            .argument("dst")
-            .try_path(args.dst, vm)?;
-
-        #[cfg(any(unix, target_os = "wasi"))]
-        let src_dir_fd = args.src_dir_fd.get_opt();
-        #[cfg(not(any(unix, target_os = "wasi")))]
-        let src_dir_fd = None;
-
-        #[cfg(any(unix, target_os = "wasi"))]
-        let dst_dir_fd = args.dst_dir_fd.get_opt();
-        #[cfg(not(any(unix, target_os = "wasi")))]
-        let dst_dir_fd = None;
-
-        crate::host_env::posix::rename(&src, src_dir_fd, &dst, dst_dir_fd).map_err(|err| {
-            let builder = err.to_os_error_builder(vm);
-            let builder = builder.filename(src.filename(vm));
-            let builder = builder.filename2(dst.filename(vm));
-            builder.build(vm).upcast()
-        })
+        rename_or_replace("rename", args, vm, crate::host_env::posix::rename)
     }
 
     #[pyfunction]
     fn replace(args: RenameArgs<'_>, vm: &VirtualMachine) -> PyResult<()> {
-        let src = PathConverter::new()
-            .function("replace")
-            .argument("src")
-            .try_path(args.src, vm)?;
-        let dst = PathConverter::new()
-            .function("replace")
-            .argument("dst")
-            .try_path(args.dst, vm)?;
-
-        #[cfg(any(unix, target_os = "wasi"))]
-        let src_dir_fd = args.src_dir_fd.get_opt();
-        #[cfg(not(any(unix, target_os = "wasi")))]
-        let src_dir_fd = None;
-
-        #[cfg(any(unix, target_os = "wasi"))]
-        let dst_dir_fd = args.dst_dir_fd.get_opt();
-        #[cfg(not(any(unix, target_os = "wasi")))]
-        let dst_dir_fd = None;
-
-        crate::host_env::posix::replace(&src, src_dir_fd, &dst, dst_dir_fd).map_err(|err| {
-            let builder = err.to_os_error_builder(vm);
-            let builder = builder.filename(src.filename(vm));
-            let builder = builder.filename2(dst.filename(vm));
-            builder.build(vm).upcast()
-        })
+        rename_or_replace("replace", args, vm, crate::host_env::posix::replace)
     }

As per coding guidelines, “When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/vm/src/stdlib/os.rs` around lines 1414 - 1468, The rename and replace
stdlib functions duplicate the same path conversion, dir-fd extraction, and
OSError mapping logic in os.rs; extract the shared flow into a helper used by
both `rename` and `replace`, with only the operation-specific parts differing.
Keep `PathConverter`, `RenameArgs`, and the `crate::host_env::posix::{rename,
replace}` calls as the unique inputs, and centralize the error builder setup so
filename handling and `to_os_error_builder` are not repeated.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/vm/src/stdlib/os.rs`:
- Around line 1414-1468: The rename and replace stdlib functions duplicate the
same path conversion, dir-fd extraction, and OSError mapping logic in os.rs;
extract the shared flow into a helper used by both `rename` and `replace`, with
only the operation-specific parts differing. Keep `PathConverter`, `RenameArgs`,
and the `crate::host_env::posix::{rename, replace}` calls as the unique inputs,
and centralize the error builder setup so filename handling and
`to_os_error_builder` are not repeated.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 527af374-e782-4110-8c34-e7a6458ffe4a

📥 Commits

Reviewing files that changed from the base of the PR and between 41d2d58 and 99a6517.

📒 Files selected for processing (7)
  • crates/host_env/src/lib.rs
  • crates/host_env/src/os.rs
  • crates/host_env/src/posix.rs
  • crates/host_env/src/posix_unix_wasi.rs
  • crates/host_env/src/posix_wasi.rs
  • crates/host_env/src/posix_windows.rs
  • crates/vm/src/stdlib/os.rs
💤 Files with no reviewable changes (1)
  • crates/host_env/src/os.rs
✅ Files skipped from review due to trivial changes (1)
  • crates/host_env/src/lib.rs
🚧 Files skipped from review as they are similar to previous changes (4)
  • crates/host_env/src/posix_unix_wasi.rs
  • crates/host_env/src/posix_wasi.rs
  • crates/host_env/src/posix.rs
  • crates/host_env/src/posix_windows.rs

@joshuamegnauth54
joshuamegnauth54 force-pushed the host_env-correct-rename-replace branch 3 times, most recently from fac030e to 01a2be5 Compare July 8, 2026 04:13
@joshuamegnauth54
joshuamegnauth54 marked this pull request as ready for review July 8, 2026 04:27
Comment thread crates/host_env/src/posix_unix_wasi.rs Outdated
@@ -0,0 +1,47 @@
//! Common POSIX implementations across Unix and WASI.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if unix and wasi share implementation, mostly it will be shared between any kind of unix-like platforms. I suggset to use posix_unix_like instead of posix_unix_wasi

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I'll rename it. 😁

@joshuamegnauth54
joshuamegnauth54 force-pushed the host_env-correct-rename-replace branch from 01a2be5 to 7118c76 Compare July 8, 2026 17:57

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/host_env/src/posix_unix_like.rs (1)

12-19: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider using impl AsRef<Path> for consistency with rename/replace.

make_dir takes path: &impl AsRef<Path> while rename and replace use impl AsRef<Path>. Both forms accept the downstream &PathBuf argument, but impl AsRef<Path> is strictly more general and would make the file's API style uniform.

♻️ Proposed refactor
 pub fn make_dir(
     dir_fd: Option<crt_fd::Borrowed<'_>>,
-    path: &impl AsRef<Path>,
+    path: impl AsRef<Path>,
     mode: fs::RawMode,
 ) -> io::Result<()> {
     let dir_fd = dir_fd.as_ref().map_or(fs::CWD, AsFd::as_fd);
     fs::mkdirat(dir_fd, path.as_ref(), mode.into()).map_err(Into::into)
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/host_env/src/posix_unix_like.rs` around lines 12 - 19, Update make_dir
in posix_unix_like.rs to take path as impl AsRef<Path> rather than &impl
AsRef<Path>, matching the API style used by rename and replace. Keep the
existing mkdirat call and path.as_ref() usage the same, and adjust any local
call sites only if needed so the signature remains consistent and more general.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/host_env/src/posix_unix_like.rs`:
- Around line 12-19: Update make_dir in posix_unix_like.rs to take path as impl
AsRef<Path> rather than &impl AsRef<Path>, matching the API style used by rename
and replace. Keep the existing mkdirat call and path.as_ref() usage the same,
and adjust any local call sites only if needed so the signature remains
consistent and more general.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 458f73c8-4e54-460a-8021-c28a94da4db1

📥 Commits

Reviewing files that changed from the base of the PR and between 01a2be5 and 7118c76.

📒 Files selected for processing (7)
  • crates/host_env/src/lib.rs
  • crates/host_env/src/os.rs
  • crates/host_env/src/posix.rs
  • crates/host_env/src/posix_unix_like.rs
  • crates/host_env/src/posix_wasi.rs
  • crates/host_env/src/posix_windows.rs
  • crates/vm/src/stdlib/os.rs
💤 Files with no reviewable changes (1)
  • crates/host_env/src/os.rs
✅ Files skipped from review due to trivial changes (1)
  • crates/host_env/src/lib.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/host_env/src/posix_windows.rs
  • crates/vm/src/stdlib/os.rs

`rename` and `replace` are the same for Unixes but different for Windows
in Python. POSIX's `rename` atomically replaces its target; there is no
`replace` in POSIX. `renameat2` with `RENAME_NOREPLACE` atomically
checks if a file exists and renames if it doesn't, but Python's `rename`
and `replace` predate `renameat2`.

For Windows, `rename` acts like the `RENAME_NOREPLACE` flag while
`replace` functions like `rename`. I implemented both using the Windows
API. Both implementations follow CPython's code by using `MoveFileExW`.
There are modern Windows APIs that may be worth using in the future.
Rust's standard library prefers the modern APIs but falls back to
`MoveFileExW` for deprecated Windows versions.
@joshuamegnauth54
joshuamegnauth54 force-pushed the host_env-correct-rename-replace branch from 7118c76 to dd8354c Compare July 8, 2026 18:15

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@youknowone
youknowone merged commit 9c064c1 into RustPython:main Jul 10, 2026
26 checks passed
@joshuamegnauth54
joshuamegnauth54 deleted the host_env-correct-rename-replace branch July 10, 2026 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants