feat: Add the file and sync_thunk crates by geeknoid · Pull Request #295 · microsoft/oxidizer

geeknoid · 2026-03-01T19:20:03Z

No description provided.

github-actions · 2026-03-01T19:22:23Z

⚠️ Breaking Changes Detected

error: failed to retrieve local crate data from git revision

Caused by:
    0: failed to retrieve manifest file from git revision source
    1: possibly due to errors: [
         failed to parse /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/75219a112dc13c40fdadc878a4ae3ed38b5da0e7/Cargo.toml: no `package` table,
         failed when reading /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/75219a112dc13c40fdadc878a4ae3ed38b5da0e7/scripts/crate-template/Cargo.toml: TOML parse error at line 9, column 26
         |
       9 | keywords = ["oxidizer", {{CRATE_KEYWORDS}}]
         |                          ^
       missing key for inline table element, expected key
       : TOML parse error at line 9, column 26
         |
       9 | keywords = ["oxidizer", {{CRATE_KEYWORDS}}]
         |                          ^
       missing key for inline table element, expected key
       ,
       ]
    2: package `file` not found in /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/75219a112dc13c40fdadc878a4ae3ed38b5da0e7

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: cargo_semver_checks::rustdoc_gen::RustdocFromProjectRoot::get_crate_source
   2: cargo_semver_checks::rustdoc_gen::StatefulRustdocGenerator<cargo_semver_checks::rustdoc_gen::CoupledState>::prepare_generator
   3: cargo_semver_checks::Check::check_release::{{closure}}
   4: cargo_semver_checks::Check::check_release
   5: cargo_semver_checks::exit_on_error
   6: cargo_semver_checks::main
   7: std::sys::backtrace::__rust_begin_short_backtrace
   8: main
   9: <unknown>
  10: __libc_start_main
  11: _start

If the breaking changes are intentional then everything is fine - this message is merely informative.

Remember to apply a version number bump with the correct severity when publishing a version with breaking changes (1.x.x -> 2.x.x or 0.1.x -> 0.2.x).

codecov · 2026-03-01T19:23:15Z

Codecov Report

❌ Patch coverage is 85.61475% with 351 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.9%. Comparing base (87033bf) to head (382ed76).

Files with missing lines	Patch %	Lines
crates/file/src/file.rs	71.2%	46 Missing ⚠️
crates/file/src/directory.rs	80.2%	41 Missing ⚠️
crates/file/src/write_only_positional_file.rs	53.5%	39 Missing ⚠️
crates/file/src/positional_file.rs	70.6%	37 Missing ⚠️
crates/file/src/read_only_positional_file.rs	61.7%	31 Missing ⚠️
crates/file/src/open_options.rs	65.3%	27 Missing ⚠️
crates/file/src/write_only_file.rs	74.2%	27 Missing ⚠️
crates/file/src/positional_file_inner.rs	91.2%	26 Missing ⚠️
crates/sync_thunk/src/stack_state.rs	87.4%	23 Missing ⚠️
crates/file/src/read_only_file.rs	79.2%	22 Missing ⚠️
... and 8 more

❌ Your project check has failed because the head coverage (96.9%) is below the target coverage (100.0%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #295     +/-   ##
========================================
- Coverage   100.0%   96.9%   -3.1%     
========================================
  Files         144     168     +24     
  Lines        8918   11358   +2440     
========================================
+ Hits         8918   11007   +2089     
- Misses          0     351    +351

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Adds a new file crate providing a capability-scoped, zero-copy, fully-async filesystem API (plus docs/examples/benchmarks) and wires it into the workspace.

Changes:

Introduces the crates/file library (Directory/Root capabilities, typed file handles, dispatcher, path sandboxing, etc.).
Adds extensive integration tests, examples, and a benchmark suite for ecosystem performance comparison.
Updates workspace metadata/docs (root README, changelog index, spelling dictionary, workspace Cargo deps).

Reviewed changes

Copilot reviewed 34 out of 36 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
crates/seatbelt/src/context.rs	Import formatting cleanup (no functional change).
crates/file/tests/integration.rs	Adds large integration test suite for `file` APIs.
crates/file/src/write_only_file.rs	Implements `WriteOnlyFile` typed handle and traits.
crates/file/src/shared_memory.rs	Adds type-erased, cloneable memory provider wrapper.
crates/file/src/root.rs	Adds `Root::bind` entrypoint for capability-based access.
crates/file/src/read_write_file.rs	Implements `ReadWriteFile` typed handle + rich read/write APIs.
crates/file/src/read_only_file.rs	Implements `ReadOnlyFile` typed handle + rich read APIs.
crates/file/src/read_dir.rs	Adds async directory iterator wrapper (`ReadDir`).
crates/file/src/path_utils.rs	Adds `safe_join` lexical sandboxing for relative paths.
crates/file/src/open_options.rs	Adds async `OpenOptions` builder for opening `ReadWriteFile`.
crates/file/src/lib.rs	Crate-level API surface + docs + re-exports.
crates/file/src/file_inner.rs	Shared file implementation: dispatching, locking, positional IO, buffer ops.
crates/file/src/dispatcher.rs	Adds small, auto-scaling worker pool for blocking FS calls.
crates/file/src/directory.rs	Implements capability-scoped directory operations and convenience read/write.
crates/file/src/dir_entry.rs	Adds capability-friendly `DirEntry` without revealing full paths.
crates/file/src/dir_builder.rs	Adds async `DirBuilder` capability-scoped directory creation.
crates/file/logo.png	Adds crate logo (Git LFS).
crates/file/favicon.ico	Adds crate favicon (Git LFS).
crates/file/examples/streaming_io.rs	Adds example for chunked streaming read/write.
crates/file/examples/positional_io.rs	Adds example for positional read/write APIs.
crates/file/examples/open_options.rs	Adds example for `OpenOptions` usage.
crates/file/examples/file_types.rs	Adds example for typed file handles and narrowing.
crates/file/examples/directory_ops.rs	Adds example for directory operations and iteration.
crates/file/examples/basic_read_write.rs	Adds basic bind/read/write example.
crates/file/benches/fs_comparison.rs	Adds Criterion benchmarks comparing against other crates.
crates/file/TODO.md	Adds design TODOs/questions for the new crate.
crates/file/README.md	Adds crate README (cargo-doc2readme generated).
crates/file/ECOSYSTEM-ANALYSIS.md	Adds long-form ecosystem comparison doc.
crates/file/Cargo.toml	Adds crate manifest (features, deps, examples, benches).
crates/file/CHANGELOG.md	Adds crate changelog stub.
crates/anyspawn/src/spawner.rs	Doc comment whitespace tweak.
README.md	Lists new `file` crate in workspace README.
Cargo.toml	Registers `file` crate + adds shared deps used by it.
CHANGELOG.md	Adds `file` crate to changelog index.
.spelling	Adds new allowed technical terms used in docs.

Comments suppressed due to low confidence (2)

crates/file/tests/integration.rs:1

This test hard-codes a Unix-specific /tmp/... path and will fail (or behave unexpectedly) on Windows. Consider generating a guaranteed-nonexistent path from a TempDir (e.g., tmp.path().join(\"definitely_not_here\")) or using platform-appropriate temp directory APIs to keep the test cross-platform.
crates/file/tests/integration.rs:1
This test accepts both success and failure as valid outcomes, which makes it non-assertive and can hide regressions. It’d be better to pick and enforce one behavior (ideally: succeed, since lexical normalization keeps the path within the capability), or split into two explicit tests depending on the intended contract of safe_join/path handling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

crates/file/src/dispatcher.rs

crates/file/src/file_inner.rs

Copilot · 2026-03-01T20:18:50Z

crates/file/src/directory.rs

+                let mut file = File::open(&full_path)?;
+                let len = usize::try_from(file.metadata()?.len()).unwrap_or(usize::MAX);
+                let mut buf = memory.reserve(len);
+                let mut total = 0;
+                while total < len {
+                    let n = read_into_buf(&mut file, &mut buf, len - total)?;
+                    if n == 0 {
+                        break;
+                    }
+                    total += n;
+                }
+                Ok(buf.consume_all())


If metadata().len() doesn’t fit in usize (notably on 32-bit targets, or very large files), this falls back to usize::MAX and then tries to reserve that much memory, which can lead to immediate OOM/panic. Prefer either returning an explicit error when the file is too large to buffer into memory, or switching to a chunked growth strategy (reserve a reasonable initial size, then grow as reads succeed) without attempting usize::MAX upfront.

crates/file/examples/basic_read_write.rs

sandersaares · 2026-03-03T08:46:21Z

crates/file/examples/file_types.rs

+    // WriteOnlyFile — can write, cannot read.
+    let mut wf = WriteOnlyFile::create(&dir, "data.txt").await?;
+    let mut buf = wf.reserve(64);
+    buf.put_slice(*b"written via WriteOnlyFile");


Can be shortened slightly as BytesView::copied_from_slice(b"blah", &wf)

sandersaares · 2026-03-03T08:49:51Z

crates/file/examples/file_types.rs

+    let mut buf = wf.reserve(64);
+    buf.put_slice(*b"written via WriteOnlyFile");
+    wf.write(buf.consume_all()).await?;
+    wf.flush().await?;


What are flush() semantics here? Is it just a pass-through to OS "flush" APIs? Or does it imply that the file types have some inherent internal buffers in the Rust universe? I assume the former but we should be careful not to give the wrong impression here.
We should also try support generic buffering layers built on Read and Write traits using something like BufRead, BufWrite and that we already have in the private repo, as well as via ConcurrentRead and ConcurrentWrite (ref ConcurrentBufRead in private repo) perhaps along the lines of #292 (comments welcome).

sandersaares · 2026-03-03T08:50:44Z

crates/file/examples/open_options.rs

+    let dir = Root::bind(tmp.path()).await?;
+
+    // Create a new file for reading and writing.
+    let mut rw = OpenOptions::new()


File sharing options would also be useful to have and demonstrate (are concurrent readers/writers allowed).

sandersaares · 2026-03-03T09:11:21Z

crates/file/examples/positional_types.rs

+    let dir = Root::bind(tmp.path()).await?;
+
+    // WriteOnlyPositionalFile — write-only access at explicit offsets.
+    let wf = WriteOnlyPositionalFile::create(&dir, "pos.txt").await?;


We should try ensure that this still remains compatible with Read, Write, ConcurrentRead, ConcurrentWrite traits. I suppose that could be layered on as an "operation" layer?

Something like fn write_from(&self, offset) -> WriteHead<'a> which implements Write and ConcurrentWrite for example, incrementing the offset internally as writes are accumulated?

This might also be a simple way to structure the forward-only file types internally, as limiting layers over the positional ones? If the wiring hooks up right.

sandersaares · 2026-03-03T09:14:21Z

crates/file/examples/streaming_io.rs

+    let mut rf = ReadOnlyFile::open(&dir, "stream.bin").await?;
+    let mut total = 0usize;
+    loop {
+        let chunk = rf.read_max(16).await?;


Not entirely sure what semantics of read_max are based on the example but is this not just reinventing stuff from ReadExt or Read traits? It is not clear to me why this is not just using the traits from bytesbuf_io for I/O functions.

geeknoid changed the title ~~Add file crate~~ feat: Add file crate Mar 1, 2026

geeknoid force-pushed the file branch 2 times, most recently from 2e7d9f7 to 5419b43 Compare March 1, 2026 19:54

geeknoid requested review from Copilot and sandersaares March 1, 2026 20:12

geeknoid force-pushed the file branch from 5419b43 to 1626a3b Compare March 1, 2026 20:13

Copilot AI reviewed Mar 1, 2026

View reviewed changes

Copilot started reviewing on behalf of geeknoid March 1, 2026 20:19 View session

geeknoid force-pushed the file branch 17 times, most recently from a6f2b67 to 9e77769 Compare March 2, 2026 14:39

sandersaares reviewed Mar 3, 2026

View reviewed changes

geeknoid force-pushed the file branch from 9e77769 to 735eb05 Compare March 3, 2026 14:30

geeknoid changed the title ~~feat: Add file crate~~ feat: Add the file and sync_thunk crates Mar 3, 2026

geeknoid force-pushed the file branch 5 times, most recently from 46e5da2 to 7757211 Compare March 3, 2026 18:18

feat: Add the file and sync_thunk crates

382ed76

geeknoid force-pushed the file branch from 7757211 to 382ed76 Compare March 3, 2026 19:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add the file and sync_thunk crates#295

feat: Add the file and sync_thunk crates#295
geeknoid wants to merge 1 commit intomainfrom
file

geeknoid commented Mar 1, 2026

Uh oh!

github-actions bot commented Mar 1, 2026 •

edited

Loading

Uh oh!

codecov bot commented Mar 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 1, 2026

Uh oh!

Uh oh!

sandersaares Mar 3, 2026

Uh oh!

sandersaares Mar 3, 2026

Uh oh!

sandersaares Mar 3, 2026

Uh oh!

sandersaares Mar 3, 2026

Uh oh!

sandersaares Mar 3, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

geeknoid commented Mar 1, 2026

Uh oh!

github-actions bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Breaking Changes Detected

Uh oh!

codecov bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sandersaares Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

sandersaares Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

sandersaares Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

sandersaares Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

sandersaares Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Mar 1, 2026 •

edited

Loading

codecov bot commented Mar 1, 2026 •

edited

Loading

sandersaares Mar 3, 2026 •

edited

Loading