Tracking Issue for directory handles

[the8472]

View all comments

Feature gate: #![feature(dirfd)]

This is a tracking issue for directory handles. Such handles provide a stable reference to an underlying filesystem object (typically directories) that are less vulnerable to TOCTOU attacks and similar races. These security properties will be platform-dependent. Platforms that don't provide the necessary primitives will fall back to operations on absolute paths.

Additionally they may also provide performance benefits by avoiding repeated path lookups when performing many operations on a directory.

Sandboxing is a non-goal. If a platform supports upwards path traversal via .. or symlinks then directory handles will not prevent that. Providing O_BENEATH-style traversal is left to 3rd-party crates or future extensions.

Public API

impl Dir {
     pub fn open<P: AsRef<Path>>(path: P) -> Result<Self>;
     pub fn metadata(&self) -> Result<Metadata>;
     /// This could be put on OpenOptions instead
     pub fn open_with<P: AsRef<Path>>(&self, path: P, options: &OpenOptions) -> Result<File>;
     pub fn create_dir<P: AsRef<Path>>(&self, path: P) -> Result<()>;
     pub fn rename<P: AsRef<Path>, Q: AsRef<Path>>(&self, from: P, to_dir: &Self, to: Q) -> Result<()>;
     pub fn remove_file<P: AsRef<Path>>(&self, path: P) -> Result<()>;
     pub fn remove_dir<P: AsRef<Path>>(&self, path: P) -> Result<()>;
     pub fn symlink<P: AsRef<Path>, Q: AsRef<Path>>(&self, original: P, link: Q);
     
     /// ... more convenience methods
}

impl DirEntry {
    pub fn open(&self) -> Result<File>
    /// This could be put on OpenOptions instead
    pub fn open_with(&self, options: &OpenOptions) -> Result<File>
    pub fn remove_file(&self) -> Result<()>
    pub fn remove_dir(&self) -> Result<()>
}

Steps / History

• ACP: Add openat/unlinkat/etc. abstractions to ReadDir/DirEntry/OpenOptions libs-team#259
• minimal dirfd implementation (1/4) #146341
• dirfd file operations (2/4) #150679
• directory handles: add Dir::metadata #157192
• getdents to get free conversion between dirfds and ReadDir
• add more *at calls
• Final comment period (FCP)¹
• Stabilization PR

Unresolved Questions

• AdRawFd can only be implemented on platforms that use dirfd, not all fd platforms. Is this fine?
• Should the Dir operations that take paths accept absolute paths (where self is effectively irrelevant)?

Footnotes

1. https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

[the8472]

During the ACP a survey of platform support was requested to make sure that fallbacks won't be needed on Tier-1 platforms.

libc/syscall survey

For restricted conversion between *DIR and a file descriptor we need fdopendir + dirfd. For free conversion we need an alternative to readdir that operates on file descriptors.

Presence of *at functions was surveyed by looking at the libc crate, so this may be incomplete.

Tier 1

Windows

• openat: NtCreateFile
• readdir: GetFileInformationByHandleEx + FILE_FULL_DIR_INFO.
• renameat:SetFileInformationByHandle + FILE_RENAME_INFORMATION
• unlinkat: SetFileInformationByHandle + FILE_DISPOSITION_INFO_EX
• mkdirat: NtCreateFile
• symlinkat: DeviceIoControl + FSCTL_SET_REPARSE_POINT
• fstatat: either NtQueryInformationByName (if available) or GetFileInformationByHandle and GetFileInformationByHandleEx + FILE_ATTRIBUTE_TAG_INFO

Linux

• readdir: getdents
• has fdopendir, dirfd, openat, renameat, unlinkat, mkdirat, symlinkat, fstatat

macos

• readdir: getattrlistbulk
• has fdopendir, dirfd, openat, renameat, unlinkat, mkdirat, symlinkat, fstatat

Tier 2

freebsd, illumos, netbsd, solaris

• readdir: getdents
• have fdopendir, dirfd, openat, renameat, unlinkat, mkdirat, symlinkat, fstatat

fuchsia

• readdir: ReadDirents
• has fdopendir, dirfd, openat, renameat, unlinkat, mkdirat, symlinkat, fstatat

ios

• readdir: getattrlistbulk
• has fdopendir, dirfd, openat, renameat, unlinkat, mkdirat, symlinkat, fstatat

wasm-emscripten

• readdir: getdents
• has fdopendir, dirfd, openat, renameat, unlinkat, mkdirat, symlinkat, fstatat

wasm-wasi

• readdir: fd_readdir
• has fdopendir, dirfd, openat, renameat, unlinkat, mkdirat, symlinkat, fstatat

redox

libc only lists renameat, unlinkat, symlinkat, fstatat

The redox syscall crate lists no *at calls at all?

uefi

Listed as no_std in the platform support documentation. Somehow has some pieces in std anyway. No libc support either.

Not checked

• tier 3 targets
• fortanix

[jackpot51]

Regarding redox, relibc is the right place to look for support. We should be able to support the same list of functions as Linux and the BSDs.

[zopsicle]

The ACP mentions “it may be possible to add the Dir methods directly to ReadDir instead”. This is problematic because ReadDir: !From<OwnedFd> (#56777). Being able to open a file with File::open, then check its file type using Metadata::file_type and convert it to a Dir object if it is a directory would be very useful IMO, for instance to hash a tree of files. That said it may be better to just address #56777 than to have the design of #120426 hinge on this.

[RalfJung]

From a Miri perspective I'd love to see this, it would let us implement support for emulation of openat and similar functions in a cross-platform way. :)

portable, insecure openat emulation based on Paths

I am somewhat surprised to see this TODO item. In other areas Rust chose to not provide lower-quality fallback implementations, e.g. famously Rust atomics are always hardware atomics and never lock emulated. I can't find discussion of this point in the ACP either. What is the reasoning here for preferring an insecure emulation over a security guarantee stated in the API?

[the8472]

Operating on a directory is a fairly basic operation, like opening a File. If we returned an error that basically says "we could do that, just the way you're opening it is unsupported on this system" then it would become less portable and likely only interesting for tools like sudo which won't support such platforms anyway.

Dir isn't just about security. It provides a handle to a directory that should keep working even if the directory gets renamed (granted, fallbacks don't provide this). It helps with performance. It allows replacing global process state (the current working directory) with local state. It can prevent non-malicious races.
So applications that aren't security-sensitive would still want to use Dir in their filesystem code.

Additionally, the platforms where openat isn't available are more likely to be single-user systems without privilege separation, on those fallbacks wouldn't be an issue anyway. Afaik UEFI has none of that.

That said, maybe we can add a fail_closed flag, impl Dir { const IS_RACEFREE: bool } or something like that for applications that need some extra assurances.

Also, this wouldn't be a runtime fallback but a platform-based fallback. All tier-1 platforms will use real handles and fail if they're not available. E.g. if something blocks openat via seccomp on linux I do not intend to try open.

[addisoncrump]

Also, this wouldn't be a runtime fallback but a platform-based fallback.

Can this be completed with a compiler warning for these platforms? Since this will be something included in the target type, I would imagine so.

[the8472]

@rustbot ping fuchsia

Since it appears that functions being available in the libc crate is not a reliable indicatior that they actually work, is there a better overview of the available posix APIs? I'm interested in the file descriptor APIs listed in #120426 (comment)