rust libc adoption fails to build on OpenBSD

[Darkshadow2]

@ridiculousfish The rust libc changes fail to build under OpenBSD 7.7. Build fails on several of the RLIMIT_* variables and _CS_PATH:

error[E0425]: cannot find value `RLIMIT_AS` in crate `libc`
   --> src/builtins/ulimit.rs:26:43
    |
26  |         pub const AS: libc::c_int = libc::RLIMIT_AS as _;
    |                                           ^^^^^^^^^ help: a constant with a similar name exists: `RLIMIT_RSS`
    |
   ::: /home/darkshadow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libc-0.2.176/src/unix/bsd/netbsdlike/mod.rs:419:1
    |
419 | pub const RLIMIT_RSS: c_int = 5;
    | --------------------------- similarly named constant `RLIMIT_RSS` defined here

error[E0425]: cannot find value `RLIMIT_SBSIZE` in crate `libc`
   --> src/builtins/ulimit.rs:73:47
    |
73  |         pub const SBSIZE: libc::c_int = libc::RLIMIT_SBSIZE;
    |                                               ^^^^^^^^^^^^^ help: a constant with a similar name exists: `RLIMIT_FSIZE`
    |
   ::: /home/darkshadow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libc-0.2.176/src/unix/bsd/netbsdlike/mod.rs:415:1
    |
415 | pub const RLIMIT_FSIZE: c_int = 1;
    | ----------------------------- similarly named constant `RLIMIT_FSIZE` defined here

error[E0425]: cannot find value `RLIMIT_NICE` in crate `libc`
   --> src/builtins/ulimit.rs:74:45
    |
74  |         pub const NICE: libc::c_int = libc::RLIMIT_NICE;
    |                                             ^^^^^^^^^^^ help: a constant with a similar name exists: `RLIMIT_CORE`
    |
   ::: /home/darkshadow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libc-0.2.176/src/unix/bsd/netbsdlike/mod.rs:418:1
    |
418 | pub const RLIMIT_CORE: c_int = 4;
    | ---------------------------- similarly named constant `RLIMIT_CORE` defined here

error[E0425]: cannot find value `RLIMIT_NTHR` in crate `libc`
  --> src/builtins/ulimit.rs:76:45
   |
76 |         pub const NTHR: libc::c_int = libc::RLIMIT_NTHR;
   |                                             ^^^^^^^^^^^ not found in `libc`

error[E0425]: cannot find value `RLIMIT_SWAP` in crate `libc`
   --> src/builtins/ulimit.rs:77:45
    |
77  |         pub const SWAP: libc::c_int = libc::RLIMIT_SWAP;
    |                                             ^^^^^^^^^^^ help: a constant with a similar name exists: `RLIMIT_STACK`
    |
   ::: /home/darkshadow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libc-0.2.176/src/unix/bsd/netbsdlike/mod.rs:417:1
    |
417 | pub const RLIMIT_STACK: c_int = 3;
    | ----------------------------- similarly named constant `RLIMIT_STACK` defined here

error[E0425]: cannot find value `RLIMIT_KQUEUES` in crate `libc`
  --> src/builtins/ulimit.rs:78:48
   |
78 |         pub const KQUEUES: libc::c_int = libc::RLIMIT_KQUEUES;
   |                                                ^^^^^^^^^^^^^^ not found in `libc`

error[E0425]: cannot find value `RLIMIT_NPTS` in crate `libc`
   --> src/builtins/ulimit.rs:79:45
    |
79  |         pub const NPTS: libc::c_int = libc::RLIMIT_NPTS;
    |                                             ^^^^^^^^^^^ help: a constant with a similar name exists: `RLIMIT_CPU`
    |
   ::: /home/darkshadow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libc-0.2.176/src/unix/bsd/netbsdlike/mod.rs:414:1
    |
414 | pub const RLIMIT_CPU: c_int = 0;
    | --------------------------- similarly named constant `RLIMIT_CPU` defined here

error[E0425]: cannot find value `_CS_PATH` in crate `libc`
   --> src/env/environment.rs:570:43
    |
570 |     let buf_size = unsafe { confstr(libc::_CS_PATH, std::ptr::null_mut(), 0) };
    |                                           ^^^^^^^^ not found in `libc`

error[E0425]: cannot find value `_CS_PATH` in crate `libc`
   --> src/env/environment.rs:574:36
    |
574 |             unsafe { confstr(libc::_CS_PATH, buf.as_mut_ptr(), buf_size) };
    |                                    ^^^^^^^^ not found in `libc`

limits supported by OpenBSD:

#define RLIMIT_CPU  0       /* cpu time in milliseconds */
#define RLIMIT_FSIZE    1       /* maximum file size */
#define RLIMIT_DATA 2       /* data size */
#define RLIMIT_STACK    3       /* stack size */
#define RLIMIT_CORE 4       /* core file size */
#define RLIMIT_RSS  5       /* resident set size */
#define RLIMIT_MEMLOCK  6       /* locked-in-memory address space */
#define RLIMIT_NPROC    7       /* number of processes */
#define RLIMIT_NOFILE   8       /* number of open files */

#define RLIM_NLIMITS    9       /* number of resource limits */

OpenBSD does have _CS_PATH in unistd.h (defined to 1), guess the libc crate I get just doesn't have it defined.

Rust version being used is 1.86.0. There's no easy way to update rust on OpenBSD, the only way I could is to upgrade from using the stable version of the OS to the unstable version.

[Darkshadow2]

I saw 9ea328e and I'll add that OpenBSD actually does use _CS_PATH (USER_CS_PATH does not exist).

[krobelus]

rust-lang/libc#4738 adds _CS_PATH to libc

Looks like there are other errors but I'm not yet sure how to easily set up an OpenBSD toolchain on Linux.
FreeBSD worked fine with

rustup target add x86_64-unknown-freebsd
cargo b --target=x86_64-unknown-freebsd

# rustup +1.86 target add x86_64-unknown-openbsd
error: toolchain '1.86-x86_64-unknown-linux-gnu' does not support target 'x86_64-unknown-openbsd'; did you mean 'x86_64-unknown-freebsd'?
note: you can see a list of supported targets with `rustc --print=target-list`
note: if you are adding support for a new target to rustc itself, see https://rustc-dev-guide.rust-lang.org/building/new-target.html

I'm hoping on #11893 or someone else, else I guess I'll boot up a sourcehut container.

[Darkshadow2]

Yeah, rust on OpenBSD only has tier 3 support, so there's no binaries or installation via rustup. Which is why I said it would be difficult to update it. If you want to test it, you'll probably have to install OpenBSD in a VM.

I did get it to compile by modifying #11893 (with target_os = "openbsd" where needed), and I did a hack to get the _CS_PATH stuff to work. By just setting it to 1 for OpenBSD, which is what it's set to in the header. That probably isn't going to change anytime soon. If you think that's acceptable/reasonable, I can make a pull request for the _CS_PATH change. I did test it, and it does get the default paths when no PATH is inherited by fish.

Rust isn't something I've used a lot, so if the libc crate for OpenBSD does get updated to have the _CS_PATH stuff, are earlier versions going to update/use it? If not, something like the above is going to be needed to support versions that don't have it.

[krobelus]

what we want to do is 1. merge the requisite changes (_CS_PATH, any RLIMIT constants) into the `rust-lang/libc` crate 2. wait for the next `rust-lang/libc` release 3. edit our `Cargo.toml` to use that release so we can use _CS_PATH and others the process should be quite lightweight. While that work is ongoing, workarounds would be okay (e.g. adding stuff to fish that belongs to libc), but only if we set the wheels in motion for a proper fix. For local testing, you can change the version number with « { path = "../libc" } » like below, to use a local copy of `rust-lang/libc`. Some other changes were necessary to make fish build with latest libc: diff --git a/Cargo.toml b/Cargo.toml index 9353250e30..420a6acff2 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -18,7 +18,8 @@ fish-gettext-maps = { path = "crates/gettext-maps" } fish-gettext-mo-file-parser = { path = "crates/gettext-mo-file-parser" } fish-printf = { path = "crates/printf", features = ["widestring"] } -libc = "0.2.155" +# libc = "0.2.155" +libc = { path = "../libc" } # lru pulls in hashbrown by default, which uses a faster (though less DoS resistant) hashing algo. # disabling default features uses the stdlib instead, but it doubles the time to rewrite the history # files as of 22 April 2024. diff --git a/src/exec.rs b/src/exec.rs index b42a01c43d..dc03e2f771 100644 --- a/src/exec.rs +++ b/src/exec.rs @@ -54,7 +54,7 @@ }; use nix::fcntl::OFlag; use nix::sys::stat; -use std::ffi::CStr; +use std::ffi::{CStr, CString}; use std::io::{Read, Write}; use std::mem::MaybeUninit; use std::num::NonZeroU32; @@ -384,13 +384,13 @@ /// specified in \c p. It never returns. Called in a forked child! Do not allocate memory, etc. fn safe_launch_process( _p: &Process, - actual_cmd: &CStr, + actual_cmd: CString, argv: &OwningNullTerminatedArray, envv: &OwningNullTerminatedArray, ) -> ! { // This function never returns, so we take certain liberties with constness. - unsafe { libc::execve(actual_cmd.as_ptr(), argv.get(), envv.get()) }; + unsafe { libc::execve(actual_cmd.as_ptr(), argv.get_mut(), envv.get_mut()) }; let err = errno(); // The shebang wasn't introduced until UNIX Seventh Edition, so if @@ -398,29 +398,35 @@ // after performing a binary safety check, recommended by POSIX: a // line needs to exist before the first \0 with a lowercase letter - if err.0 == ENOEXEC && is_thompson_shell_script(actual_cmd) { + let mut actual_cmd_ptr = actual_cmd.into_bytes_with_nul(); + let actual_cmd = + |actual_cmd_ptr| unsafe { CStr::from_bytes_with_nul_unchecked(actual_cmd_ptr) }; + + if err.0 == ENOEXEC && is_thompson_shell_script(&actual_cmd(&actual_cmd_ptr)) { // Construct new argv. // We must not allocate memory, so only 128 args are supported. const maxargs: usize = 128; let nargs = argv.len(); - let argv = unsafe { slice::from_raw_parts(argv.get(), nargs) }; + let argv = unsafe { slice::from_raw_parts(argv.get_mut(), nargs) }; if nargs <= maxargs { // +1 for /bin/sh, +1 for terminating nullptr - let mut argv2 = [std::ptr::null(); 1 + maxargs + 1]; + let mut argv2 = [std::ptr::null_mut(); 1 + maxargs + 1]; let bshell = PATH_BSHELL.as_ptr() as *const c_char; argv2[0] = bshell as *mut c_char; argv2[1..argv.len() + 1].copy_from_slice(argv); // The command to call should use the full path, // not what we would pass as argv0. - argv2[1] = actual_cmd.as_ptr(); + argv2[1] = actual_cmd_ptr.as_mut_ptr().cast(); unsafe { - libc::execve(bshell, &argv2[0], envv.get()); + libc::execve(bshell, &argv2[0], envv.get_mut()); } } } set_errno(err); - safe_report_exec_error(errno().0, actual_cmd, argv, envv); + let actual_cmd = + |actual_cmd_ptr| unsafe { CStr::from_bytes_with_nul_unchecked(actual_cmd_ptr) }; + safe_report_exec_error(errno().0, &actual_cmd(&actual_cmd_ptr), argv, envv); exit_without_destructors(exit_code_from_exec_error(err.0)); } @@ -440,7 +446,7 @@ // Ensure the terminal modes are what they were before we changed them. safe_restore_term_mode(); // Bounce to launch_process. This never returns. - safe_launch_process(p, &actual_cmd, &argv, &envp); + safe_launch_process(p, actual_cmd, &argv, &envp); } // Returns whether we can use posix spawn for a given process in a given job. @@ -918,7 +924,7 @@ } fork_child_for_process(j, p, &dup2s, pgroup_policy, |p| { - safe_launch_process(p, &actual_cmd, &argv, &envv) + safe_launch_process(p, actual_cmd, &argv, &envv) }) } diff --git a/src/fork_exec/spawn.rs b/src/fork_exec/spawn.rs index f733b8bbed..a8a5faee69 100644 --- a/src/fork_exec/spawn.rs +++ b/src/fork_exec/spawn.rs @@ -116,7 +116,7 @@ }; // Set our flags. - let mut flags: i32 = 0; + let mut flags: libc::c_short = 0; flags |= libc::POSIX_SPAWN_SETSIGDEF; flags |= libc::POSIX_SPAWN_SETSIGMASK; if desired_pgid.is_some() { diff --git a/src/universal_notifier/kqueue.rs b/src/universal_notifier/kqueue.rs index 098b6ad135..3657a1bef0 100644 --- a/src/universal_notifier/kqueue.rs +++ b/src/universal_notifier/kqueue.rs @@ -119,7 +119,7 @@ )]; // Use a zero timeout because we check for more events than we have notifications for, to // drain all events up front. - let timeout = libc::timespec { + let timeout = nix::libc::timespec { tv_nsec: 0, tv_sec: 0, };

[krobelus]

if the libc crate for OpenBSD does get updated to have the _CS_PATH stuff, are earlier versions going to update/use it?

Our Cargo.toml specifies libc = "0.2.155";
that means that cargo can use any version that's semver-compatible with that version.
If it helps, we could also pin it to an exact version.

Rust does not provide anything like #ifdef _CS_PATH,
so I don't think we can write code that will automagically use libc::_CS_PATH if it's available.
(unless we somehow check for the version of the libc crate)

So we're likely gonna do (ugly) user-agent sniffing.

#[cfg(target_os = "openbsd")]
pub const CS_PATH = 1;
#[cfg(not(target_os = "openbsd"))]
pub const CS_PATH = libc::_CS_PATH;

probably the same on NetBSD?

[Darkshadow2]

That was basically my hack (this is against what is currently there):

diff --git a/src/env/environment.rs b/src/env/environment.rs
index 16f749c7a..b6b5e9bc6 100644
--- a/src/env/environment.rs
+++ b/src/env/environment.rs
@@ -568,10 +568,12 @@ fn setup_user(vars: &EnvStack) {
 pub(crate) static FALLBACK_PATH: Lazy<&[WString]> = Lazy::new(|| {
     // _CS_PATH: colon-separated paths to find POSIX utilities. Same as USER_CS_PATH.
     // BSDs only have the "USER_" form, while Linux only has the "CS_" form.
-    #[cfg(any(apple, bsd))]
+    #[cfg(all(any(apple, bsd), not(target_os = "openbsd")))]
     let cs_path = libc::USER_CS_PATH;
     #[cfg(not(any(apple, bsd)))]
     let cs_path = libc::_CS_PATH;
+    #[cfg(target_os = "openbsd")]
+    let cs_path = 1;

     let buf_size = unsafe { libc::confstr(cs_path, std::ptr::null_mut(), 0) };
     let paths: Vec<WString> = if buf_size > 0 {