Why does forkpty fail?

If I the nix::pty::forkpty function fails, how can I get more information on the error? By failing, I mean returning a parent instead of a child. I have researched for hours on this, but I have not found an answer. Is there anyone who can help? If more information is required, I can give it, but I do not know what else I should include.

what do you mean by it "failing"? forkpty calls fork internally, so it returns twice, once in the address space of the parent process, and once in the child process.

to quote the man page:

The forkpty() function combines openpty(), fork(2), and login_tty() to create a new process operating in a pseudoterminal.

... (otherwise on success) the child process of forkpty() return 0, and the parent process of forkpty() returns the process ID of the child process.

see:

@nerditation, by failing, I mean that it returns a parent instead of a child. I took a look at the nix source, and it seems like it returns a child if there are no errors, and a parent if it runs into errors.

the nix crate is just a wrapper over libc. if the fork fails, the call returns -1 and set errno, which gets translated into rust Result::Err(nix::Errno). so if you gets a Ok, then it is NOT failing.

I'm not sure if you understand how fork works (or, even how pty works), but as I said earlier, on success, it returns TWICE, once for the parent, and once for the child.

if you "think" it fails because you didn't see the println!() messages from the child process, because that's the correct behavior: the child process have the newly created pseudo terminal configured as it's controlling terminal.


BTW, this has nothing to do with the nix crate or rust the language. nix is a very thin layer on top of Linux (or POSIX-like in general) system APIs. that's the main reason the docs are very terse: the users are supposed to be familiar with Posix systems so it just links to the man pages for the underlying C functions. in case you are not familiar with certain APIs, you are supposed to read the linked man pages for detailed explanations.

2 Likes

Please forgive me for the crime of being a noob (in all seriousness), but how do I get the child file descriptor, if it returns twice? I have not found any method for doing that from a PID. Also, I played around with the code a little bit, and got an EAGAIN error. Looking at the man page, it apparently means that I "do not have any available pseudoterminals." Does this have something to do with some default cargo settings? I am pretty sure that I have enough resources.

After a successful fork you have two processes, and to know which one you're in (or to check for an error), you examine the return value. From the (previously linked) docs:

Return Value

On success, the PID of the child process is returned in the parent, and 0 is returned in the child. On failure, -1 is returned in the parent, no child process is created, and errno is set appropriately.

fork example.

3 Likes

Yes, that I knew...Here is the code I am trying to use:

fn spawn_shell() -> Option<OwnedFd> {
    unsafe {
        let res = nix::pty::forkpty(None, None).expect("Fork failed");
        match res.fork_result {
            ForkResult::Parent { .. } => (),
            ForkResult::Child => {
                let shell_name = c"/bin/bash";
                let args: &[&[u8]] = &[b"bash\0", b"--noprofile\0", b"--norc\0"];

                let args: Vec<&'static CStr> = args
                    .iter()
                    .map(|v| {
                        CStr::from_bytes_with_nul(v).expect("Should always have null terminator")
                    })
                    .collect::<Vec<_>>();


                nix::unistd::execvp(shell_name, &args).expect("Could not spawn shell");
                std::process::exit(1);
            }
        }
        Some(res.master)
    }
}

My problem is that I always end up in the parent with PID, instead of inside the child? Am I misunderstanding this? My instinct tells me that I am.

The child process probably exists, but you can't see any output from it because it's attached to the pty, and you're not doing anything in the parent with the pty to read that output.

That makes sense...but isn't the match statement only supposed to match to either parent or child?

Well, forking means control flow goes two ways in two different processes. In the parent, the Parent branch will be executed, but it does nothing. In the child, the Child branch will be executed, but its output goes to the pty which is not being read, so you don't see the output.

You need to use the pty master in the Parent branch, not just ignore it. (Or why are you using a pty at all?)

1 Like

I think I figured out the problem (at least my problem, idk but I doubt at it was a probelm for anyone else). I thought that an EAGAIN error meant that my computer had too few resources to open another pseudo terminal. Apparently it just meant that there was nothing read. Although I probably should have guessed that, the man pages were a little misleading. Thanks, everyone, for schooling me in this...

Run such things under strace -ff <command> to see all the syscalls that are happening. That should provide an overview what all the involved actors are doing.

1 Like

@the8472, thx! I think this will help a lot in the future