Nix Crate Forkpty always Returns Parent

I am pretty sure that I am not understanding this correctly...I am trying to open a pty for the sh shell using the nix libary with this code:

    let fd = unsafe {
        let res = nix::pty::forkpty(None, None).unwrap();
        match res.fork_result {
            ForkResult::Parent { child } => {
                println!("NOOO!")
            }
            ForkResult::Child => {
                let shell_name =
                    CStr::from_bytes_with_nul(b"sh\0").expect("Should always have null terminator");
                nix::unistd::execvp::<CString>(shell_name, &[]).unwrap();
                return;
            }
        };
        res.master
    };

However, it always returns the parent. What am I doing wrong here? I want to get forkpty to return the child, not the parent.

I took a look at the src code and man page of forkpty and, from what I can make out, the function returns the parent if it runs into an error? Does that have anything to do with it?

I think the rust code is correct but for some reason forkpty is actually failing to make a child process.
Apparently nix::pty::forkpty returns ForkResult::Child if the C code would return 0 otherwise it returns ForkResult::Parent { child }

https://man7.org/linux/man-pages/man3/forkpty.3.html
According to this forkpty fails for these reasons
openpty() fails if: ENOENT There are no available terminals. login_tty() fails if ioctl(2) fails to set fd to the controlling terminal of the calling process. forkpty() fails if either openpty() or fork(2) fails.

Maybe you can try calling these functions individually to see what fails.

The mistake is in some other part of code which you have not pasted here.

Hmm...I tried the openpty and fork function from the nix crate...they both worked (I tried is_ok on them). Does it have something to do with login_tty? How can I test that? Also, I tried changing the code to

    let fd = unsafe {
        let res = nix::pty::forkpty(None, None).unwrap();
        match res.fork_result {
            ForkResult::Parent { child } => {
                let shell_name =
                    CStr::from_bytes_with_nul(b"sh\0").expect("Should always have null terminator");
                nix::unistd::execvp::<CString>(shell_name, &[]).unwrap();
                return;
            }
            ForkResult::Child => {
                println!("NOOO!")
            }
        };
        res.master
    };

And I get a full shell in the current process. However, I don't want it in the current process, but in a child process.

When fork(2) fails the result would still say Ok It would not show an error in rust. I meant look at output of pub unsafe fn fork after unwrapping it. If it is Child then it is working fine . If it is Parent
then that is an error. I think there is some error when you call fork. fork(2) - Linux manual page

Maybe try this

use nix::errno::Errno;
let res=nix::pty::forkpty(None, None).unwrap();
let res2=match res.fork_result{
            ForkResult::Parent { child } => {
              child
            }
_=>{return;}
}
let Err=Errno::result(res2);
println!("{}",Err);

This is my best guess about how to get actual information about what is causing the error after looking at source code.

1 Like

Yes, fork returns a parent instead of a child...I used this code to test:

    unsafe {
        use nix::errno::Errno;
        let res = nix::pty::forkpty(None, None).unwrap();
        let res2 = match res.fork_result {
            ForkResult::Parent { child } => child,
            _ => {
                return;
            }
        };
        let err = Errno::result(res2.as_raw());
        println!("{:?}", err);
    }

The result is not very helpful, it is just the PID wrapped in ok. Did I test it wrong?
By the way, thank you for guiding me through this :slight_smile:

@PrashantShekharRao, I think I have found the root cause of this error...I decided to give up on this and switch to using openpty with Command. When I tried to read from the file, I get an error that is a bit more descriptive:

thread 'main' panicked at src/main.rs:57:14:
Could not read from file: Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" }

The thing is, I am pretty sure that I have more than enough resources...

I am not sure how to fix that. I don't really know anything about linux os . I was just googling. Still I think the resources it's referring to are not stuff like RAM . Maybe you can try pasting some other code from your project. Since from the current code it is not clear what is causing the error.

Hmm...Here is a github repo with the code. I put the code in a l loop and continued on EAGAIN. I get a whole lot of twos...

The link is not working.

mb it was private. Hoepfully it works now.

What are you trying to achieve in the parent and the child process respectively?

In the parent, I want to do nothing. In the child, I want to start a shell

Nvm it works now, idk how I even got it to work. I think that it was working since I started getting the twos. I took a look at the docs of nix::unistd::read, which told me that it return a bucket of bytes, not a String (which makes sense, now that I think of it). Converted it to a string, and boom! I get the prompt. Thanks to the two people in this thread, @PrashantShekharRao especially...Honestly, I still don't understand how it works. If I have a match statement, shouldn't only one of the code blocks be ran?

Anyway, now I can finally get on to the "fun part" of building a terminal...THANKS!!!

1 Like

In conclusion, the fix here was to put the read in a loop, ignore EAGAIN errors (which means no content), add the bytes to a buffer, and convert the buffer to a str literal to display.

1 Like