Debugging some async code (BugStalker)

help
1

I am trying to understand why GlobalProtect-openconnect, a Rust program to set up a VPN, is not working for me. This involves exchanges with my school's servers, and so I need to capture that behavior, not some unit test.

I tried debugging in VSCode and emacs and found the results uninterpretable in the former, and I may have given up during setup on the latter when I noticed no reports of success getting debugging to work in emacs. Recent discussion suggested BugStalker was the best bet, and so I'm trying that. It's not much more interpretable. Can anyone help me to understand my recent results, or, more generally, understand my original problem?

Before getting in to the details, I should mention two expectations. First, I expect the failure to connect is not a result of a defect in the program, since the manufacturer's own VPN client (closed source) also can not connect. Second, I suspect that the second bug I caught may have been induced by the debugger itself. The program is definitely failing before the point it fails when used without the debugger.

ross@barley3:~/src/GlobalProtect-openconnect/target/debug$ rustup run stable bs gpclient -- --fix-openssl connect --as-gateway xxx.edu

BugStalker greets

(bs) b main
New breakpoint 1 at 0x000000001BAA66: /home/ross/src/GlobalProtect-openconnect/apps/gpclient/src/main.rs:8 
New breakpoint 2 at 0x00000000000013: /home/ross/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/intrinsics.rs:2714 
(bs) b r 2
Removed breakpoint 2 at 0x00000000000013: /home/ross/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/intrinsics.rs:2714 
(bs) run

thread 'main' panicked at /home/ross/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/bugstalker-0.3.0/src/debugger/process.rs:212:17:
run debugee fail with: No such file or directory (os error 2)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Program exit with code: 101
(bs) bt
Error: program is not being started
  1. I would expect the error to be caught so I can examine the program state, but the response to bt and the message about Program exit suggest that didn't happen. Is there a way to stop it while the program is live?
  2. No indication of what file was being opened or who was opening it. The only file mentioned, process.rs, does exist at the specified location. The indicated line is panic!("run debugee fail with: {err}"); which doesn't much help with finding the ultimate source of the error.
  3. 2 main functions seems like one too many, though the system apparently can cope.

Then I started stepping through the code more slowly. It uses tokio and the top-level application is

#[tokio::main]
async fn main() {
  cli::run().await;
}

cli::run and other functions are also marked as async.

step at cli::run().await puts me in the tokio code (I naively thought it would put me in cli::run()---see Issue 21 in BugStalker). Tracing into that eventually leads to

(bs) next
 814             Kind::MultiThread => self.build_threaded_runtime(),
(bs) next
 818     }
(bs) step
core::result::Result<T,E>::expect<tokio::runtime::runtime::Runtime, std::io::error::Error> at /home/ross/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:1057
Error: hook: No such file or directory (os error 2)

result.rs is not present at the specified location, and so perhaps is the cause of the os error 2. ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src does not exist. At this point the debugger shows I have 5 threads running, 2 of them with stack depths over 40. Since the tracebacks don't include source line numbers, it's hard to track the details. Some of them have nested std::panicking::try calls.