How do I perform interactive debugging in console?

help
1

Hello,

I'm kind of confused about how to debug my rust code, I tried searching around the internet but accurate technical information regarding this subject is surprisingly scarce!

Someone recommended me to read this article which says I should use rust-lldb with binaries under target/debug/deps but when I try it I don't get the original source code as output from the debugger, instead I just get the disassembled machine code.

So I'd like to know the proper way to debug rust.

If I was working with C I'd simply enable the option to add the debug symbols and use gdb on the resulting binary directly, how do I do the same in Rust?

Bonus Question: Why does the deps directory have a million copies of my program? Can I disable generation of these additional artifacts?

2

If I was working with C I'd simply enable the option to add the debug symbols and use gdb on the resulting binary directly, how do I do the same in Rust?

cargo build does this by default, there is nothing you need to do. (You should be able to see debuginfo mentioned in the output of Cargo.)

The linked article using rust-lldb worked for me. You would need to provide more information to get any help. For example, full transcript to get the disassembled machine code instead of source code others can reproduce, ideally starting from cargo new.

3

I typically use rust-gdb (which pretty prints rust objects, but it is similar to rust-lldb) on the binary in target/debug/deps like you mentioned. On linux, you can confirm the binary you're debugging has debug symbols by running file [path_to_bin] and check for debug_info, not stripped. You can also manually specify that debug symbols are added to any profile and no optimizations are applied via

[profile._profilename_]
debug = true # adds debug info
opt-level = 0 # debug build

but this should already be set in a cargo build which builds debug mode by default.

deps is large (I have seen 10s of GB) because it caches builds of all dependencies so future builds are "incremental" and much faster. It can be cleared entirely via cargo clean or just for a single dependency via cargo clean -p [package_name] [-r] where the optional -r specifies the target/release folder (vs the default target/debug).

4

@sanxiyn Here's a completely new project directory that I started with:

TheDcoder@arch /t/rust> ls
Cargo.toml  src/
TheDcoder@arch /t/rust> cat Cargo.toml 
[package]
name = "guess"
version = "0.1.0"
edition = "2021"

[dependencies]
rand = "0.8.5"
TheDcoder@arch /t/rust> cargo build
    Updating crates.io index
  Downloaded getrandom v0.2.12
  Downloaded libc v0.2.153
  Downloaded 2 crates (776.8 KB) in 3.55s
   Compiling libc v0.2.153
   Compiling cfg-if v1.0.0
   Compiling ppv-lite86 v0.2.17
   Compiling getrandom v0.2.12
   Compiling rand_core v0.6.4
   Compiling rand_chacha v0.3.1
   Compiling rand v0.8.5
   Compiling guess v0.1.0 (/tmp/rust)
    Finished dev [unoptimized + debuginfo] target(s) in 5.65s
TheDcoder@arch /t/rust> rust-lldb target/debug/guess
(lldb) command script import "/home/TheDcoder/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/etc/lldb_lookup.py"
(lldb) command source -s 0 '/home/TheDcoder/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/etc/lldb_commands'
Executing commands in '/home/TheDcoder/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/etc/lldb_commands'.
(lldb) type synthetic add -l lldb_lookup.synthetic_lookup -x ".*" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(alloc::([a-z_]+::)+)String$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^&(mut )?str$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^&(mut )?\\[.+\\]$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(std::ffi::([a-z_]+::)+)OsString$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(alloc::([a-z_]+::)+)Vec<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(alloc::([a-z_]+::)+)VecDeque<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(alloc::([a-z_]+::)+)BTreeSet<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(alloc::([a-z_]+::)+)BTreeMap<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(std::collections::([a-z_]+::)+)HashMap<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(std::collections::([a-z_]+::)+)HashSet<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(alloc::([a-z_]+::)+)Rc<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(alloc::([a-z_]+::)+)Arc<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(core::([a-z_]+::)+)Cell<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(core::([a-z_]+::)+)Ref<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(core::([a-z_]+::)+)RefMut<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^(core::([a-z_]+::)+)RefCell<.+>$" --category Rust
(lldb) type summary add -F lldb_lookup.summary_lookup  -e -x -h "^core::num::([a-z_]+::)*NonZero.+$" --category Rust
(lldb) type category enable Rust
(lldb) target create "target/debug/guess"
Current executable set to '/tmp/rust/target/debug/guess' (x86_64).
(lldb) b main
Breakpoint 1: 2 locations.
(lldb) r
Process 65540 launched: '/tmp/rust/target/debug/guess' (x86_64)
Process 65540 stopped
* thread #1, name = 'guess', stop reason = breakpoint 1.2
    frame #0: 0x000055555555f420 guess`main
guess`main:
->  0x55555555f420 <+0>:  pushq  %rax
    0x55555555f421 <+1>:  movq   %rsi, %rdx
    0x55555555f424 <+4>:  leaq   0x8395c(%rip), %rax       ; __rustc_debug_gdb_scripts_section__
    0x55555555f42b <+11>: movb   (%rax), %al
(lldb) exit
Quitting LLDB will kill one or more processes. Do you really want to proceed: [Y/n] y
TheDcoder@arch /t/rust> rust-gdb target/debug/guess
GNU gdb (GDB) 14.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from target/debug/guess...
(gdb) break main
Breakpoint 1 at 0xb420
(gdb) r
Starting program: /tmp/rust/target/debug/guess 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".

Breakpoint 1, 0x000055555555f420 in main ()
(gdb) s
Single stepping until exit from function main,
which has no line number information.
std::rt::lang_start<()> (main=0x55555555f0f0 <guess::main>, argc=1, argv=0x7fffffffe678, sigpipe=0) at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167
167	        &move || crate::sys_common::backtrace::__rust_begin_short_backtrace(main).report().to_i32(),
(gdb) s
166	    let Ok(v) = lang_start_internal(
(gdb) s
std::rt::lang_start_internal () at library/std/src/rt.rs:147
147	    panic::catch_unwind(move || unsafe { init(argc, argv, sigpipe) }).map_err(rt_abort)?;
(gdb) s
std::panic::catch_unwind<std::rt::lang_start_internal::{closure_env#1}, ()> () at library/std/src/panic.rs:142
142	    unsafe { panicking::r#try(f) }
(gdb) c
Continuing.
Guess the number!
Please input your guess.
<snip>
[Inferior 1 (process 65608) exited normally]
(gdb) quit

GDB does seem to recognize Rust source code but it's not the same as my code:

use std::io;
use std::cmp::Ordering;
use rand::Rng;

fn main() {
	println!("Guess the number!");

	let secret_number = rand::thread_rng().gen_range(1..=100);
	println!("The secret number is: {secret_number}");

	loop {
		println!("Please input your guess.");
		
		let mut guess = String::new();
		
		io::stdin()
			.read_line(&mut guess)
			.expect("Failed to read line");
		
		let guess: u32 = match guess.trim().parse() {
			Ok(num) => num,
			Err(_) => continue
		};
		println!("Your guess: {guess}");
		
		match guess.cmp(&secret_number) {
			Ordering::Less => println!("Greater"),
			Ordering::Greater => println!("Lesser"),
			Ordering::Equal => {
				println!("Spot on!");
				break;
			},
		};
	}
}

> file target/debug/guess
target/debug/guess: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=09412593cb28b5dd80f1801123b48bbbdbf04a1b, for GNU/Linux 4.4.0, with debug_info, not stripped

So it confirms that the binary does have debug info but for some reason both GDB and LLVM are not able to use it properly :confused:

Is there a way to set the location of this "cache" or even disable it entirely?

5

Ugh. Yes, that's bad and confusing, but debugging is actually working fine. Let me explain.

(lldb) b main
Breakpoint 1: 2 locations.

Why 2 locations? There is only one main, so it should report where main is, for example main.rs:6. You can ask LLDB why.

(lldb) breakpoint list
Current breakpoints:
1: name = 'main', locations = 2
  1.1: where = guess`guess::main::h42b2d61870ef14f5 + 7 at main.rs:6:2, address = guess[0x000000000000bc17], unresolved, hit count = 0
  1.2: where = guess`main, address = guess[0x000000000000c060], unresolved, hit count = 0

All Unix processes start at main, but that main is not same as Rust main. To LLDB, Rust main is actually guess::main. Low level main is autogenerated for you, doing Rust runtime initialization and then calling Rust main.

So by doing b main, you set breakpoints at both main and guess::main. There is no source for main, but there is source for guess::main. Also main runs before guess::main. That's why you are seeing machine code.

Now problem is understood, there are many ways to get it to work. For example, you can delete the extra breakpoint.

(lldb) breakpoint delete 1.2

Simply continuing also works, because you won't return to main.

(lldb) c

You can also specify the full symbol, or the source location. This is probably why you couldn't find this problem in search. It only happens for main, any other functions work, all source locations work. That's also how I tested when I replied it is working for me.

(lldb) b guess::main
(lldb) b main.rs:6
6

Actually they start at _start, and run the setup for the C runtime. Yes C has a runtime. It consists of:

  • Setting up stack and arguments for main.
  • Dealing with environment variable block pointer and other things the kernel might pass in the auxiliary vector.
  • Handling return from main being turned into an exit syscall.
  • Calling static constructors/destructores. (GCC has them as an extension for C, and C++ of course has them by default.)
  • Probably something else that I forgot.

In Rust's case there is then another layer of Rust runtime setup after the C runtime. Panic handling, stack overflow detection and probably a few more things.

Actually actually, _start is just a convention. The program starts at whatever address is set in various ELF headers. (ELF bring the file format for binaries on Linux and several other *nix, not OSX, it has its own format.)

Is all of this nitpicking? Yes, but knowing it might save you some day when you have a particularly weird issue to debug.

7

@Vorpal Appreciate the detailed explanation.

@sanxiyn What a coincidence... but I would expect the main symbol to be reserved for the "user-generated" main function, not the main function of Rust or libc.

But in any case, why did gdb still show cryptic Rust source code while lldb did not?

And which debugger should I prefer? I'd prefer gdb purely based on familiarity, but does it have any disadvantages compared to lldb for the average user?

8

In the gdb case you did an additional step command which steps into std::rt::lang_start as that is the first function with source code available. In the lldb case you didn't step at all.

I prefer gdb too. Lldb is simply missing too many useful things like debugging the child process when forking. (set follow-fork-mode child in gdb) Or the non-stop mode to only stop a single thread at a time rather than the entire process to minimize interference with a highly concurrent system. I also don't like the verbosity of a lot of commands with lldb.