Newbie FFI: Segfaults and undefined symbols for architecture

n8henrie · July 12, 2022, 6:31pm

I've come back to working on FFI bindings for TaskWarrior (a C++ project), which I had some some incredible help on last year (thanks again!):

I had a few things working by creating a wrapper.h and wrapper.cpp thanks to @MoAlyousef and an analogous CXX example contributed by @dtolnay.

I wanted to revisit to see if I could get things working just by using the TaskWarrior header files and shared libraries directly -- which may not be possible, I suppose.

Currently, I'm focusing on the Context class, which seems to do most of the heavy lifting. I can get it to build with this build.rs:

github.com

n8henrie/taskwarrior-rs/blob/no-wrapper/taskwarrior-sys/build.rs

use std::env;
use std::path::PathBuf;

use cmake::Config;

struct FakeDst();
impl FakeDst {
    fn display(&self) -> &'static str {
        return "/Users/n8henrie/git/taskwarrior-rs/target/debug/build/taskwarrior-sys-c885456ffaf97d6f/out";
    }
}

fn main() {
    // println!("cargo:rerun-if-changed=wrapper.h");
    // println!("cargo:rerun-if-changed=wrapper.cpp");

    let dst = Config::new("vendor/taskwarrior")
        .profile("release")
        .define("PROJECT_VERSION", "2.6.2")
        .build();

This file has been truncated. show original

The resulting binary in ../target/debug/build/... runs and the layout tests pass.

However, it doesn't work with a simple smoke test. Looking at the generated bindings, I created two different versions of a very basic test, modeled after [taskwarrior's main.cpp code]^[1].

The code in question is in test_new, here.

It constructs a fake CLI argument list and just runs list; I had this part working before, so I think it's code is ok.

I was hoping to use the rusty-looking bindings, which I got to compile (but not link) like so:

let mut ctx = Context::new(Context_context);
Context::setContext(&mut ctx);
let status = ctx.initialize(i32::try_from(argv.len() - 1)?, argv.as_mut_ptr());
dbg!(status);
// let status = ctx.run();
// dbg!(status);

Unfortunately, this gives me what I think is a linking error at test time, right at the first line (Context::new(Context_context);):

  = note: Undefined symbols for architecture arm64:
            "Context::Context(Context const&)", referenced from:
                taskwarrior_sys::Context::new::hbd1a16f83b64984f in taskwarrior_sys-e91a44beb3be72f4.r25ysju3oi22kyb.rcgu.o
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

The generated bindings include:

extern "C" {
    #[link_name = "\u{1}__ZN7ContextC1ERKS_"]
    pub fn Context_Context(this: *mut Context, arg1: *const Context);
}
...
impl Context {
    #[inline]
    pub unsafe fn new(arg1: *const Context) -> Self {
        let mut __bindgen_tmp = ::std::mem::MaybeUninit::uninit();
        Context_Context(__bindgen_tmp.as_mut_ptr(), arg1);
        __bindgen_tmp.assume_init()
    }
}

I've tried changing my build.rs a hundred different ways to allowlist_{type,function,var} .*[cC]ontext.*, but it doesn't seem to change anything.

I can see that __ZN7ContextC1ERKS_ exists in a few places:

$ rg -uuuFl '__ZN7ContextC1ERKS_' ../target/
../target/debug/deps/libtaskwarrior_sys-ee3ffa94e667fce4.rlib
../target/debug/deps/libtaskwarrior_sys-ee3ffa94e667fce4.rmeta
../target/debug/deps/taskwarrior_sys-e91a44beb3be72f4.r25ysju3oi22kyb.rcgu.o
../target/debug/build/taskwarrior-sys-c885456ffaf97d6f/out/bindings.rs
../target/debug/incremental/taskwarrior_sys-2vr0tyl3mgg5z/s-gbl50gj9tl-mpb122-1oquhnrfll5mc/r25ysju3oi22kyb.o
../target/debug/incremental/taskwarrior_sys-3tmt2vpuboc4y/s-gbl50gj9tm-wqcw88-2vqm4g5d9uo/query-cache.bin
../target/debug/incremental/taskwarrior_sys-2vr0tyl3mgg5z/s-gbl50gj9tl-mpb122-1oquhnrfll5mc/query-cache.bin
../target/debug/libtaskwarrior_sys.rlib

Instead of the rusty-looking bindings, if I use the extern "C" bindings (with the underscore) directly, I get a little further:

let ctx = Context_context;
  Context_setContext(ctx);
  let status = Context_initialize(
    ctx,
    i32::try_from(argv.len() - 1)?,
    argv.as_mut_ptr(),
);
dbg!(status);
// let status = Context_run(Context_getContext());
// dbg!(status);

In this case, I get a segfault at Context_initialize.

Questions:

In the first case, what symbol is undefined, and why?
- I assume Context::Context, but could it be Context (just before the const)?
- I think I'm linking all the shared libraries and allowlisting all related types, and similar code links from the "underscore" approach -- why is it undefined?
In the second case, I think I'm following pretty closely what the original cpp code does (below) -- why does it segfault from Rust?

int status {0};
Context globalContext;
Context::setContext (&globalContext);
status = Context::getContext ().initialize (argc, argv);
if (status == 0)
    status = Context::getContext ().run ();

Many thanks in advance for any pointers and suggestions!

taskwarrior/main.cpp at develop · GothenburgBitFactory/taskwarrior · GitHub ↩︎

bjorn3 · July 12, 2022, 6:59pm

n8henrie:

Instead of the rusty-looking bindings, if I use the extern "C" bindings (with the underscore) directly, I get a little further:
let ctx = Context_context;
  Context_setContext(ctx);
  let status = Context_initialize(
    ctx,
    i32::try_from(argv.len() - 1)?,
    argv.as_mut_ptr(),
);
dbg!(status);
// let status = Context_run(Context_getContext());
// dbg!(status);
In this case, I get a segfault at Context_initialize.

What is the definition and type of argv?

n8henrie · July 12, 2022, 9:22pm

let cmd = CString::new("list")?;
let mut argv: Vec<*const c_char> =
            vec![CString::new("")?.as_ptr(), cmd.as_ptr(), std::ptr::null()];

github.com

n8henrie/taskwarrior-rs/blob/no-wrapper/taskwarrior-sys/src/lib.rs#L17


      
          mod tests {
              use super::*;
              use ::std::ffi::CString;
              use std::os::raw::c_char;
          
          
    type Result<T> = ::std::result::Result<T, Box<dyn ::std::error::Error>>;
          
          
    #[test]
              fn test_new() -> Result<()> {
                  let cmd = CString::new("list")?;
                  let mut argv: Vec<*const c_char> =
                      vec![CString::new("")?.as_ptr(), cmd.as_ptr(), std::ptr::null()];
          
          
        unsafe {
                      let ctx = Context_context;
                      Context_setContext(ctx);
                      let status = Context_initialize(
                          ctx,
                          i32::try_from(argv.len() - 1)?,
                          argv.as_mut_ptr(),
                      );

To match up with:

int main (int argc, const char** argv)
...
status = Context::getContext ().initialize (argc, argv);

github.com

GothenburgBitFactory/taskwarrior/blob/develop/src/main.cpp#L35


      
          ////////////////////////////////////////////////////////////////////////////////
          
          
#include <cmake.h>
          #include <iostream>
          #include <new>
          #include <cstring>
          #include <Context.h>
          #include <regex>
          
          
////////////////////////////////////////////////////////////////////////////////
          int main (int argc, const char** argv)
          {
            int status {0};
          
          
  Context globalContext;
            Context::setContext (&globalContext);
          
          
  // Lightweight version checking that doesn't require initialization or any I/O.
            if (argc == 2 && !strcmp (argv[1], "--version"))
            {
              std::cout << VERSION << "\n";

I haven't made any changes to argv since I had it working with the minimal C wrapper approach, which is still available on the master branch: https://github.com/n8henrie/taskwarrior-rs/tree/master/taskwarrior-sys

RustyYato · July 13, 2022, 1:52am

CString::new("")?.as_ptr()

Allocates, gets a ptr, then at the end of the statement frees the string. So you have a use after free bug.

You should change it to CStr::from_bytes_with_nul instead

CStr::from_bytes_with_nul(b"\0")?.as_ptr()

n8henrie · July 13, 2022, 6:32am

Thanks for pointing that out!

Doesn't seem to change the behavior of the issues above, but I'm glad to know.

riking · July 13, 2022, 8:54am

I believe this is the mission statement of the google/autocxx repository - have you investigated that yet?

n8henrie · July 13, 2022, 5:14pm

Yes, I did about a year ago and couldn't get it to work.

Just tried again today:

fn initialize(_uhoh: BindingGenerationFailure)
autocxx bindings couldn’t be generated: Pointer pointed to something unsupported

¯\_(ツ)_/¯

system · October 11, 2022, 5:15pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.