Cc-rs / FFI "Multiple Definition Error", but compiles okay manually

help
1

Hello all,

I'm trying to dip my toes into FFI by building my first -sys crate, based on https://github.com/wez/atomicparsley, for use in another one of my projects. I've hardly done any C or C++ work, so I have some learning to do! The project recommends using cmake, but I've found I can use clang++ *.cpp (or g++ *.cpp on linux) with a few -D definitions and it compiles just fine, so I thought cc-rs would work as well (though I'm also aware of cmake-rs by the same author -- thank you @alexcrichton!).

Unfortunately, when I run cargo test, I'm getting an error about duplicate definitions of main, which seem to be coming from here; if I manually change each of these int main definitions and cargo clean in between, the error goes away.

My question is: can you see any obvious flags that I'm missing in my build.rs that explain why it's building fine from the cli but throwing an error when built from cc-rs?

I have the atomicparsley repo added as a git submodule in vendor.

// build.rs
use std::{env, path::PathBuf};

fn main() {
    println!("cargo:rerun-if-changed=wrapper.h");
    let src = [
        "vendor/src/CDtoc.cpp",
        "vendor/src/arrays.cpp",
        "vendor/src/compress.cpp",
        "vendor/src/extracts.cpp",
        "vendor/src/iconv.cpp",
        "vendor/src/id3v2.cpp",
        "vendor/src/main.cpp",
        "vendor/src/metalist.cpp",
        "vendor/src/parsley.cpp",
        "vendor/src/sha1.cpp",
        "vendor/src/util.cpp",
        "vendor/src/uuid.cpp",
    ];

    cc::Build::new()
        .cpp(true)
        .files(src.iter())
        .define("PACKAGE_VERSION", "\"20211003.181952.0\"")
        .define("BUILD_INFO", "\"90ad66d789bf55aa3738e3d3f7e21436ac04b59c\"")
        .compile("atomicparsley");

    let bindings = bindgen::Builder::default()
        .header("wrapper.h")
        .clang_arg("-xc++")
        .clang_arg("-std=c++11")
        .generate()
        .expect("Unable to generate bindings");

    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
    bindings
        .write_to_file(out_path.join("bindings.rs"))
        .expect("Couldn't write bindings!");

    // Comment these out to compile on linux
    println!("cargo:rustc-link-lib=framework=Foundation");
    println!("cargo:rustc-link-lib=framework=Cocoa");
    println!("cargo:rustc-link-lib=framework=IOKit");
}

// src/lib.rs
include!(concat!(env!("OUT_DIR"), "/bindings.rs"));

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn it_works() {
        unsafe {
            ShowVersionInfo();
        }
    }
}

// wrapper.h
void ShowVersionInfo();

Compiling on MacOS Big Sur (on an M1), from the root of the atomicparsley repo:

$ /usr/bin/clang++ --version
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ /usr/bin/clang++ -DBUILD_INFO=\"foo\" -DPACKAGE_VERSION=\"bar\" -framework Cocoa -framework Foundation -framework IOKit src/*.cpp src/*.mm
src/main.cpp:3801:17: warning: operator '?:' has lower precedence than '*'; '*' will be evaluated first [-Wparentheses]
                ? (keyword_strlen * 4)
                ^
src/main.cpp:3801:17: note: place parentheses around the '*' expression to silence this warning
                ? (keyword_strlen * 4)
                ^
src/main.cpp:3801:17: note: place parentheses around the '?:' expression to evaluate it first
                ? (keyword_strlen * 4)
                ^
1 warning generated.
src/parsley.cpp:2754:58: warning: illegal character encoding in string literal [-Winvalid-source-encoding]
    const char *custom_genre_atom = "moov.udta.meta.ilst.<A9>gen";
                                                         ^~~~
src/parsley.cpp:2755:61: warning: illegal character encoding in string literal [-Winvalid-source-encoding]
    const char *cstm_genre_data_atom = "moov.udta.meta.ilst.<A9>gen.data";
                                                            ^~~~
src/parsley.cpp:2781:59: warning: illegal character encoding in string literal [-Winvalid-source-encoding]
            if (strncmp(verboten_genre_atom->AtomicName, "<A9>gen", 4) == 0) {
                                                          ^~~~
src/parsley.cpp:2838:44: warning: illegal character encoding in string literal [-Winvalid-source-encoding]
        APar_FindAtom("moov.udta.meta.ilst.<A9>lyr.data", true, VERSIONED_ATOM, 0);
                                           ^~~~
4 warnings generated.
src/nsfile.mm:135:42: warning: 'changeFileAttributes:atPath:' is deprecated: first deprecated in macOS 10.5 - Use -setAttributes:ofItemAtPath:error: instead [-Wdeprecated-declarations]
    if (![[NSFileManager defaultManager] changeFileAttributes:output_attributes
                                         ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSFileManager.h:220:1: note: 'changeFileAttributes:atPath:' has been explicitly marked deprecated here
- (BOOL)changeFileAttributes:(NSDictionary *)attributes atPath:(NSString *)path API_DEPRECATED("Use -setAttributes:ofItemAtPath:error: instead", macos(10.0,10.5), ios(2.0,2.0), watchos(2.0,2.0), tvos(9.0,9.0));
^
src/nsfile.mm:143:41: warning: 'fileAttributesAtPath:traverseLink:' is deprecated: first deprecated in macOS 10.5 - Use -attributesOfItemAtPath:error: instead [-Wdeprecated-declarations]
        [[NSFileManager defaultManager] fileAttributesAtPath:inFile
                                        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSFileManager.h:219:1: note: 'fileAttributesAtPath:traverseLink:' has been explicitly marked deprecated here
- (nullable NSDictionary *)fileAttributesAtPath:(NSString *)path traverseLink:(BOOL)yorn API_DEPRECATED("Use -attributesOfItemAtPath:error: instead", macos(10.0,10.5), ios(2.0,2.0), watchos(2.0,2.0), tvos(9.0,9.0));
^
2 warnings generated.

$ ./a.out --version
AtomicParsley version: bar foo (utf8)

Also on MacOS:

$ cargo clean; CXX=/usr/bin/clang++ cargo test
... compiling rust stuff...
error: linking with `cc` failed: exit status: 1
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-arch" "arm64" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.16sx213jmsoo6x85.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.1to3wvar9qk3tdwr.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2e5zn6my1q0brh9m.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2ni19pr22er02fvl.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2qx04b2h1d2iihd3.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2zbipg7xh1mwjl6g.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.3j75uj13wzlhfx3c.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.3kss1ezdynmn1zxq.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4b3fkvqvo3qf4gon.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4r3aw9u41fwno8hq.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4w1qn8f1cxjfrvyn.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4xp5ypzgr3qz7rj9.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.55o8cjtsw9h71z2g.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.x8w4534r50augy0.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4w5vy4g6t2s070vm.rcgu.o" "-L" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps" "-L" "/Users/n8henrie/git/atomicparsley-sys/target/debug/build/atomicparsley-sys-6b2017ea8a0d6804/out" "-L" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-Wl,-force_load" "-Wl,/Users/n8henrie/git/atomicparsley-sys/target/debug/build/atomicparsley-sys-6b2017ea8a0d6804/out/libatomicparsley.a" "-lc++" "-framework" "Foundation" "-framework" "Cocoa" "-framework" "IOKit" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libtest-06ff11d645105e2b.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libterm-f05ecaeca66814e6.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgetopts-6ad0d034fc8e15c7.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunicode_width-c4beacdc35405cb8.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_std-fc0f4647def297a8.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-000cdec9267bfd7b.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-2669f3cbce8358f4.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-86461a1c60728ccb.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-f17574752cb5ddba.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-3bb606c936cc0d28.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-8139a4b0cda20184.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-ea0823eca3e9abf9.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-4f19e1259f6028e7.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-435daca85b8e10b5.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-d73085abefd284c6.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-e6a09ca0044b34e5.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-e07333f48f53c71e.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-2a49b0d9fbc7a459.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-b66dda66aafe36c9.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-34d0b58da984bf31.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-01275baa20724171.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: duplicate symbol '_main' in:
              /Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.x8w4534r50augy0.rcgu.o
              /Users/n8henrie/git/atomicparsley-sys/target/debug/build/atomicparsley-sys-6b2017ea8a0d6804/out/libatomicparsley.a(main.o)
          ld: 1 duplicate symbol for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

On linux, I'm using g++ and getting essentially the same error:

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-arch" "arm64" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.16sx213jmsoo6x85.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.1to3wvar9qk3tdwr.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2e5zn6my1q0brh9m.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2ni19pr22er02fvl.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2qx04b2h1d2iihd3.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.2zbipg7xh1mwjl6g.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.3j75uj13wzlhfx3c.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.3kss1ezdynmn1zxq.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4b3fkvqvo3qf4gon.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4r3aw9u41fwno8hq.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4w1qn8f1cxjfrvyn.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4xp5ypzgr3qz7rj9.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.55o8cjtsw9h71z2g.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.x8w4534r50augy0.rcgu.o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.4w5vy4g6t2s070vm.rcgu.o" "-L" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps" "-L" "/Users/n8henrie/git/atomicparsley-sys/target/debug/build/atomicparsley-sys-6b2017ea8a0d6804/out" "-L" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-Wl,-force_load" "-Wl,/Users/n8henrie/git/atomicparsley-sys/target/debug/build/atomicparsley-sys-6b2017ea8a0d6804/out/libatomicparsley.a" "-lc++" "-framework" "Foundation" "-framework" "Cocoa" "-framework" "IOKit" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libtest-06ff11d645105e2b.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libterm-f05ecaeca66814e6.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgetopts-6ad0d034fc8e15c7.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunicode_width-c4beacdc35405cb8.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_std-fc0f4647def297a8.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-000cdec9267bfd7b.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-2669f3cbce8358f4.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-86461a1c60728ccb.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-f17574752cb5ddba.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-3bb606c936cc0d28.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-8139a4b0cda20184.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-ea0823eca3e9abf9.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-4f19e1259f6028e7.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-435daca85b8e10b5.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-d73085abefd284c6.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-e6a09ca0044b34e5.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-e07333f48f53c71e.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-2a49b0d9fbc7a459.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-b66dda66aafe36c9.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-34d0b58da984bf31.rlib" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-01275baa20724171.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/n8henrie/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: duplicate symbol '_main' in:
              /Users/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-9e5ca01b129fc97c.x8w4534r50augy0.rcgu.o
              /Users/n8henrie/git/atomicparsley-sys/target/debug/build/atomicparsley-sys-6b2017ea8a0d6804/out/libatomicparsley.a(main.o)
          ld: 1 duplicate symbol for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

On Linux (x86_64), I'm compiling fine:

$ g++ --version
g++ (GCC) 11.1.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ g++ -DPACKAGE_VERSION='"foo"' -DBUILD_INFO='"bar"' src/*.cpp
...
$ ./a.out --version
AtomicParsley version: foo bar (utf8)

But I get the same error (once I comment out the frameworks in build.rs, which I'll put behind a target flag I imagine).

  = note: /usr/bin/ld: /home/n8henrie/git/atomicparsley-sys/target/debug/build/atomicparsley-sys-229cafcd421f7f8e/out/libatomicparsley.a(main.o): in function `main':
          /home/n8henrie/git/atomicparsley-sys/vendor/src/main.cpp:4477: multiple definition of `main'; /home/n8henrie/git/atomicparsley-sys/target/debug/deps/atomicparsley_sys-151331ae4d429589.2q32se0uw345kcst.rcgu.o:2q32se0uw345kcst:(.text.main+0x0): first defined here
          collect2: error: ld returned 1 exit status

Also, if there are any style issues please let me know! I'm currently working on getting things to compile correctly with just the ShowVersionInfo as a hello world, then expand to the rest of the app.

TIA for any ideas on why this compiles (links?) from the CLI but not from cc-rs!

Resources for other learners:

2

I forgot to mention -- is it possibly because when compiled from the CLI directly, it's using the ifdefs to make it so that only one int main is compiled? If so, is there a way I should be setting these manually with cc?

3

If your build script fails, you'll see that cc prints out a bunch of stuff as it is going along. That means if you were to deliberately add a panic!() at the bottom of your build script's main() function you'll see exactly which commands it is running.

The duplicate symbol '_main' implies that both your program and atomicparsley are trying to define your test program's entry point... Does the vendor/src/main.cpp contain a main() function, by any chance? If so, you should probably leave that file out or patch atomicparsley so the function won't be compiled and libatomicparsley.a will be a library, not an executable.

4

Oh!

I thought it was complaining that there were multiple int main()s defined in the C++ codebase and erroring when trying to (first) compile just the C++ codebase (prior to any of my rust code). That's why I was so confused why it would compile from CLI but not from cc.

Previously in trying to debug this, I tried renaming the three different int main() functions to int main123, int main456, and int main789. cargo test works fine in this case, which I had previously thought was because I gave them each a unique name so they wouldn't conflict with each other.

However, after your explanation, I revisited this approach and then individually removed the suffix. Sure enough, with the two that are protected by ifdef, it makes no difference whether or not I add a suffix. The only one that matters (since I'm compiling from MacOS at least) is this one -- which must be conflicting with the rust entrypoint. Huzzah! Thanks a ton!

I just wish I would have asked about 10 hours of debugging ago :slight_smile:

I was wondering how that worked. Is that the usual approach to using FFI with projects that were originally executables -- to apply a patch to the original codebase, which then gets (manually?) reapplied to maintain compatibility going forward?

5

@Michael-F-Bryan I thought I recognized your username. Thanks for https://adventures.michaelfbryan.com/posts/parsing-pdfs-in-rust/, I love that post!

1 Like
6

I dunno, it depends on the project I guess. If it were me, I would avoid using an executable as a library and patching out main() because there's a strong possibility that it'll be tailored for a CLI program and not acceptable for use in a library.

For example, a lot of CLI programs (especially those written in C) will handle errors by printing the error message and calling exit() to halt with an appropriate exit code. That's perfectly acceptable in a CLI program because it simplifies error handling and general control flow (imagine never needing to return Result in your Rust), but if a library I am using aborts my program the moment things go wrong instead of returning an error I can handle gracefully, I'm going to throw it out quick, fast, and in a hurry.

That said, on Windows I was once told by a software vendor that they provided both an executable and a library, then when I asked if I could use their library so I could use their internal functions and datastructures (I wanted better control over inputs and error handling), their instructions were "use LoadLibrary() to load some-program.exe as a library at runtime, then call its main() function". So I guess it happens in the real world, too :man_shrugging:

Also... When you think about it, a library and a program are just a bunch of functions (ELF executables just have e_entry set to _start). We don't even differentiate between the two when compiling from the command-line because Linux doesn't care if you change the filename with -o. So in theory, if you did want to convert an executable into a library, removing the main() function (or just not compiling main.cpp at all) ought to do it.

:smiling_face_with_three_hearts:

Amusingly enough, I texted the website's author the night I was writing that tool and the day after I published the article he'd made an update so our contact list gets sent out in CSV form with the weekly newsletter.

I got 1 use out of the hack tool before it was replaced by a proper solution :joy:

2 Likes
7

Well hopefully this will at least be better than my current approach of shelling out as a Command :slight_smile:

Yes, I had figured out that workaround, but it looks like a lot of work happens in real_main, which is in main.cpp. I guess a patch is probably the way to go.

I was just thinking -- as this is a --lib project, I don't have a fn main defined. Where is this secret main coming from? Must be an effect of cargo test, since cargo build works. Is there any way to have cargo test do some kind of name mangling?

8

The compiler generates a main() function which will execute your tests. The easiest way to see this is with cargo expand:

$ cd /tmp
$ cargo new --lib temp
     Created library `temp` package
$ cd temp && mkdir tests
$ echo '#[test] fn dummy_test() {}' > tests/dummy_test.rs
$ cargo expand --test dummy_test
    Checking temp v0.1.0 (/tmp/temp)
    Finished dev [unoptimized + debuginfo] target(s) in 0.70s

#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
extern crate test;
#[cfg(test)]
#[rustc_test_marker]
pub const dummy_test: test::TestDescAndFn = test::TestDescAndFn {
    desc: test::TestDesc {
        name: test::StaticTestName("dummy_test"),
        ignore: false,
        allow_fail: false,
        compile_fail: false,
        no_run: false,
        should_panic: test::ShouldPanic::No,
        test_type: test::TestType::IntegrationTest,
    },
    testfn: test::StaticTestFn(|| test::assert_test_result(dummy_test())),
};
fn dummy_test() {}
#[rustc_main]
pub fn main() -> () {
    extern crate test;
    test::test_main_static(&[&dummy_test])
}

We could tell the compiler not to generate this main() function by explicitly adding the smoke_test integration test to Cargo.toml and setting the harness field to false, but then it would call atomicparsley's main() function when you run cargo test.

Which is probably not what you want.

The key part is that we load the binary at runtime using something like dlopen() or LoadLibrary() (see the libloading crate for a cross-platform wrapper) instead of linking to it at compile time.

That way you won't get duplicate symbol conflicts because your program and atomicparsley will be two separate binaries, but on the downside you'll need to make sure libatomicparsley.so is accessible at runtime.

Bindgen also has a method for generating bindings that use libloading under the hood. The feature was introduced in this PR if you want to see how it is meant to be used or learn about the feature's history.

1 Like
9

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.