Linking custom cuda kernel to rust project

help
1

Hi!

I've been struggling to link my custom cuda kernel to my rust project,
My first attempt leaded to this unanswered question on stackoverflow :

Now I am trying to do it manually, I run these commands to generate a library from my cuda kernel :
nvcc --compiler-options '-fPIC' -Wno-deprecated-gpu-targets -ccbin clang-3.8 -rdc=true -c kernel.cu
nvcc --compiler-options '-fPIC' -Wno-deprecated-gpu-targets -ccbin clang-3.8 -dlink -o kernel_link.o kernel.o -lcudadevrt -lcudart
ar crus libkernel.a kernel.o

And I have this build.rs :
fn main() {

    println!("cargo:rustc-link-lib=dylib=cuda");
    println!("cargo:rustc-link-lib=dylib=cudart");
    println!("cargo:rustc-link-lib=dylib=cublas");
    println!("cargo:rustc-link-lib=dylib=curand");

    println!("cargo:rustc-link-search=native={}", "kernel");
    println!("cargo:rustc-link-lib=static=kernel");
}

But I have this error :

error: linking with cc failed: exit code: 1
|
= note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" [SOME_OBJECT_FILES] "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" "-L" "/home/ltei/Dev/Workspaces/rust_cumatrix/target/debug/deps" "-L" "kernel" "-L" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "cuda" "-l" "cudart" "-l" "cublas" "-l" "curand" "-Wl,-Bstatic" "-Wl,--whole-archive" "-l" "kernel" "-Wl,--no-whole-archive" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libtest-2710c660d8990d9e.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgetopts-43a4de39b20b8116.rlib" "/home/ltei/Dev/Workspaces/rust_cumatrix/target/debug/deps/liblibc-dca5860987df25ef.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libterm-a8e72805fef1512a.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-0006dc6e9901bcad.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-8d1c3982c0670998.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_jemalloc-2a12cd93029b9807.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-a5d3ff19e13d9f37.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_system-c5f69e7df1f06d84.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-e2e7ce88a6c41eea.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-3d7473d271611dc2.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_unicode-58e7a51af24928de.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6806ae6018eec5e7.rlib" "/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-85fd2d595ec0faf9.rlib" "-Wl,-Bdynamic" "-l" "util" "-l" "util" "-l" "dl" "-l" "rt" "-l" "pthread" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util" "-l" "util"
= note: kernel/libkernel.a(kernel.o) : Dans la fonction « __sti____cudaRegisterAll_41_tmpxft_0000443a_00000000_7_kernel_cpp1_ii_0df2aa9a() » :
/tmp/tmpxft_0000443a_00000000-13_kernel.ii:(.text+0x214) : référence indéfinie vers « __cudaRegisterLinkedBinary_41_tmpxft_0000443a_00000000_7_kernel_cpp1_ii_0df2aa9a »
collect2: error: ld returned 1 exit status

Do you have any idea on how to link a custom kernel to a rust project?

Edit : I fixed this issue! If you have this problem, go check this working example :

1 Like
2

Interesting challenge!

I guess you may see the following SO questions, which say this link error related to the Dynamical Parallelism of CUDA.
https://stackoverflow.com/questions/22115197/dynamic-parallelism-undefined-reference-to-cudaregisterlinkedbinary-linking
https://stackoverflow.com/questions/22076052/cuda-dynamic-parallelism-makefile

I've try it with "vectorAdd.cu" in the CUDA sample, but it works. Could you give a minimal example "kernel.cu" to reproduce this error?

3

Thank you for your answer! Unfortunately I already went to these posts, that is where my manual commands come from, also I tried to link the kernel object files to a C program, and it works fine!
Here is my kernel.cu file :
#include "kernel.h"

__global__ void vectorAdd_ker(float* vector, int len, float value) {
      int tid = blockIdx.x * blockDim.x + threadIdx.x;
      if (tid < len) { vector[tid] += value; }
}

void Kernel_vectorAdd(float* vector, int len, float value) {
   dim3 gridDim;
   dim3 blockDim;

   blockDim.x = 1024;
   gridDim.x = (len + blockDim.x - 1) / blockDim.x;

  vectorAdd_ker <<<gridDim, blockDim>>> (vector, len, value);
}

int Kernel_test() {
  return 7;
}

I tried to link the vectorAdd.cu file instead of my kernel.cu, but the error is the same!

4

Ok I can compile now!
I forgot to add the link object file in the last command that creates a library from the kernel
So it's : ar crus libkernel.a kernel.o kernel_link.o
Instead of : ar crus libkernel.a kernel.o

I can compile and everything is ok, I can declare Kernel_vectorAdd in an extern block in Rust, but when I try to use it, I got an undefined reference error

1 Like
5

Hey! Would it be possible for you to paste your whole working example with vectorAdd.cu?
I tried an other method ;
I linked nvcc object files to a c code, and it works fine,
Then I linked gcc object files to rust and it works fine,
But when I try to link c object files which are linked to nvcc object files to rust, I have an error

6

Sorry for replaying so late (´・ω・｀)
I've uploaded a repository for this:
https://github.com/termoshtt/link_cuda_kernel
I confirmed that it works on my container with nvidia-docker and GTX 1050.