Linking custom cuda kernel to rust project


#1

Hi!

I’ve been struggling to link my custom cuda kernel to my rust project,
My first attempt leaded to this unanswered question on stackoverflow :


Now I am trying to do it manually, I run these commands to generate a library from my cuda kernel :
nvcc --compiler-options ‘-fPIC’ -Wno-deprecated-gpu-targets -ccbin clang-3.8 -rdc=true -c kernel.cu
nvcc --compiler-options ‘-fPIC’ -Wno-deprecated-gpu-targets -ccbin clang-3.8 -dlink -o kernel_link.o kernel.o -lcudadevrt -lcudart
ar crus libkernel.a kernel.o

And I have this build.rs :
fn main() {

    println!("cargo:rustc-link-lib=dylib=cuda");
    println!("cargo:rustc-link-lib=dylib=cudart");
    println!("cargo:rustc-link-lib=dylib=cublas");
    println!("cargo:rustc-link-lib=dylib=curand");

    println!("cargo:rustc-link-search=native={}", "kernel");
    println!("cargo:rustc-link-lib=static=kernel");
}

But I have this error :

error: linking with cc failed: exit code: 1
|
= note: “cc” “-Wl,–as-needed” “-Wl,-z,noexecstack” “-m64” “-L” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib” [SOME_OBJECT_FILES] “-Wl,–gc-sections” “-pie” “-Wl,-z,relro,-z,now” “-nodefaultlibs” “-L” “/home/ltei/Dev/Workspaces/rust_cumatrix/target/debug/deps” “-L” “kernel” “-L” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib” “-l” “cuda” “-l” “cudart” “-l” “cublas” “-l” “curand” “-Wl,-Bstatic” “-Wl,–whole-archive” “-l” “kernel” “-Wl,–no-whole-archive” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libtest-2710c660d8990d9e.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgetopts-43a4de39b20b8116.rlib” “/home/ltei/Dev/Workspaces/rust_cumatrix/target/debug/deps/liblibc-dca5860987df25ef.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libterm-a8e72805fef1512a.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-0006dc6e9901bcad.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-8d1c3982c0670998.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_jemalloc-2a12cd93029b9807.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-a5d3ff19e13d9f37.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_system-c5f69e7df1f06d84.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-e2e7ce88a6c41eea.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-3d7473d271611dc2.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_unicode-58e7a51af24928de.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6806ae6018eec5e7.rlib” “/home/ltei/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-85fd2d595ec0faf9.rlib” “-Wl,-Bdynamic” “-l” “util” “-l” “util” “-l” “dl” “-l” “rt” “-l” “pthread” “-l” “pthread” “-l” “gcc_s” “-l” “c” “-l” “m” “-l” “rt” “-l” “pthread” “-l” “util” “-l” “util”
= note: kernel/libkernel.a(kernel.o) : Dans la fonction « __sti____cudaRegisterAll_41_tmpxft_0000443a_00000000_7_kernel_cpp1_ii_0df2aa9a() » :
/tmp/tmpxft_0000443a_00000000-13_kernel.ii:(.text+0x214) : référence indéfinie vers « __cudaRegisterLinkedBinary_41_tmpxft_0000443a_00000000_7_kernel_cpp1_ii_0df2aa9a »
collect2: error: ld returned 1 exit status

Do you have any idea on how to link a custom kernel to a rust project?

Edit : I fixed this issue! If you have this problem, go check this working example :


#2

Interesting challenge!

I guess you may see the following SO questions, which say this link error related to the Dynamical Parallelism of CUDA.


I’ve try it with “vectorAdd.cu” in the CUDA sample, but it works. Could you give a minimal example “kernel.cu” to reproduce this error?


#3

Thank you for your answer! Unfortunately I already went to these posts, that is where my manual commands come from, also I tried to link the kernel object files to a C program, and it works fine!
Here is my kernel.cu file :
#include "kernel.h"

__global__ void vectorAdd_ker(float* vector, int len, float value) {
      int tid = blockIdx.x * blockDim.x + threadIdx.x;
      if (tid < len) { vector[tid] += value; }
}

void Kernel_vectorAdd(float* vector, int len, float value) {
   dim3 gridDim;
   dim3 blockDim;

   blockDim.x = 1024;
   gridDim.x = (len + blockDim.x - 1) / blockDim.x;

  vectorAdd_ker <<<gridDim, blockDim>>> (vector, len, value);
}

int Kernel_test() {
  return 7;
}

I tried to link the vectorAdd.cu file instead of my kernel.cu, but the error is the same!


#4

Ok I can compile now!
I forgot to add the link object file in the last command that creates a library from the kernel
So it’s : ar crus libkernel.a kernel.o kernel_link.o
Instead of : ar crus libkernel.a kernel.o

I can compile and everything is ok, I can declare Kernel_vectorAdd in an extern block in Rust, but when I try to use it, I got an undefined reference error


#5

Hey! Would it be possible for you to paste your whole working example with vectorAdd.cu?
I tried an other method ;
I linked nvcc object files to a c code, and it works fine,
Then I linked gcc object files to rust and it works fine,
But when I try to link c object files which are linked to nvcc object files to rust, I have an error


#6

Sorry for replaying so late (´・ω・`)
I’ve uploaded a repository for this:


I confirmed that it works on my container with nvidia-docker and GTX 1050.