Virtualized GPU that executes remotely and keeping your data local

help
1

sorry if I posted this again, but this has more context on what I'm trying to build and libs what I'm trying to use

The idea is to build something like this:

Virtualization for GPU that allows you to run local GPU apps and the code is actually run in the cloud, keeping your data local, without changing your apps, at least not much.

Functionality:

  • vGPU is a virtualization layer for a GPU
  • your local app "runs" on local vGPU
  • local app decrypts the actual local data and sends the (CUDA) instructions to the remote GPU-Coordinator
  • GPU-Coortdnator distribute the instructions to multiple real GPUs
  • then it sends the results back to vGPU which sends them to the local app

Three advatages emerge:

  • You can execute AI models without haing a local GPU
  • Execute on multiple GPUs, in the end, for some use cases, it could be faster than running on a local GPU
  • Execute ML models without storing your data on untrusted providers. It will keep some blocks in memory which are needed for executing current instructions. But at least they are obfruscated and protected by OS not to be easily copied by other processes from memory. Data in transit is secured by TLS

I know it will be slow, but in cases where the data flow is small compared to processing time it could be a reasonable compromise for the security it gives you.

Also because instructions are distributed to multiple GPUs, when possible, it could offer better performance, in some cases, than locally

schema rvirt-gpu/website/resources/schema2.png at main · radumarias/rvirt-gpu · GitHub

implementation ideas Implementation · radumarias/rvirt-gpu Wiki · GitHub

I know how to move the instructions and results around and also how to execute them on real GPU, but I don't know how to build the driver for vGPU and what libs to use.

2

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.