Hello rustoceans! I'm making a game streamer such as Geforce Now or Google Stadia. The host start server, make password and choose a game to stream. The client can join to server and "play" in the game.
Live encoding and streaming is a relatively tricky task just based one the sheer amount of data you have to handle. I don't have an immediate solution to your problem, but I wrote some similar code for Android (minus the streaming part, it just encodes a game in real-time to an H.264 video).
The most important point to get it to work fast was to forego most of the CPU<->GPU communication and keep the frames on GPU. I am not certain how the situation looks like on Desktop machines (there seems to be NVENC on Nvidia GPUs for example), but Android basically offers you a hardware H.264 encoder. I ended up using OpenGL to blit my framebuffer directly to the hardware encoder's input buffer. The only thing touching the CPU is the final encoded H.264 stream, which you can then forward to the client.
Hope that helps!