I have an idea for a small project and I was wondering what would be my best bet for overlaying/rendering visuals (images and text in particular) on top of other full screen Windows applications/games (probably DirectX 11, 12). I had a quick look but I couldn't find much. Anyone had any success in doing that?
I imagine my best chance would be writing Rust bindings to a C++ library that does that. Thanks!
For note, this is extremely hard. The usual way it's done (like via steam as in your image) is DLL hooking to hook the normal render calls so it can inject it's own. It's very platform dependent, rendering API dependent, among more. The easiest way would be to just use an existing C/C++ library, rewriting it is a pretty huge task (though very welcome).
Cooperative is of course a way, but steam uses library injection and hooking. The Steamworks API allows the program to control it a bit more (like turn it on or off and various other things) but the rendering itself is still done via the library injection (otherwise it wouldn't work with everything).
Yes, it is indeed hard and I have no experience with the windows API and not much experience with anything else except OpenGL when it comes to graphics.
For the overlay part, I did look into some libraries and asked on discord, and currently I am giving https://github.com/nefarius/Indicium-Supra a look. It also has a imgui integration example, which is very useful as I might be able to make some bindings for it then use them in conjunction with imgui-rs. For the application part, I might as well use a Rust only GUI library, even though I don't think any of them are mature enough if I want some more complex widgets/components or more complex styling.
Another way is to actually build an Electron + wasm + Rust application. I am not a big fan of Electron and I would prefer to have a small and fast application, but in Electron I should be able to either:
(but also) create a window that's always on top, no decoration, transparent background, "input transparency" but opaque controls (that's what HearthstoneDeckTracker is doing in C#/WPF by setting things like Topmost=true, AllowTransparency=true, ShowInTaskbar=false, WindowStyle=None, ShowInTaskbar=False)
For the second point, I wonder if I would actually be able to achieve the same using winit to create the window and imgui-rs for the UI, although from what I can see, I don't think I would be able to duplicate the input transparency part. The UI needs be hovereable, displaying tooltips and click on it should not further send a click command to the game, but if someone is clicking anywhere else where it's just a transparent background, the game should receive the input.
I'm still exploring and experimenting, any feedback is appreciated. Thanks!
I was able to achieve the equivalent of C#/WPF functionality by obtaining the raw window handle and setting the right flags myself. That means I should be able to use imgui-rs with winit for the overlay part.
However, I'm not really keen on using imgui-rs for my whole UI, so I will have to find another solution for that. Thanks for your help!
Development seems to have gone a bit dormant in recent months, but itch.io are/were working on a screen capture tool in Rust, which involves some similar hooking into graphics APIs. There might be some helpful code to reference there, if you decide to go that route: https://github.com/itchio/capsule
They seem to be using a crate called detour to do the hooking.
Yes, I did came across detour multiple times and I think it's great for the hooking part. I was trying to see if there's something that works at higher level that, as I am not familiar with DirectX API at all and I wouldn't know what and when to hook. https://github.com/nefarius/Indicium-Supra mentioned earlier also uses detour and it aims to "eliminate the need for in-depth knowledge about Direct3D and API-hooking". Quote from their Github:
Indicium-Supra consists of a self-contained library (DLL) which exposes a minimalistic API for rendering custom content in foreign processes eliminating the need for in-depth knowledge about Direct3D and API-hooking.
I will explore with winit + imgui-rs first, as it seems it will get me further with less effort (side project to which I can not allocate as much time as I'd like to ).