Mouse cursor detection in gui by using bitmap or color

Based on the answers from these topics:

I learned how a 2D graphics library (such as tiny skia) is used to draw widgets as a button, while a window handler (such as winit) is used to handle events such as window events and mouse and keyboard events, and then an intermediate between the two libraries is used to link them (such as softbuffer )

Now I'm trying to figure out how to use bitmap in order to check if the mouse cursor touches a widget to trigger the mouse event, I need an example for that.

I searched the net and found titles that might be related to the topic:

-bitmap collision detection.
-Per-pixel collision detection.
-Image based collision detection.

Are they the same title, or are there differences between them in terms of practical application?

If you specifically want image based detection, you could draw each element with a unique color (basically count up from #000000, #000001, etc.) and without transparency, text or anti aliasing. Just the shapes, so it doesn't blend the colors. Then check the color of the pixel under the cursor to see which element was clicked. This may or may not be an outdated technique, though. I'm not up to speed.

A probably simpler and faster technique is to check if the cursor is within the bounding box of any element. This works best if they are mostly rectangular or any other simple shape. You have to take care of layering and overlap, though.

1 Like

There is another thing I forgot to mention. I want to use the method you mentioned, knowing that the widgets may share the same color or theme shown to the user, this may be related to the offscrean buffer

Correct, this technique requires multiple render passes. Once with "colors" which are really just unique identifiers which allows you to determine which widget contains the pointer by extracting the "color" at that point. That render pass would never be displayed on the window (either because the buffer is overwritten with the final render before it gets displayed, or because a completely separate buffer is used)

3 Likes

Is there a written example of this using the softbuffer library?

I recommend just picking an approach and trying. You can learn a lot by reading examples, especially learning about the many alternate ways to do things, but nothing compares to figuring out something (anything) by yourself, even when what you produce is far from your ideals.

The problem is that whenever I look for resources related to what I'm trying to do, algorithms or examples that I can't find, it seems that there are very few resources related to writing a custom gui.

I would be very grateful if someone would share good resources on the subject

How does the mouse cursor interact with the bitmap if it is in a separate buffer and is not visible? And how to create an offscreen buffer?

I doubt you'll find resources covering how to do it in the specific language you want with the specific libraries (or library characteristics) you want, with the specific techniques that you select. I suspect you need some basics first, such as how bitmaps can be represented in memory, how to address individual pixels within, and how to adjust to the different formats used by different libraries (see the for loop in my example). Resources which teach the basics don't usually focus on how to create a gui library, but I suspect they may exist somewhere for some other language.

The mouse cursor never inherently “interacts with” any bitmap that is visible. Your application receives mouse position coordinates in events, and you can then write code that does whatever you want — such as checking the value at those coordinates the other bitmap.

3 Likes

The steps would be

  1. Get the coordinates of the mouse cursor
  2. Transform the coordinates into the same coordinate system your hit test buffer uses, if necessary
  3. Get the value of the hit test buffer at that coordinate
  4. Convert the value you get from the buffer back into whatever widget identifier you're using

Now you know what widget the mouse is currently over.

1 Like

I found these pages that show this technique in a 3D environment with OpenGL. The specific implementation will be different in your case, but the idea is the same. I hope they help illustrating how it works and give some clues towards how you can implement it.

Edit: Right, it's sometimes called a "visibility buffer" or an "ID buffer".