MS UIAutomation in Rust windows project

I am trying to use MS UIAutomation in my Rust windows project. I've found out your project here: https://lib.rs/install/uiautomationcore-sys

But unforcedly, I cannot find out any documentation of how to use it.

My goal is to get positions of taskbar buttons at the Windows taskbar. Something like at the PowerToys project:

  winrt::com_ptr<IUIAutomationElementArray> elements;
  winrt::com_ptr<IUIAutomationElement> child;
  ...
  if (VARIANT var_rect; child->GetCurrentPropertyValue(UIA_BoundingRectanglePropertyId, &var_rect) >= 0) {
  if (var_rect.vt == (VT_R8 | VT_ARRAY)) {
    LONG pos;
    double value;
    pos = 0; SafeArrayGetElement(var_rect.parray, &pos, &value);
    button.x = (long)value;
    pos = 1; SafeArrayGetElement(var_rect.parray, &pos, &value);
    button.y = (long)value;
    pos = 2; SafeArrayGetElement(var_rect.parray, &pos, &value);
    button.width = (long)value;
    pos = 3; SafeArrayGetElement(var_rect.parray, &pos, &value);
    button.height = (long)value;
  }

Or in the FastWindowSwitcher project:

  VARIANT propertyValue = { 0 };
  HRESULT hr = p_element.GetCurrentPropertyValue(UIA_BoundingRectanglePropertyId, &propertyValue);
  ...
  double* data = nullptr;
  hr = SafeArrayAccessData(propertyValue.parray, (void**)&data);
  ...
  QRect rectangle(data[0], data[1], data[2], data[3]);
  ...
  return rectangle;

Maybe there is a better way to do it now?

Thanks,
Yura

This would be a @retep998 question, but I think you might be out of luck - it doesn't look like winapi exports the UI Automation libraries. retep might be able to point you in the right direction, though, in terms of contributing the bindings yourself. You could then use winapi in combination with something like comedy to get similar code to the PowerToys example.

1 Like

uiautomationcore-sys was a placeholder from the winapi 0.2 era when each import library had a separate sys crate. Nowadays all bindings are simply in winapi directly.
That said, the UI automation stuff simply has not been bound yet in winapi so it would need someone to contribute bindings and then for that pull request to be reviewed and merged.

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.