Anyone can explain why xml2 not loaded

I try to build this project

After building I try to load library to python and getting error

from ctypes import *

DLLNAME = "../xdrk/target/release/build/xdrk-94dcb1f6d064f882/out/lib/libxdrk-x86_64.so"

XRKDLL = cdll.LoadLibrary(DLLNAME)
Traceback (most recent call last):
  File "/app/test_xrk.py", line 1, in <module>
    import xrk
  File "/app/xrk.py", line 18, in <module>
    XRKDLL = cdll.LoadLibrary(DLLNAME)
  File "/usr/lib/python3.7/ctypes/__init__.py", line 434, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.7/ctypes/__init__.py", line 356, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /xdrk/target/release/build/xdrk-94dcb1f6d064f882/out/lib/libxdrk-x86_64.so: undefined symbol: xmlFree

What I did wrong? I've already rechecked everything I can.

I'm using debian. libxml2 and libxml2-dev installed

maybe the LD_LIBRARY_PATH for your python envrionment isn't setup correctly. did you try to load the libxml2 library directly?

first get the linker resolved dependency name, you get something like this:

~$ ldd xxxxxx.so | grep libxml2
    ...
    libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f3a04d59000)
    ...

try load the library in your python environment, you should see something like this:

>>> cdll.LoadLibrary("libxml2.so.2")
<CDLL 'libxml2.so.2', handle 5586b99ee2c0 at 0x7fda4392ac20>

since cdll.LoadLibrary just pass the argument to the dlopen system library, python doesn not know the exact reason the search failed, you should refer to the dlopen man page.

@nerditation Thank you so much for answering my question! Here is the command output and I see that libxml2.so.2 is missing in the ldd output, yet it is available in the system and it shows in the output from Python. I'm sorry I don't understand anything about Rust at all and I don't know what LD_LIBRARY_PATH is and where and what value I should set it to.

~$ ldd libxdrk-x86_64.so
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fffff843000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fffff6be000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fffff6a4000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fffff4e4000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ffffffd6000)
Python 3.7.3 (default, Oct 31 2022, 14:04:00)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> cdll.LoadLibrary("libxml2.so.2")
<CDLL 'libxml2.so.2', handle 958ab0 at 0x7fffff9fc1d0>
>>>

It looks like libxdrk-x86_64.so depends on libxml2.so but doesn't declare this dependency using DT_NEEDED. The xdrk rust crate does add a dependency on it to fix this, but if it is opened with RTLD_LOCAL as python's ctypes module does by default, libxdrk-x86_64.so still won't be able to find libxml2.so. I'm not a python expert, but I you could try CDLL(DLLNAME, mode=RTLD_GLOBAL) or something like that instead of cdll.LoadLibrary(DLLNAME)

I believe @nerditation and @bjorn3 both have a piece of the puzzle here.

You should first load libxml2 into your program using the RTLD_GLOBAL mode and only then load libxdrk.

from ctypes import *

dll = "/path/to/libxdrk-x86_64.so"

# first do this
CDLL('libxml2.so.2', mode=RTLD_GLOBAL)

# now the load of libxdrk should succeed
xdrk = cdll.LoadLibrary(dll)

# now you can call the functions defined in src/bindings.rs
# like open_file, etc
filename = "./testdata/path/to/testfile.drk".encode("utf-8")
xdrk.argtypes = [c_char_p]
ret = xdrk.open_file(filename)
print(ret)

# there will be some other errors like a missing XML file
# which you will need to solve
1 Like

after a second inspection, I realized what OP was trying to load was not the rust binding, but rather the pre-built C library (which is copied to OUT_DIR by build.rs). the C library is a pre-built binary file(not built from source), it is under the aim directory of the repository. in such case, the added linker flag only affect rust code built by cargo.

so the issue actually has nothing to do with rust or cargo, this crate is just a wrapper for ffi bindings to their C library, rust code that depends on this crate builds and links just fine (but they don't have 32 bit support).

yes, manually load the missing library into global scope is a correct solution for such situation. you can achieve the same effect using LD_PRELOAD. another alternative is to build a wrapper library which is linked properly.

It worked! Thank you so much @hax10 ! I was missing this line:

CDLL('libxml2.so.2', mode=RTLD_GLOBAL)

Admittedly, I've heard that the Rust developer community is such a nice and responsive community, but I didn't believe it was true!

Do you know if it is possible to compile this project(xdrk) for arm64?

As @nerditation pointed out above, the .so file is pre-built. I assume the source code is not available, so you would have to contact the device manufacturer who designed the file format and ask if they have a shared library for ARM 64.

The source codes from the manufacturer are publicly available here. But the problem is that it is a Visual Studio project and my knowledge is not enough to compile it for ARM :disappointed:

Unfortunately, that download is only partly open-source. The main functionality is pre-compiled.

If you are running Windows on ARM-64, you could probably run the 32-bit or 64-bit DLL in that download using emulation.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.