Encoding problem of cmake path on windows


#1

Dear Rust users,

I am trying to compile a library with cmake from a build.rs file. The only response I get from cargo build is build script output was not valid utf-8. I run Rust on a Windows 10, 64bit, msvc version.

I am new to Rust so hopefully there is an obvious solution that I haven’t found yet. I have searched the web for some time and was only able to find a discussion where people had strange names in their path. Mine, however, should only include “normal” characters (path is C:\Users\My-Name\Documents\dev\mycrate\mycrate-sys\target\debug\build\mycrate-sys-000682e067ebf920\out).

I’d be very happy if someone could tell me how to resolve the problem.


#2

Did you escaped the backslashes? Like C:\Users\Name\ should be included in the source code as C:\\Users\\Name\\.

If you did, can you post the content of your build.rs script ?


#3

Thank you for the quick reply. I assumed, the cmake crate does this for me.

My build.rs looks like the example in the cmake crate:

extern crate cmake;

fn main(){
    let dst = cmake::build("libzmq");

    println!("cargo:rustc-link-search=native={}", dst.display());
    println!("cargo:rustc-link-lib=static=zmq");
}

Since I wasn’t able to see the output of the build script, I inserted the following lines in the middle to save the content of the path to a file.

let mut f = File::create("lib.log").unwrap();
let name: String = format!("{}", dst.display());
f.write(&name.into_bytes());

I just tried to replace every backslash with two with the idea that maybe there is a problem with parsing them, but had no luck. :frowning:


#4

Well, that looks strange. What is the content of lib.log here?


#5

C:\Users\My-Name\Documents\dev\zmq\zmq-sys\target\debug\build\zmq-sys-000682e067ebf920\out


#6

Well, I am out of ideas here. It might be an issue with cargo trying to parse the output of the build script.

A way to test that would be to replace all \ by / in the path, but I did not find the needed methods for that.


#7

I might have found something. I took a close look at the build directory and found that I can see everything that the build script outputs in a file called output. It seems like the cmake output is not suppressed but forwarded. There are symbols that can’t be resolved, as cmake outputs explanations in my language (german).
I will see what can be done there. Right now, I am installing a different language for my windows and hopefully, cmake will follow.

Thank you for your help. You helped me look in the right place and understand the build process a bit better. :slight_smile:


#8

I’m glad I helped you! I think this is a bug in cargo, which should not try to convert the captured output of build tools to str, but keep OsStr. You should fill a bug about it :slight_smile:


#9

OsStr is not quite appropriate, as on Windows OsStr is UTF-16 but standard output is still a stream of bytes, so Cargo has a [u8] in the system code page. The quick and dirty fix would be to use String::from_utf8_lossy in custom_build.rs; I’m sure there’s a way to do the ISO-8859-1 -> UTF-8 conversion correctly, but it’s been too long to remember what that was.


#10

[quote=“sorear, post:9, topic:5333”]
on Windows OsStr is UTF-16
[/quote]It isn’t. You can convert from str with zero cost (there is impl AsRef<OsStr> for str) on any platform.


#11

After a reinstall of ms visual studio, this time the english version, I am now able to compile everything as expected.

So for everyone with similar problems: Don’t use a localized version of msvc.

I opened a new issue for this.
The lossy conversion would work but the build process might break if there are non-utf-8 characters in lines starting with cargo:, I suppose. In the best case, the build script output is split into lines first and only those lines starting with cargo: are checked for their utf-8 correctness.