M1 Macs, Rust and openssl

ZiCog · January 23, 2021, 7:57pm

Anyone know how to install openssl on a Mac M1 so that I can get my projects that use the openssl crate to build?

My partner in business decided to splash some cache on new machines for us. When he suggested MacBook M1 I agreed, having heard good things of the new fangled ARM based Macs. And especially as my MS Surface Pro's battery exploded once and now the replacement only holds a charge for an hour or so. Great machine, puts my old big box PC to shame. Only problem is I have never used a Mac and have no idea how to drive it.

I managed to install rustup, Cargo and Rust using the same procedure I use for Debian. Happily "Hello World" compiled out of the box. But I can't get any of my projects that use openssl to build.

After much googling around I think I have MacPorts installed. And openssl:

    **✗** brew install openssl
    Updating Homebrew...
    ==> **Auto-updated Homebrew!**
    Updated 1 tap (homebrew/core).
    ==> **New Formulae**
    aliddns cloudflare-wrangler osmcoastline
    ==> **Updated Formulae**
    Updated 116 formulae.
    Warning: openssl@1.1 1.1.1i is already installed and up-to-date
    To reinstall 1.1.1i, run `brew reinstall openssl@1.1`
    **➜** **conq-sm-decoder** **git:(** **master** **)** **✗** brew reinstall openssl@1.1
    ==> **Downloading https://homebrew.bintray.com/bottles/openssl%401.1-1.1.1i.arm64_**
    Already downloaded: /Users/zicog/Library/Caches/Homebrew/downloads/f5bd034aadaad80de4a9374ea974c7698a93a5a96c2671052d309a778ad6deca--openssl@1.1-1.1.1i.arm64_big_sur.bottle.tar.gz
    ==> **Reinstalling** **openssl@1.1**
    ==> **Pouring openssl@1.1-1.1.1i.arm64_big_sur.bottle.tar.gz**
    ==> **Caveats**
    A CA file has been bootstrapped using certificates from the system
    keychain. To add additional certificates, place .pem files in
    /opt/homebrew/etc/openssl@1.1/certs
    and run
    /opt/homebrew/opt/openssl@1.1/bin/c_rehash
    openssl@1.1 is keg-only, which means it was not symlinked into /opt/homebrew,
    because macOS provides LibreSSL.
    If you need to have openssl@1.1 first in your PATH run:
    echo 'export PATH="/opt/homebrew/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc
    For compilers to find openssl@1.1 you may need to set:
    export LDFLAGS="-L/opt/homebrew/opt/openssl@1.1/lib"
    export CPPFLAGS="-I/opt/homebrew/opt/openssl@1.1/include"
    ==> **Summary**
    🍺 /opt/homebrew/Cellar/openssl@1.1/1.1.1i: 8,067 files, 18MB

On the other hand I have also tried this:

    **✗** sudo /opt/local/bin/port -N install openssl +universal
    ---> Computing dependencies for openssl
    ---> Dependencies to be installed: zlib
    ---> Fetching archive for zlib
    ---> Attempting to fetch zlib-1.2.11_0+universal.darwin_20.arm64-x86_64.tbz2 from https://cph.dk.packages.macports.org/zlib
    ---> Attempting to fetch zlib-1.2.11_0+universal.darwin_20.arm64-x86_64.tbz2.rmd160 from https://cph.dk.packages.macports.org/zlib
    ---> Installing zlib @1.2.11_0+universal
    ---> Activating zlib @1.2.11_0+universal
    ---> Cleaning lib
    ---> Fetching archive for openssl
    ---> Attempting to fetch openssl-1.1.1i_0+universal.darwin_20.arm64-x86_64.tbz2 from https://cph.dk.packages.macports.org/openssl
    ---> Attempting to fetch openssl-1.1.1i_0+universal.darwin_20.arm64-x86_64.tbz2.rmd160 from https://cph.dk.packages.macports.org/openssl
    ---> Installing openssl @1.1.1i_0+universal
    ---> Activating openssl @1.1.1i_0+universal
    ---> Cleaning openssl
    ---> Scanning binaries for linking errors
    ---> No broken files found.
    ---> No broken ports found.

Which may or may not have been a good idea. I have no idea.

But my project build fails to build:

> ✗ cargo build              
>    Compiling openssl-sys v0.9.60
> error: failed to run custom build command for `openssl-sys v0.9.60`
> Caused by:
>   process didn't exit successfully: `/Users/zicog/conveqs/conq-sm-decoder/target/debug/build/openssl-sys-29c757dffae9a1e2/build-script-main` (exit code: 101)
>   --- stdout
>   cargo:rustc-cfg=const_fn
>   cargo:rerun-if-env-changed=AARCH64_APPLE_DARWIN_OPENSSL_NO_VENDOR
>   AARCH64_APPLE_DARWIN_OPENSSL_NO_VENDOR unset
>   cargo:rerun-if-env-changed=OPENSSL_NO_VENDOR
>   OPENSSL_NO_VENDOR unset
>   --- stderr
>   thread 'main' panicked at 'don't know how to configure OpenSSL for aarch64-apple-darwin', /Users/zicog/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-src-111.6.0+1.1.1d/src/lib.rs:178:18
>   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Any advice for a Mac neophyte much appreciated.

blonk · January 23, 2021, 8:27pm

The openssl-sys crate tries to use pkg-config. First thing I would check is if pkg-config --cflags --libs returns the expected result.

uberjay · January 23, 2021, 8:42pm

I think this is the important part:

ZiCog:

    openssl@1.1 is keg-only, which means it was not symlinked into /opt/homebrew,
    because macOS provides LibreSSL.
    If you need to have openssl@1.1 first in your PATH run:
    echo 'export PATH="/opt/homebrew/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc
    For compilers to find openssl@1.1 you may need to set:
    export LDFLAGS="-L/opt/homebrew/opt/openssl@1.1/lib"
    export CPPFLAGS="-I/opt/homebrew/opt/openssl@1.1/include"

Although it's not mentioned, the homebrew-installed pkgconfig file (which allows the build process to discover the correct flags to pass to the C compiler) is also not automatically added to your search path. At least, it wasn't for me. My homebrew is installed in /usr/local/opt, so I've added this to my search path using the PKG_CONFIG_PATH environment variable:

$ echo $PKG_CONFIG_PATH
/usr/local/opt/openssl/lib/pkgconfig

Try building like so:

PKG_CONFIG_PATH=/opt/homebrew/opt/openssl/lib/pkgconfig cargo build

If it works, hooray! Add it to your ~/.zshrc (Based on your output, it looks likely to be your shell.)

ZiCog · January 23, 2021, 8:51pm

Whatever I have done I don't seem to have a pig-config:

    **✗** pkg-config --cflags --libs
    zsh: command not found: pkg-config

ZiCog · January 23, 2021, 8:56pm

I also don't have a $PKG_CONFIG_PATH

**✗** echo $PKG_CONFIG_PATH

**➜** **conq-sm-decoder** **git:(** **master** **)** **✗**

Where is one supposed to get all this stuff?

blonk · January 23, 2021, 9:13pm

I've only used pkgsrc myself, so I'm not sure how MacPorts works. But I would:

Make sure that MacPorts' bin directory is in PATH.
Make sure that pkg-config is installed.
Try to build the openssl crate again.

... and I say this while stressing that I haven't actually used MacPorts, but I assume it works sort of like pkgsrc when it comes to pkg-config.

notriddle · January 23, 2021, 9:14pm

There’s also the option of adding the vendored feature on rust-openssl.

ZiCog · January 23, 2021, 9:20pm

I already do. I have this in my Cargo.toml:

[dependencies]
...
openssl = { version = "0.10.32", features = ["vendored"] }
...

uberjay · January 23, 2021, 9:22pm

Sorry, you appear to be using both Homebrew and macports... I'm not sure that's a great idea. If you use homebrew, install pkgconfig (brew install pkgconfig).

PKG_CONFIG_PATH is there for you to explicitly add additional search paths for pkgconfig files -- it's not typically set by the OS.

Where you get all this stuff... not sure what you mean, but I learned the majority of this building software in Linux. Fortunately much of it carries over to macOS. (though, not all... )

uberjay · January 23, 2021, 9:24pm

Ah, well, maybe all of this pkgconfig business isn't relevant, if you're trying to build the vendored openssl... in that case, I am not sure what to suggest!

ZiCog · January 23, 2021, 10:49pm

So I removed that "vendored" from my openssl dependency. BOOM! everything builds.

Now I don't recall why I put the "vendored" into my Cargo.toml in the first place. It was a while ago. Presumably it will now fail to build elsewhere...

ZiCog · January 23, 2021, 11:54pm

So now I removed openssl from my dependencies altogether and it still builds. Now I really don't know why I had it in there in the first place. Not like I would have put it there on a whim. The reason is lost in time...

Perhaps not. I have no idea. Both "Homebrew" and "macports" are barley more than just words to me at the moment. I gather they do something akin to apt, rpm, emerge, etc.

It's a long while since I did that much. Back in the days of tinkering with gentoo and Linix From Scratch at the turn of the century.

Seems I have some homework to do.

Fun fact:

Compile times (cargo build --release, from clean) for that project on various machines around here:

MacBook Pro M1, 16GB                              (2021) -  39 seconds.
PC Intel Core i7-2600K 3.4GHz, 8GB + Samsung SSD  (2011) - 111 seconds.
MS Surface Pro 4, 8GB                             (2015) - 224 seconds.
Raspberry Pi 4, 8GB                               (2020) - 450 seconds.

Likely this M1 is the fastest machine I have ever owned

Thanks all.

cfsamson · January 24, 2021, 6:41pm

I used the “vendored” feature when using openssl crate on windows since it can be a hassle to get a crate to build across different Windows PC’s otherwise. If you primarily built this on Windows earlier that might be the reason

Btw, on Windows I had to use the openssl_probe crate as well to locate the root certificates.

ZiCog · January 24, 2021, 8:43pm

Ah yes, Windows. That would be it. As far as I recall I never did get the thing to build on Windows. I gave up as it's not a platform we need to support and it builds and runs inside the WSL anyway.

I noticed that openssl_probe is pulled in. I have no specified it. I presume that is he nats crate which I use with TLS.

Thanks.

blonk · January 24, 2021, 9:34pm

For future reference: When using open source libraries on Windows, if you want to stick to msvc, you'll probably want to use vcpkg. And if you use openssl:x64-windows-static-md you'll get a static library which uses a dynamic CRT.

vcpkg has come a long way since the first time I tried to use it with Rust crates. It's a little peculiar in some ways, but I've been happy with it for the past couple of months.