Default_lib_allocator info?


#1

Hi all!

Currently working on updating Rust for some of my size-optimized stuff, and I’ve got a setup that looks something like this:

main.rs:

#![no_std]
#![no_main]
#![feature(lang_items, link_args, alloc)]

mod rust_boilerplate {
    #[lang = "eh_personality"] extern fn eh_personality() {}
    #[lang = "panic_fmt"] extern fn panic_fmt() -> ! { loop {} }
}

#[allow(unused_attributes)]
#[link_args = "/NODEFAULTLIB /SUBSYSTEM:CONSOLE /SAFESEH:NO /DYNAMICBASE:NO /LTCG"]
extern {}

extern crate alloc;

use alloc::boxed::Box;

#[allow(non_snake_case)]
#[no_mangle]
pub unsafe extern fn mainCRTStartup() -> i32 {
    let derr = Box::new(5);
    *derr
}

cargo.toml:

[package]
name = "no-std-test"
version = "0.1.0"
authors = ["yupferris <jake@fusetools.com>"]

[profile.release]
opt-level = 3
debug = false
rpath = false
lto = true
debug-assertions = false
codegen-units = 1
panic = "abort"

[dependencies]

When building this, however, we get the following output:

$ cargo run --release -v
   Compiling no-std-test v0.1.0 (file:~/dev/projects/no-std-test)
     Running `rustc --crate-name no_std_test 'src\main.rs' --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C lto -C metadata=b1fc9671c7652cf9 -C extra-filename=-b1fc9671c7652cf9 --out-dir '~\dev\projects\no-std-test\target\release\deps' -L 'dependency=~\dev\projects\no-std-test\target\release\deps'`
error: no #[default_lib_allocator] found but one is required; is libstd not linked?

I’ve never seen default_lib_allocator before, and it seems that it hasn’t been documented yet. Further, the latest details in the rust book I’ve been able to find about custom allocators doesn’t seem to be accurate or work with the latest nightly. Where might I be able to find more information on what’s required to fix this issue? Note that I’m fine with reading source and actually quite happy if I can fix this by providing my own allocator in some way, as we have our own code for that used in some other places as well, but I need some pointers where to start.

For some additional context, this is for some 64kb intros in the demoscene (like http://www.pouet.net/prod.php?which=68375 and http://www.pouet.net/prod.php?which=69658). The stuff we’re doing for these intros is somewhat low-level and can depend on low-level boostrapping/linkage details that I’ve observed can change somewhat often between Rust versions. This is understandable, and until now, we’ve avoided most issues by simply using a specific nightly version of Rust (specifically nightly-2016-10-03-i686-pc-windows-msvc, or 1.14.0-nightly 144af3e97 2016-10-02). However, I think we’re a bit overdue for an update, so I started taking a look. Last time I tried this we had some other issues that iirc were related to Options doing stack unwinding even with the abort panic strategy or something, but I don’t know if that’s still going to be a problem or not until I get a bit further here, hehe… :slight_smile:


#2

See this and this for help on using the system allocator instead of jemalloc on the latest nightly Rust. You are correct it is not documented.

The FAQ also has a section dealing with binary size.

Finally, on the latest Rust nightly you can put in your Cargo.toml to optimize for size:

[profile.release]
opt-level = 's'

#3

Thanks for the links! Definitely pointed me in the right direction.

Just so I get the details here straight…

The original error (no #[default_lib_allocator] found but one is required) means that we need something that satisfies the default_lib_allocator API so that libraries we’re linking (such as alloc in this case) are actually able to alloc/free memory. I wasn’t able to find anything marked with this attribute with some searching in the rust repo, but I was able to find this module with the name __default_lib_allocator in libstd/heap.rs that appears to implement something very similar to the allocator API, from liballoc/allocator.rs. I guess somehow this module is understood to be the default allocator for libs when libstd is linked (probably due to the individual symbol names, as some similar imports in alloc/heap.rs appear to represent the same API, but the names are slightly different so the mechanism is a bit unclear to me), but since we don’t use std, this __default_lib_allocator module is not available.

It looks like if we have an instance of an object that implements the allocator API marked with the global_allocator attribute, the compiler will know to redirect allocator calls to this global instance (likely by linking imported allocator symbols in eg. alloc/heap.rs against the global allocator instance instead of the __default_lib_allocator module). An example where this is done is the smallest-hello-world test, where liballoc_system is imported and an instance of its System allocator which implements the allocator API is marked as global_allocator. However, this could be any object instance that implements this API.

Trying all of this locally I’m now getting the following errors:

          no_std_test-d658219492340857.0.o : error LNK2001: unresolved external symbol _GetProcessHeap@0
          no_std_test-d658219492340857.0.o : error LNK2001: unresolved external symbol _HeapAlloc@12
          no_std_test-d658219492340857.0.o : error LNK2001: unresolved external symbol _HeapFree@12
          no_std_test-d658219492340857.0.o : error LNK2001: unresolved external symbol _GetStdHandle@4
          no_std_test-d658219492340857.0.o : error LNK2001: unresolved external symbol _WriteFile@20
          no_std_test-d658219492340857.0.o : error LNK2001: unresolved external symbol _rust_begin_unwind
          C:\msys64\home\ferris\dev\projects\no-std-test\target\release\deps\no_std_test-d658219492340857.exe : fatal error LNK1120: 6 unresolved externals

…which is great, as I haven’t explicitly linked kernel32 yet. However, I’m also seeing _rust_begin_unwind looming there already, we’ll see if that causes any issues. :slight_smile: EDIT: it does, and it appears to be the same issue described here: https://github.com/rust-lang/rust/issues/38281. I’ll continue looking into that separately.

Thanks again!