Here is my program:
#[global_allocator]
static GLOBAL_ALLOC: pstd::localalloc::Perm = pstd::localalloc::Perm;
fn main() {
use ironpress::html_to_pdf;
let _pdf = html_to_pdf("<h1>Hello</h1><p>World</p>").unwrap();
// std::fs::write("output.pdf", pdf).unwrap();
}
Output ( on cargo run ):
malloc(): corrupted top size
Aborted (core dumped)
Perm is my global allocator, quite likely it is a problem with that, although I am using it quite extensively in other programs without a problem, so maybe the problem is elsewhere. I guess I need to somehow interpret the dumped core, but I don't even know where it is.
Try running with valgrind and without your custom allocator. If that passes, the issue is probably with your allocator. If it fails the issue is with the program.
Without the custom allocator, output is:
georgebarwood@penguin:~/projects/ironbug$ cargo valgrind run
Compiling ironbug v0.1.0 (/home/georgebarwood/projects/ironbug)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.89s
Running `/home/georgebarwood/.cargo/bin/cargo-valgrind target/debug/ironbug`
Error leaked 486.2 KiB in 14 blocks
Info stack trace (user code at the bottom)
at malloc (vg_replace_malloc.c:381)
at alloc (library/alloc/src/alloc.rs:95)
at alloc::alloc::Global::alloc_impl_runtime (library/alloc/src/alloc.rs:190)
at alloc_impl (library/alloc/src/alloc.rs:312)
at <alloc::alloc::Global as core::alloc::Allocator>::allocate (library/alloc/src/alloc.rs:429)
at do_alloc<alloc::alloc::Global> (src/raw/alloc.rs:19)
at hashbrown::raw::RawTableInner::new_uninitialized (src/raw/mod.rs:1613)
at hashbrown::raw::RawTableInner::fallible_with_capacity (src/raw/mod.rs:1672)
at prepare_resize<alloc::alloc::Global> (src/raw/mod.rs:2738)
at resize_inner<alloc::alloc::Global> (src/raw/mod.rs:2936)
at reserve_rehash_inner<alloc::alloc::Global> (src/raw/mod.rs:2824)
at hashbrown::raw::RawTable<T,A>::reserve_rehash (src/raw/mod.rs:1013)
at hashbrown::raw::RawTable<T,A>::reserve (src/raw/mod.rs:962)
at hashbrown::raw::RawTable<T,A>::find_or_find_insert_index (src/raw/mod.rs:1167)
at find_or_find_insert_index<u32, u16, std::hash::random::RandomState, alloc::alloc::Global, u32> (src/map.rs:1863)
at hashbrown::map::HashMap<K,V,S,A>::insert (src/map.rs:1843)
at std::collections::hash::map::HashMap<K,V,S,A>::insert (library/std/src/collections/hash/map.rs:1296)
at ironpress::parser::ttf::parse_cmap_format4 (ttf.rs:475)
at ironpress::parser::ttf::parse_cmap (ttf.rs:375)
Summary Leaked 486.2 KiB total (0 other errors)
georgebarwood@penguin:~/projects/ironbug$
With the allocator, I have
georgebarwood@penguin:~/projects/ironbug$ cargo valgrind run
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.28s
Running `/home/georgebarwood/.cargo/bin/cargo-valgrind target/debug/ironbug`
error: invalid valgrind usage: --7678-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV) - exiting
--7678-- si_code=128; Faulting address: 0x0; sp: 0x1003da6e20
valgrind: the 'impossible' happened:
Killed by fatal signal
host stacktrace:
==7678== at 0x5804C869: get_bszB_as_is (m_mallocfree.c:301)
==7678== by 0x5804C869: get_bszB (m_mallocfree.c:313)
==7678== by 0x5804C869: vgPlain_arena_malloc (m_mallocfree.c:1809)
==7678== by 0x58004FE4: vgMemCheck_new_block (mc_malloc_wrappers.c:370)
==7678== by 0x580051CA: vgMemCheck_malloc (mc_malloc_wrappers.c:405)
==7678== by 0x5809A08B: do_client_request (scheduler.c:1956)
==7678== by 0x5809A08B: vgPlain_scheduler (scheduler.c:1519)
==7678== by 0x580E1B6C: thread_wrapper (syswrap-linux.c:101)
==7678== by 0x580E1B6C: run_a_thread_NORETURN (syswrap-linux.c:154)
sched status:
running_tid=1
Thread 1: status = VgTs_Runnable (lwpid 7678)
client stack range: [0x1FFEFEF000 0x1FFF000FFF] client SP: 0x1FFEFFA970
valgrind stack range: [0x1003CA7000 0x1003DA6FFF] top usage: 18792 of 1048576
Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.
If that doesn't help, please report this bug to: www.valgrind.org
In the bug report, send all the above text, the valgrind
version, and what OS and version you are using. Thanks.
I am wondering if maybe my allocator crashes with programs that leak memory. Well, that was a rather random thought, but I could try a simple program that leaks. Probably not it though.
I also tried miri while I was waiting for valgrind to install, and got this, which I guess might be a clue:
georgebarwood@penguin:~/projects/ironbug$ MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-permissive-provenance" cargo +nightly miri run
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.12s
Running `/home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo-miri runner target/miri/x86_64-unknown-linux-gnu/debug/ironbug`
error: unsupported operation: Miri does not support file-backed memory mappings
--> /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/memmap2-0.9.10/src/unix.rs:99:23
|
99 | let ptr = mmap(
| _______________________^
100 | | ptr::null_mut(),
101 | | map_len as libc::size_t,
102 | | prot,
... |
105 | | aligned_offset as off_t,
106 | | );
| |_____________^ unsupported operation occurred here
|
= help: this is likely not a bug in the program; it indicates that the program performed an operation that Miri does not support
= note: stack backtrace:
0: memmap2::os::MmapInner::new
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/memmap2-0.9.10/src/unix.rs:99:23: 106:14
1: memmap2::os::MmapInner::map
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/memmap2-0.9.10/src/unix.rs:246:9: 252:10
2: memmap2::MmapOptions::map::<&std::fs::File>
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/memmap2-0.9.10/src/lib.rs:431:9: 437:10
3: fontdb::Database::load_font_file_impl
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/fontdb-0.23.0/src/lib.rs:269:37: 269:75
4: fontdb::Database::load_font_file::<&std::path::PathBuf>
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/fontdb-0.23.0/src/lib.rs:262:9: 262:48
5: fontdb::Database::load_fonts_dir_impl
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/fontdb-0.23.0/src/lib.rs:377:41: 377:67
6: fontdb::Database::load_fonts_dir_impl
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/fontdb-0.23.0/src/lib.rs:384:17: 384:54
7: fontdb::Database::load_fonts_dir_impl
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/fontdb-0.23.0/src/lib.rs:384:17: 384:54
8: fontdb::Database::load_fontconfig
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/fontdb-0.23.0/src/lib.rs:575:13: 575:55
9: fontdb::Database::load_system_fonts
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/fontdb-0.23.0/src/lib.rs:465:21: 465:43
10: ironpress::system_fonts::system_fontdb::{closure#0}
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ironpress-1.4.3/src/system_fonts.rs:410:9: 410:31
11: std::sync::OnceLock::<fontdb::Database>::get_or_init::<{closure@ironpress::system_fonts::system_fontdb::{closure#0}}>::{closure#0}
at /home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sync/once_lock.rs:321:50: 321:53
12: std::sync::OnceLock::<fontdb::Database>::initialize::<{closure@std::sync::OnceLock<fontdb::Database>::get_or_init<{closure@ironpress::system_fonts::system_fontdb::{closure#0}}>::{closure#0}}, !>::{closure#0}
at /home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sync/once_lock.rs:543:19: 543:22
13: std::sync::Once::call_once_force::<{closure@std::sync::OnceLock<fontdb::Database>::initialize<{closure@std::sync::OnceLock<fontdb::Database>::get_or_init<{closure@ironpress::system_fonts::system_fontdb::{closure#0}}>::{closure#0}}, !>::{closure#0}}>::{closure#0}
at /home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sync/once.rs:226:40: 226:60
14: std::sync::Once::call_once_force::<{closure@std::sync::OnceLock<fontdb::Database>::initialize<{closure@std::sync::OnceLock<fontdb::Database>::get_or_init<{closure@ironpress::system_fonts::system_fontdb::{closure#0}}>::{closure#0}}, !>::{closure#0}}>
at /home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sync/once.rs:226:9: 226:61
15: std::sync::OnceLock::<fontdb::Database>::initialize::<{closure@std::sync::OnceLock<fontdb::Database>::get_or_init<{closure@ironpress::system_fonts::system_fontdb::{closure#0}}>::{closure#0}}, !>
at /home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sync/once_lock.rs:542:9: 555:11
16: std::sync::OnceLock::<fontdb::Database>::get_or_try_init::<{closure@std::sync::OnceLock<fontdb::Database>::get_or_init<{closure@ironpress::system_fonts::system_fontdb::{closure#0}}>::{closure#0}}, !>
at /home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sync/once_lock.rs:410:9: 410:27
17: std::sync::OnceLock::<fontdb::Database>::get_or_init::<{closure@ironpress::system_fonts::system_fontdb::{closure#0}}>
at /home/georgebarwood/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sync/once_lock.rs:321:15: 321:55
18: ironpress::system_fonts::system_fontdb
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ironpress-1.4.3/src/system_fonts.rs:408:5: 412:7
19: ironpress::system_fonts::load_system_default_fonts
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ironpress-1.4.3/src/system_fonts.rs:440:14: 440:29
20: ironpress::HtmlConverter::convert_to_writer::<std::vec::Vec<u8>>
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ironpress-1.4.3/src/lib.rs:485:9: 485:67
21: ironpress::HtmlConverter::convert
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ironpress-1.4.3/src/lib.rs:364:9: 364:47
22: ironpress::html_to_pdf
at /home/georgebarwood/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ironpress-1.4.3/src/lib.rs:120:5: 120:39
23: main
at src/main.rs:7:16: 7:57
note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace
error: aborting due to 1 previous error
georgebarwood@penguin:~/projects/ironbug$
Could it be my allocator is somehow not compatible with memory mapped files?
This message is saying that Miri doesn't work with mmap, which is used by the ironpress crate that you're using. This is a limitation of Miri which has to be specifically coded to work with each system call, and it only supports a subset of them. This is not a bug in your code.
Oh, I think I just figured out what it probably is.
My allocator "redirects" requests it cannot handle (based on size/alignment) to the System allocator, but there is a problem with how I have done this for shrink, if the new allocation can be handled I need to free the System allocation and handle the new allocation, rather than using the System shrink. Otherwise when it is later freed it will be freed from the wrong allocator.
I will fix that and see if it then works ok.
[ Edit, yup, that did the trick, everything is now fine. Total logic error in my grow/shrink code, I never really thought it through properly when I did it, it was sort-of experimental but I never got round to thinking about it carefully. ]