Hello,
I'm confusing to dealing with memory leak in rust.
I'd like to write an async function that periodically update the state with the latest values from data store.
The code looks like belows
use async_trait::async_trait;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::RwLock;
use tokio::time;
struct MyState {
value: i32,
}
type Result<T> = std::result::Result<T, std::io::Error>;
#[async_trait]
trait MyStore {
async fn get_latest_state(&self, id: u32) -> Result<MyState>;
}
struct MyStoreImpl {}
#[async_trait]
impl MyStore for MyStoreImpl {
async fn get_latest_state(&self, _id: u32) -> Result<MyState> {
Ok(MyState { value: 10 })
}
}
async fn update_state(
id: u32,
state: Arc<RwLock<HashMap<u32, MyState>>>,
store: Arc<dyn MyStore + Send + Sync>,
) -> Result<()> {
let mut interval = time::interval(Duration::from_millis(100));
loop {
interval.tick().await;
let new_state = store.get_latest_state(id).await?;
if let Some(state) = state.write().await.get_mut(&id) {
*state = new_state;
}
}
}
#[tokio::main]
async fn main() {
let app = Arc::new(MyStoreImpl {});
let state = Arc::new(RwLock::new(HashMap::new()));
update_state(1, state, app).await;
}
[package]
name = "leak"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
async-trait = "0.1.68"
tokio = { version = "1.28.2", features = ["full"] }
However, when I run this codes in kubernetes, the memory usage grows and eventually OOMKilled.
I searched some tool, and used valgrind. but there was no definitely lost, but some possibly lost on HashMap.
Here are some results. I currently don't have the full result in text right now.
12,688,316 bytes in 15,168 blocks are possibly lost in loss record 224 of 224.
malloc
...
hashbrown::raw::alloc::inner::do_alloc (alloc.rs.95)
...
hashbrown::rustc_entry::<impl hashbrown::map::HashMap...::rustc_entry
std::collections::hash::map::HashMap<K,V,S>::entry (map.rs:855)
After looking at the results, I'm still have no idea how to resolve it.
When I modified the way to update entry stored in hashmap, the memory usage does not grow as before and it seem that the memory leak was resolved.
// before
if let Some(state) = state.write().await.get_mut(&id) {
*state = new_state;
}
// after
state
.write()
.await
.get_mut(&id)
.and_then(|s| Some(*s = new_state));
I don't understand why these codes results in different memory footprints.