When read/write from file. occurs problems

hi.
i wrote a (WAL) for internal persistent data to file.

format: [ Id ][ object_size ] | [object_value]

and i used bincode for convert rust struct to bytes and that
use something like newline and whitespace that create problems.

and some of number in id like number 13 make like this problem and
I just convert integer to bytes (to_be_bytes or to_le_bytes)

and after start again loader, cannot fetch from file
because changed format.

my wal file:

(data)
�������09370156893�������

��������������

�������

�������������$$$$$$$$$$$$

(data)
�������09370156893�����������������������������������������$$$$$$$$$$$$

(data)
�������09370156893�����������������������������������������$$$$$$$$$$$$

pub fn put(&mut self, mut object: Vec<u8>) -> Result<(), Status>{

            let obj_len = object.len();
            let object_fixed_size = self.object_fixed_size as usize;
            
            // check size of new object size must be equal or lower than (object_fixed_size)
            if object_fixed_size >= obj_len {
                
                let diff = object_fixed_size - obj_len;
                
                // if object size is lower than (object_fixed_size) then add padding
                if diff > 0 {
                    // need padding to write to be fixed size
                    object.append(&mut vec![36; diff]);    
                }
                
                // 10u8 == \n (newline)
                object.push(10);


                match self.file.write_all(&mut object) {
                    Ok(_) => Ok(()),
                    Err(e) => Err(Status::Error(e.to_string()))
                }

            } else {
                Err(Status::ErrorObjectIsBig)
            }
        }


        // line actually is first offset of next line 
        
        pub fn get(&mut self, mut line: i64) -> Result<GetResult, Status> {
            if line <= 9 {
                line = 9;
            }

            // set seek to line
            let _ = self.file.seek(SeekFrom::Start(line as u64));
            
            // read object
            let mut buf_obj = vec![0; self.object_fixed_size as usize];
            let num_readed = self.file.read(&mut buf_obj).unwrap();

            if num_readed == 0 {
                return Err(Status::Eof);
            }
            
            // +1 is for skip from newline charector. 
            let next_line = line + self.object_fixed_size + 1;
            

            Ok(GetResult {
                // default == $/36 is padding
                object: decode1(buf_obj, 36),
                line: next_line 
            })

        }


 fn decode1(bf_obj: Vec<u8>, delimiter: u8) -> Vec<u8> {
        let res = bf_obj.iter().enumerate().find(|(_, e)| e == &&delimiter);
        match res {
            None => bf_obj,
            Some((index,_)) => {
                bf_obj[0..index].to_vec()
            }
        }
    }

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.