Understanding how to add functionnality

Hi,

I am working on a document generator using Typst.
I use the typst_as_lib - Rust

I want to create a generic document generator so I was thinking to use mutiple Hasmap to init data before generation.

But it seems that Hashmap is not implemented for a trait.

I was thinking to use some vec in place... that I can init from Hashmap.
But maybe, I also can extends the missing functionnalities. As it appears to use a Macro, I have some problem to undestand how to take in account HashMap data ?

Can you help me a little on the way please ?

I make a example that works (without hashmap). Then I try to make a more generic structure like that :

use derive_typst_intoval::{IntoDict, IntoValue};
use std::{collections::HashMap, fs};
use typst::foundations::{Bytes, Dict, IntoValue};
use typst_as_lib::TypstEngine;

//static FONT: &[u8] = include_bytes!("./fonts/texgyrecursor-regular.ttf");

#[derive(Debug, Clone)]
pub enum TemplateFile {
    Document1 ,
}

impl TemplateFile {
    pub fn as_filename(&self) -> &'static str {
        match self {
            TemplateFile::Document1 => include_str!("./template.typ"),
        }
    }
}

#[derive(Debug, Clone, IntoValue, IntoDict)]
pub struct GenerateDocument {
    data_keys: Vec<String>,
    data_values: Vec<String>,

    //datas: HashMap<String, String>,  // PROBLEM HERE...
}

impl GenerateDocument {
    pub fn new() -> Self {
        Self {
            data_keys: Vec::new(),
            data_values: Vec::new(),
        }
    }

    pub fn init_values(&mut self, values: HashMap<String, String>) {
        for (key, value) in values {
            self.data_keys.push(key);
            self.data_values.push(value);
        }
    }

    pub fn generate_document(&self, template: TemplateFile, output_filename: &str) {
        let template = TypstEngine::builder()
            .main_file(template.as_filename())
            // .fonts([FONT])
            .build();

        // Run it
        let doc = template
            .compile_with_input(self.clone())
            .output
            .expect("typst::compile() returned an error!");

        // Create pdf
        let options = Default::default();
        let pdf = typst_pdf::pdf(&doc, &options).expect("Could not generate pdf.");
        fs::write(output_filename, pdf).expect("Could not write pdf.");
    }
}

impl From<GenerateDocument> for Dict {
    fn from(value: GenerateDocument) -> Self {
        value.into_dict()
    }
}

The problem is about the last impl with Dict...

In fact, I use another way to make something generic. I think my approach was to complex. And I only use a function to generate document that take in parameter a structure in which I define my data.

By the way, I wonder if I can expend possible data with hashmap value...

I'm having difficulty understanding your post. But maybe you mean this:

    pub fn add_hashmap_values(&mut self, values: HashMap<String, String>) {
        for (key, value) in values.into_iter() {
            self.data_keys.push(key);
            self.data_values.push(value);
        }
    }

Or if you wanted to do it generically, here is one way:

    pub fn add_values(&mut self, values: impl IntoIterator<Item = (String, String)>) {
        for (key, value) in values.into_iter() {
            self.data_keys.push(key);
            self.data_values.push(value);
        }
    }

This works for passing a Hash<String, String> or Vec<(String, String)>.

1 Like

Thank you for your time.
In fact, I realise that it is possible to pass some values from the rust code to the template.
But the type of these values is limited to String, f64, Bytes... But not directly HashMap... I was thinking HashMap can be helpful with a generic way where we match a key with a type of data...

But I used specific structure finaly...

So I use another way in which you create a struture of data with this kind of type :

pub fn generate_document<T>(
    template: TemplateFile,
    content: T,
    output_filename: &str,
) -> Result<String, DocumentGeneratorError>
where
    T: From<T> + Into<Dict> 
{
    let template = TypstEngine::builder()
        .main_file(template.as_filename())
        .fonts([FONT]) // default font mandotory or nothing is written
        .build();

    // Run it
    match template.compile_with_input(content).output {
        Err(e) => return Err(DocumentGeneratorError::ErrorCompileTypst(e)),
        Ok(doc) => {
            // Create pdf
            let pathname = Path::new(GENERATED_FILE_DIR).join(output_filename);
            let options = Default::default();
            match typst_pdf::pdf(&doc, &options) {
                Err(e) => return Err(DocumentGeneratorError::ErrorGeneratePdf(e)),
                Ok(pdf) => match fs::write(pathname.clone(), pdf) {
                    Err(e) => Err(DocumentGeneratorError::ErrorWritePdf(e)),
                    Ok(_) => Ok(String::from(pathname.to_str().unwrap())),
                },
            }
        }
    }
}

...

 #[derive(Debug, Clone, IntoValue, IntoDict)]
    struct PocContent {
        one_thing: String, // here I can pass what I want with a authorized type now...
    }

    impl From<PocContent> for Dict {
        fn from(value: PocContent) -> Self {
            value.into_dict()
        }
    }

By the way, I discover other problem using this lib because it requires 2024 edition but my project use an very old version of rust 1.77.2.

I have to look how it is possible to make some crate coexists now... ?

As far as I know, you are actually allowed to have dependencies whose Rust Editions are different from the one of your crate

You definitely are so allowed, or everyone would have to move to editions in lock step, defeating their main purpose (avoiding an ecosystem-splitting Rust 2.0).

(In fact thanks to macros, the edition of Rust code can change in a single source file.)

4 Likes

(this is not really relevant to your question, but)

you can avoid this nested matching by using the question mark operator ? like this:

let doc = template
    .compile_with_input(content)
    .output
    .map_err(|e| DocumentGeneratorError::ErrorCompileTypst(e))?;

// Create pdf
let pathname = Path::new(GENERATED_FILE_DIR).join(output_filename);
let options = Default::default();
let pdf =
    typst_pdf::pdf(&doc, &options).map_err(|e| DocumentGeneratorError::ErrorGeneratePdf(e))?;

let _ =
    fs::write(pathname.clone(), pdf).map_err(|e| DocumentGeneratorError::ErrorWritePdf(e))?;

Ok(String::from(pathname.to_str().unwrap()))

See this section of the Rust book for more information: Recoverable Errors with Result - The Rust Programming Language

1 Like

Are you trying to implement From<GenerateDocument> for Dict, and if so, why? Is there some generic function that takes a generic parameter which must implement the Into<Dict> trait (if you implement From<GenerateDocument> for Dict, the compiler generates Into<Dict> for GenerateDocument for you, which seems to be what you want)?

Also I think the #[derive(IntoDict)] isn't doing what you want it to do. It takes a struct, which consists of pairs of "field name" + "field value", and turns that into a dictionary, which contains mappings from "field name" to "field value".

So in the case of GenerateDocument, it creates a Dict with a key "data_keys" mapping to a Vec<String>, and a key "data_values" mapping to a Vec<String>. And replacing that with HashMap<String, String> would create a Dict with just one key ("datas"), and the entire HashMap as the corresponding value.

But what you probably want is to have each "data_key"+"data_value" as a separate mapping? In that case, I don't really know how you would achieve that, because I'm not familiar with the library..

Oh Thank you, this is more elegant.

I just remove Err for type compatibities. It is ok.

1 Like

It was exactly what I wanted in the begining " a HashMap<String, String> that would create a Dict with just one key ("datas"), and the entire HashMap as the corresponding value."

In a generic way, I was thinking to have only one value as a HashMap.
And in the HashMap, I can add key/data as I want.
Like this :

values: HashMap<String, String>,
vecs: HashMap<String, Vec<String>>,

etc...

But HashMap not work at now in the lib.

To make the implementation needed more generic is also rust compilation poetry because is a trait... I follow recommandation of compilation but it goes on all direction. I have to undestand these case :

impl From<DefaultContent> for Dict {
    fn from(value: DefaultContent) -> Self {
        value.into_dict()
    }
}

// No compile...

impl From<dyn DefaultContent> for Dict {
    fn from(value: impl DefaultContent) -> Self {
        value.into_dict()
    }
}

// No Compile etc...

For my problem of compatibilities. I need to read more documentation.

Today I have a project that use 1.77.2 rust version.
A lot of package like this :

pack_1
---Cargo.toml
pack_2
---Cargo.toml
...
document_generator // my new crate
---Cargo.toml
---rust-toolchain.toml  // I use this file with the channel = "1.86.0"
Cargo.toml // use all the pack

All the Cargo.toml use edition 2018 except my new crate that need 2024.

rust compilation poetry is like :

error: failed to load manifest for workspace member `D:\Projets_E2E\e2e-bo\document_generator`

Caused by:
  failed to parse manifest at `...\document_generator\Cargo.toml`

Caused by:
  feature `edition2024` is required

  The package requires the Cargo feature called `edition2024`, but that feature is not stabilized in this version of Cargo (1.77.2 (e52e36006 2024-03-26)).
  Consider trying a newer version of Cargo (this may require the nightly release).
  See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#edition-2024 for more information about the status of this feature.

So I will need to update the version of the project. But as a lot of thinks must be changed, I was thinking to do it crate by crate (pack by pack) using the possibilities to have this rust-toolchain.toml file.
But it seems that the edition is also a problem that I need to undestand.

In theory, upgrading your rustc/cargo version shouldn't break anything-- The newest version of the compiler will still interpret your code according to the 2018 rules, but it also understands the 2024 edition so that it can compile and link in the newer crates to your existing project.

I rustup update now...

I have my main project that uses a lot of package.

pack_1
---Cargo.toml
pack_2
---Cargo.toml
...
document_generator // my new crate
---Cargo.toml
---rust-toolchain.toml  // I use this file with the channel = "1.86.0"
Cargo.toml // use all the pack

It uses 1.77.2 version of rust to compile and all the package are in 2018 edition.

I add my package document_generator with a unit test in.
I use, to test only my document_generator code this command : cargo test --all generate
When I am in 1.86.0 rust version, it works.
I can use 2018, 2021, 2014 edition in document_generator/Cargo.toml. It works.
So I keep 2018 for now to be like the rest of the main project :

[package]
name = "document_generator"
version = "0.0.1" 
edition = "2018"

[dependencies]
derive_typst_intoval = "0.3.0"
typst = "0.13.1"
typst-as-lib = "0.14.4"
typst-pdf = "0.13.1"
thiserror = "2.0.12"

Then to launch all the project, I create a document_generator/rust-toolchain.toml like that :

[toolchain]
channel = "1.86.0"

But now, compilation fail like if 2024 is required (it was working without just before...) :

.../main_project > rustc -V                 
rustc 1.77.2 (25ef9e3d8 2024-04-09)
.../main_project > cargo test --all generate
error: failed to download `typst-as-lib v0.14.4`

Caused by:
  unable to get packages from source

Caused by:
  failed to download replaced source registry `crates-io`

Caused by:
  failed to parse manifest at `...\.cargo\registry\src\index.crates.io-6f17d22bba15001f\typst-as-lib-0.14.4\Cargo.toml`    

Caused by:
  feature `edition2024` is required

  The package requires the Cargo feature called `edition2024`, but that feature is not stabilized in this version of Cargo (1.77.2 (e52e36006 2024-03-26)).
  Consider trying a newer version of Cargo (this may require the nightly release).
  See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#edition-2024 for more information about the status of this feature.

So it is like, rust version are incompatible or I am using rust-toolchain.toml file in a bad way ?

After reading the documentation Introduction - The Rust Edition Guide, I undestand that edition is about "changing the language and requirement" that can affect the all code. But here, my problem seems to be a problem of version that at the end require a 2024 edition ? So I am a kind of lost...

It looks like you are in a directory main_project here, and it isn't clear how this relates to the document_generator directory where the rust-toolchain.toml file is. Please could you show use the full directory structure here and the location where you are running cargo test? The rust-toolchain.toml only applies when you are running the commands under the directory where it is.

Also, can you explain why you need version 1.77.2? If you are able to update the complier (and you seem to have got 1.86.0 working at one point), then there isn't really a reason to have an older version set globally. If you need an old version for a specific project for some reason, then you can have a rust-toolchain.toml just for that project and run rustup update to update the toolchain for everything else.

main_project // I am in this directory when I launch my command cargo test... cargo run... I replace it by '...' in the previous post.
--pack_1
----Cargo.toml
--pack_2
----Cargo.toml
...
--document_generator // my new crate
----Cargo.toml
----rust-toolchain.toml  // I use this file with the channel = "1.86.0"-
--Cargo.toml // use all the pack

This main_project is a big one. I will have to upgrade edition in a second time and use latest stable version of rust in a first time I think. The problem is that I would like to do this "pack" by "pack" because it is huge.

If I compile with 1.86.0 version. Compilation succeed but when running, I have a segmentation fault !!!! That I don't have in 1.77.2.
So I think I have to look more precisely to the change that occurs in new version of the crate used in each pack...

I don't think you can have different crates compiled with different toolchain versions, but you can have different crates compiled with different editions. Because document_generator is a subdirectory rather than a parent directory of main_project, its toolchain doesn't apply when in main_project. The editions shouldn't matter that much and can be updated when you want to migrate to the features of the new edition, but the version should be consistent throughout the project (and ideally as recent as possible).

The main problem here seems to be the segmentation fault. Such issues should only occur if there is incorrect unsafe code, so I would suggest checking any unsafe in your project, running under miri to find exactly where the issue is, and possibly updating dependencies (in case the issue is in an old version of a dependency that has since been fixed).

1 Like

Thank you. I will look that.

So the problem here is as follows:

  1. you try calling value.into_dict() here, which would require value, which is of type DefaultContent, to have a method called into_dict
  1. that method would be created by the #[derive(IntoDict)] derive macro (don't know why the struct was suddenly renamed from DefaultContent to GenerateDocument, but let's ignore that for now):
  1. ... but that macro requires all the field types to implement IntoDict themselves, which HashMap does not

Having looked at IntoDict and Dict now, I don't really understand why it isn't implemented for HashMap -- it would've been very logical. Unfortunately, you can't add the implementation yourself (in your crate), because neither the HashMap type nor the IntoDict trait is defined in your crate (this is called "orphan rule": Traits: Defining Shared Behavior - The Rust Programming Language.

So I would recommend opening an issue under GitHub - typst/typst: A new markup-based typesetting system that is powerful and easy to learn. to ask the authors to either add that implementation, or explain why adding it wouldn't make sense

1 Like

Thank you for your time. I think I will do that now that I have found a issue to my problem of version, etc...

Finally, I try to change my rust version to the minimum required with my crate (1.82.0) (using 2018 edition for now). And it seems to work. I will upgrade in future with more time.

Finally, During this post :

  • I learn what is edition.
  • I realise we can use only one rust-toolchain.toml file for the main Cargo.toml in the project.
  • I learn best tips with map_err and generic.
  • I understand better the crate.

Thank you very much to you all !