Callback to recursive function

Have you tried it with the closure expression directly inserted into the traverse_documents call? As I mentioned, that’s sometimes a relevant difference to help the compiler infer the right signature, especially for details like the precise lifetimes of references.

I believe, if I recall correctly, the compiler will currently assume explicitly reference-type arguments into a higher-ranked closure type, so the separate step of assigning let probe 1 = …; first might result in the lifetime as for<'a> FnMut(&'a Document, u16). These details are subtle (and quite arbitrary) [1] and can be surprising, so generally insert callback closures directly into the place where they’re used.


  1. for example if you don’t annotate the type ttt: &Document, the behavior will change, I believe ↩︎

1 Like

I haven't tried it. I can try it now, but even if it works, it would be nice to get back to this problem later, for deeper understanding.

Small demo, this confirms exactly the problem… e.g.

this doesn’t compile

fn main() {
    let mut v = vec![];
    let f = |r: &u8| {
        v.push(r);
    };
    [1, 2, 3].iter().for_each(f);
}

Screenshot_20230731_214337

whereas this does work fine

fn main() {
    let mut v = vec![];
    let f = |r| {
        v.push(r);
    };
    [1, 2, 3].iter().for_each(f);
}

the takeaway to remember is just “these inference heuristics are weird” and the best solution is to refactor into the style of directly writing the closure in the function call:

e.g. like so… (rust-analyzer is quite nice)

and now it won’t make any issues either way

// works
fn main() {
    let mut v = vec![];
    [1, 2, 3].iter().for_each(|r: &u8| {
        v.push(r);
    });
}
// works, too
fn main() {
    let mut v = vec![];
    [1, 2, 3].iter().for_each(|r| {
        v.push(r);
    });
}
1 Like

I followed your advice and made it "lambda style" (not creating the probe() closure explicitly but just pass the closure parameters and body directly as parameter to traverse_children()).

And it worked!

I also looked at your last example above. My conclusion is that by not using a named closure, the values you want to save are still alive when you need them (in assignment and push()). It makes sense.

Personally, I prefer to use functions or closures with names. It's easier to read, test and understand them. But for now I think I can live with this solution.

I have now fiddled with this issue and got a lot of helpful feedback, mainly from @steffahn. I think it's about time to wrap up what I consider an acceptable solution. In this I have stripped away lots of irrelevant information to make the files shorter, changed some names and in some cases kept some "ugly" parts.

An example input file is provided. It is still on Figma REST API JSON format, but heavily stripped. Since the root is a bit different from the other children nodes, I keep them separated.

The main.rs file

// main.rs

#![allow(dead_code)]

use std::fs;
use recur::instruct::Envelope;
use recur::instruct::Document;
extern crate serde;
extern crate serde_derive;
extern crate serde_json;

#[derive(Debug)]
struct Outstruct<'tree> {
    id: String,
    level: u16,
    node: &'tree Document,
    parent: Option<&'tree Document>,
}

fn main() {
    let infile:&str = "./ex01.json";
    let json: String;

    json = fs::read_to_string(infile).expect("Read error");

    let result = serde_json::from_str(&json);

    let res: Envelope = match result {
        Ok(file) => file,
        Err(error) => panic!("Problem reading JSON {:?}", error),
    };

    let mut outs: Vec<Outstruct> = Vec::new();

    let mut nbr_of_calls_to_probe = 0;
    traverse_children(&res.document.children, |ttt, lvl, par| {
        nbr_of_calls_to_probe += 1;
        let elem = Outstruct {
            id: ttt.id.clone(),
            level: lvl,
            node: ttt,
            parent: par,
        };
        outs.push(elem);
    });

    println!("outs[0].name: {}", (*outs[0].node).name);
    println!("outs[1].name: {}", (*outs[1].node).name);
    println!("outs[1].parent.name: {}", outs[1].parent.unwrap().name);
    println!("nbr_of_calls_to_probe: {}", nbr_of_calls_to_probe);

    dbg!(
        &outs[0].node.name,
        nbr_of_calls_to_probe,
    );
}

fn traverse_children<'tree>(tree: &'tree Vec<Document>,
    mut probe: impl FnMut(&'tree Document, u16, Option<&'tree Document>)) {

    fn recursive<'tree>(node: &'tree Document,
        probe: &mut impl FnMut(&'tree Document, u16, Option<&'tree Document>),
        level: u16,
        parent: Option<&'tree Document>) {

        probe(node, level, parent);

        if !&node.children.is_none() {
            // This might be an empty array!
            let childcount = &node.children.as_ref().expect("IGNORED").len();
            if *childcount > 0 {
                for ii in 0..*childcount {
                    recursive(&node.children.as_ref().expect("Ignored")[ii],
                        probe, level + 1, Some(node));
                }
            }
        }
    }

    for child in tree.iter() {
        recursive(child, &mut probe, 0, None);
    }
}

The lib.rs file
pub mod instruct;

The Cargo.toml file

[package]
name = "recur"
version = "0.1.0"
edition = "2021"
description = "Read a recursive JSON file."

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
serde_derive = "1.0"
http = "0.2.9"
reqwest = { version = "0.11", features = ["json"] } # reqwest with JSON parsing support
futures = "0.3" # for our async / await blocks
tokio = { version = "1.12.0", features = ["full"] } # for our async runtime

The instruct.rs file (instruct is short for INput STRUCTure)

// instruct.rs

use serde::Deserialize;
use serde::Serialize;
extern crate serde;
extern crate serde_json;

#[derive(Serialize, Deserialize)]
#[derive(Debug)]
pub struct Envelope {
    #[serde(rename = "document")]
    pub document: DocumentClass,

    pub name: String,
}

#[derive(Serialize, Deserialize)]
#[derive(Debug)]
pub struct Document {
    #[serde(rename = "children")]
    pub children: Option<Vec<Document>>,

    #[serde(rename = "id")]
    pub id: String,

    #[serde(rename = "name")]
    pub name: String,

    #[serde(rename = "type")]
    pub document_type: ChildType,
}

#[derive(Serialize, Deserialize)]
#[derive(Debug)]
pub struct DocumentClass {
    #[serde(rename = "children")]
    pub children: Vec<Document>,

    #[serde(rename = "id")]
    pub id: String,

    #[serde(rename = "name")]
    pub name: String,

    #[serde(rename = "type")]
    pub document_type: ChildType,
}

#[derive(Serialize, Deserialize)]
#[derive(Debug)]
#[derive(PartialEq)]
pub enum ChildType {
    #[serde(rename = "CANVAS")]
    Canvas,

    #[serde(rename = "DOCUMENT")]
    Document,

    #[serde(rename = "FRAME")]
    Frame,

    #[serde(rename = "INSTANCE")]
    Instance,

    #[serde(rename = "LINE")]
    Line,

    #[serde(rename = "TEXT")]
    Text,
}

An example JSON input file (named ex01.json)

{
    "document": {
        "children": [
            {
                "children": [
                    {
                        "children": [
                            {
                                "children": [],
                                "id": "381:369",
                                "name": "YellowComp",
                                "type": "INSTANCE"
                            },
                            {
                                "children": [],
                                "id": "381:373",
                                "name": "GreenInst",
                                "type": "INSTANCE"
                            },
                            {
                                "children": [
                                    {
                                        "children": [],
                                        "id": "393:15",
                                        "name": "270",
                                        "type": "INSTANCE"
                                    },
                                    {
                                        "children": [],
                                        "id": "393:9",
                                        "name": "90",
                                        "type": "INSTANCE"
                                    },
                                    {
                                        "children": [],
                                        "id": "409:5",
                                        "name": "PointY",
                                        "type": "FRAME"
                                    }
                                ],
                                "id": "381:374",
                                "name": "RedFrame",
                                "type": "FRAME"
                            },
                            {
                                "children": [
                                    {
                                        "children": [],
                                        "id": "411:8",
                                        "name": "Black",
                                        "type": "FRAME"
                                    },
                                    {
                                        "children": [],
                                        "id": "411:9",
                                        "name": "White",
                                        "type": "FRAME"
                                    }
                                ],
                                "id": "389:2",
                                "name": "Pink",
                                "type": "FRAME"
                            },
                            {
                                "id": "416:2",
                                "name": "Moltas is best!",
                                "type": "TEXT"
                            },
                            {
                                "id": "434:2",
                                "name": "txtHCentATop",
                                "type": "TEXT"
                            },
                            {
                                "id": "434:3",
                                "name": "txtHLeftATop",
                                "type": "TEXT"
                            },
                            {
                                "children": [
                                    {
                                        "id": "439:16",
                                        "name": "Multiline",
                                        "type": "TEXT"
                                    }
                                ],
                                "id": "441:17",
                                "name": "Multiframe",
                                "type": "FRAME"
                            }
                        ],
                        "id": "381:366",
                        "name": "BlueFrame",
                        "type": "FRAME"
                    },
                    {
                        "children": [
                            {
                                "id": "491:3",
                                "name": "linLine01",
                                "type": "LINE"
                            },
                            {
                                "id": "491:5",
                                "name": "LinLine02",
                                "type": "LINE"
                            }
                        ],
                        "id": "491:2",
                        "name": "pagLines",
                        "type": "FRAME"
                    }
                ],
                "id": "61:0",
                "name": "Product",
                "type": "CANVAS"
            },
            {
                "children": [
                    {
                        "children": [],
                        "id": "381:365",
                        "name": "_Null2",
                        "type": "FRAME"
                    },
                    {
                        "children": [],
                        "id": "450:2",
                        "name": "Frame 1",
                        "type": "FRAME"
                    }
                ],
                "id": "16:0",
                "name": "ColFontNil",
                "type": "CANVAS"
            },
            {
                "children": [
                    {
                        "children": [],
                        "id": "381:364",
                        "name": "_Null3",
                        "type": "FRAME"
                    }
                ],
                "id": "13:3",
                "name": "Base Components",
                "type": "CANVAS"
            },
            {
                "children": [
                    {
                        "children": [],
                        "id": "381:363",
                        "name": "_Null4",
                        "type": "FRAME"
                    }
                ],
                "id": "13:4",
                "name": "Complex Components",
                "type": "CANVAS"
            }
        ],
        "id": "0:0",
        "name": "Document",
        "type": "DOCUMENT"
    },
    "lastModified": "2023-08-03T19:07:29Z",
    "name": "Example01",
    "version": "1.0"
}

Final comments
Besides learning more Rust, I wanted to create a function that could traverse a recursive JSON structure, and have it call a callback function for each node encountered, so I could reuse the traversal function regardless of whatever the callback wanted to do.

It was also important to allow the callback to create data which could be used after the call. In this case I let it build a list, outs, of type Outstruct, which contains some data from the input. Here, node is a reference to the current node in the input, and parent is a reference to its parent node. This has to be an option, since the top node has no parent.

I only use one lifetime tag, tree, since everything is supposed to have the same lifetime. And no data needs to be mutable in this case.

To prove that it's working, I print out some results, both with the println!() macro and dbg!().

I first tried to create the callback as an ordinary function, but found that a closure was easier to handle.

The lack of proper error handling is intentional - to keep the code as short as possible and avoid losing focus.

I earlier created Figma REST API consumers in Python and C++, but since I'm so new to Rust I can't have any opinion yet.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.