Seeking API design ideas for my software 3D renderer

[parasyte]

I've been writing a software 3D renderer for a few months. The API is designed after OpenGL ES2.0, for better or worse. It has traits for vertex shader and fragment shader. The entire fixed function pipeline works well, and performance is excellent for what it is. The trouble I ran into is when I tried making the type system generic to allow executing the fixed function pipeline with arbitrary shader implementations (which are different types, not known at compile time).

I wrote about the design on my blog; relevant code snippets are included.

It appeared simple enough; global state like a vector of shader programs, render target, depth buffer, viewport matrix, etc. The shader programs are different types, e.g. one that does Gouraud lighting, and another that does water simulation. So I tried a vector of trait objects. But the shader program trait has associated types. AFAICT, the associated types are concrete types, so I wasn't able to populate the vector dynamically.

The complexity was too high with the trait object design, so I scrapped it. I would love to keep the API simple. I really just want to share code and state within the fixed function pipeline across many shader programs. It's easy enough to create multiple instances of my base struct, but then I can't easily share internet state (render target, depth buffer, viewport matrix, ...) between them.

Here's some pseudo code to show the kind of API I wanted:

let SAR = 16.0 / 9.0; // Scene Aspect Ratio

let diffuse_texture = Texture::new( ... );

let uniform = gouraud_shader::Uniform::new(
    cgmath::perspective(cgmath::Deg(60.0_f32), SAR, 1.0, 100.0),
    cgmath::Vector3::new(-1.0, 1.0, 1.0).normalize(),
    diffuse_texture,
);

let vertex_shader = gouraud_shader::Vertex::new();
let shader_program = gouraud_shader::ShaderProgram::new(uniform, vertex_shader);
let mut gl = SoftwareGL::new(shader_program);

loop {
    // Input handling ...

    gl.draw_arrays();

    // Copy the render target to the physical display ...
    // Wait for V-Sync ...
}

This works great. But now I want to swap shader_program at runtime with a method like gl.use_program(program_index). I got it working trivially when shader_program was a new instance of the same type. But using a different type like water_shader::ShaderProgram::new( ... ) has so far tripped me up.

The second attempt I tried was rebinding my gl variable with different instances of SoftwareGL. This only works if I recreate all of the internal state every time I rebind the variable.

The third attempt was to create a separate variable for two different SoftwareGL instances. I was able to move state between them with a method that moves the struct fields, and then some fancy juggling with variable bindings. It ends up looking like this:

// Create gl1
let mut gouraud_shader_program = ...
let mut gl1 = SoftwareGL::new(gouraud_shader_program);

// Swap shaders :(
let (pixels, zbuffer, viewport, shader_tmp) = gl1.move_state();
gouraud_shader_program = shader_tmp; // Save shader program

// Create gl2
let mut water_shader_program = ...
let mut gl2 = SoftwareGL::new_from(pixels, zbuffer, viewport, water_shader_program);

loop {
    // Swap shader programs
    let (pixels, zbuffer, viewport, shader_tmp) = gl2.move_state();
    water_shader_program = shader_tmp; // Save shader program
    gl1 = SoftwareGL::new_from(pixels, zbuffer, viewport, gouraud_shader_program);

    gl1.draw_arrays();

    // Swap shader programs
    let (pixels, zbuffer, viewport, shader_tmp) = gl1.move_state();
    gouraud_shader_program = shader_tmp; // Save shader program
    gl2 = SoftwareGL::new_from(pixels, zbuffer, viewport, water_shader_program);

    gl2.draw_arrays();
}

This syntax is in need of some love. Any ideas?

BTW, I'm trying to keep up periodic updates on my blog for anyone interested. And why not, here's some #screenshotsaturday tax:

[vitalyd]

It would seem that the shader program should be an arg to draw_arrays rather than a field of SoftwareGL? Alternatively, add a function swap_shader_program that takes a shader as an arg, sets it internally, and returns the previous shader program.

Is SoftwareGL yours or from a different crate?

[parasyte]

Thanks for the response.

The SoftwareGL struct is mine. And SoftwareGL::draw_arrays method passes &self.program as an arg to the rasterizer (fixed function pipeline), so it's basically setup that way already. Just with a little syntax sugar.

The swap_shader_program method you suggested is what I tried first. It won't work because the internal type would be different. e.g.:

struct SoftwareGL<T> {
    pixels: [u8, SIZE],
    zbuffer: [u8, SIZE],
    viewport: cgmath::Matrix4<f32>,
    program: T,
}

impl<T> ShaderProgram<T> {
    pub fn new(program: T) -> Self {
        let pixels: [u8; SIZE] = [u8; 0];
        let zbuffer: [u8; SIZE] = [u8; 0];
        let viewport = cgmath::Matrix4::identity();

        Self { pixels, zbuffer, viewport, program }
    }

    pub fn swap_shader_program<T>(&mut self, program: T) {
        self.program = program;
    }
}

fn main() {
    let shader_program1 = ...
    let gl = SoftwareGL::new(shader_program1);

    let shader_program2 = ...
    gl.swap_shader_program(shader_program2); // Error: Wrong type for T
}

I could unwrap the draw_arrays functionality, though. Call the rasterizer directly from main. Then I need to keep the rest of the internal state in main, too. I was trying to use the SoftwareGL struct for its encapsulation of internal state details. If that makes sense.

[parasyte]

OH! You're absolutely right, passing shader_program to draw_arrays() is the right way to do this. I was still stuck on object-oriented methodologies, wanting the shader to be part of the SoftwareGL state.

Thanks @vitalyd, that's exactly what I needed.

[vitalyd]

Ok cool. Yeah, the arg approach is the more straightforward one. I mentioned swapping the shader as an option in case you had multiple functions you wanted to call where you'd need the shader - passing it as an arg to each may be an ergonomic hit in that case. If that's not the case, then all good.

If you did, however, want to swap the shader then you can still do that via trait objects; you couldn't do it via concrete shader types because those are different types, as you said, but trait objects would work.

[parasyte]

I tried using trait objects, but the compiler complained about the associated types. I have multiple traits; one for each struct, for example the vertex shader and fragment shader. I turned ProgramShader into a trait, too. Each of these traits has to be linked together with associated types. But those types prevent trait objects from working as expected. There are several open issues related to combinations of generics, traits, and associated types. I might have hit one of those bugs.

I'll revisit that problem iif it's really necessary. There are only two methods that need a shader arg as of now (draw_arrays and draw_elements). The ergonomics are acceptable.