Implementing a unique module system

[Ethindp]

Hello all,

I haven't done a lot in the way of Rust programming, however I am running into a strange issue I can't seem to solve. I'm trying to implement a module system into an application. I'm modeling it sort of after the Linux kernel module system, though not as macro-filled as the C version is. These modules aren't Linux kernel modules but just normal dynamic libraries.

Some background: I recently came up with an interesting idea that would be both insane and awesome at the same time. The idea was a central console-like utility that would be able to manage lots of cloud services simultaneously (like AWS, Digital Ocean, VULTR, ...). At the start, at least. But it could be expanded to manage far, far more than just that; a central, unified hub where you could manage everything at once.

This would be done through modules so that users wouldn't need to modify the main applications source code at all; instead, they could write a module in any language they choose, and the application could then call on that module to do various tasks. For example, when authenticating to the module and asking the module to temporarily store the users authentication credentials, the module could have an "authenticate" function.

Or, if the user was using Digital Ocean, for example, the module could define a "list_droplets" function.

Back to the module system:

When the application loads a module, it first calls get_module_info(). This function returns a struct that looks as follows:

extern crate semver;
use semver::Version;
use std::collections::HashMap;
use std::collections::HashSet;
use std::collections::VecDeque;

/// The module_info structure contains information about the module that will be loaded.
/// All modules must define a get_module_info() function, and it must return this struct.
pub struct module_info {
    /// The name field identifies the module with a human-readable name.
    /// This field is required and must be set.
    name: &str,
    /// The required_modules field contains a list of other modules that must be loaded before this one. This is relative to the modules directory.
    /// Note that circular module dependency loops are impossible; you cannot put this module in this field or the load will immediately fail.
    /// Modules in this queue must not contain file extensions (i.e. dll, so, dylib, ...). This will be determined automatically by the module loader.
    /// Placing any of these extensions in any module requirement will cause that requirement to be skipped.
    /// This field is optional.
    required_modules: Option<VecDeque<&str>>,
    /// The license field defines the license that this module is licensed under.
    /// This must be a Software Package Data Exchange (SPDX) identifier. The list of allowed identifiers can be found at https://spdx.org/licenses.
    /// A module is considered a "free and open-source software module" (FOSSM) if and only if it is FSF and OSI approved.
    /// This field is required.
    license: &str,
    /// The author field indicates who authored this module. Multiple authors may be specified.
    /// This field is required.
    authors: &[str],
    /// The description field describes what this module does.
    /// A description can have multiple strings; these will be newlines when this description is displayed.
    /// This field is optional.
    description: &[str],
    /// The version field indicates the version of the module.
    /// The version field must follow the semantic versioning specification (https://semver.org).
    /// This field is required.
    version: Version,
    /// The extra_info field is a hash map containing keys and values of extra data that will be presented to the user during module loading.
    /// This item is optional.
    extra_info: Option<HashMap<&str, &str>>,
}

After the user has approved the module in the application, the module_init() function is called. Here's where I get stuck.

I know that the following functions should exist in the module:

• get_module_info()
• module_init()
• module_cleanup()

The module however can define any other functions it pleases. My original idea was that the application would then be able to call these functions. However, I've never defined a vtable and so am not sure on what the safest and most idiomatic way of going about this would be.

I thought of a get_module_export_table() function being defined in each module, which would returna vtable to a list of library function names, but then I ran into the idea of "How do I get the parameters and return types of these functions?"

I then thought of using RPC, but the same problem reared its head again. So, my question is: how should I go about implementing this module system in a safe and reliable way?

[prataprc]

Looks like trait objects are the way to go. Every module can have capabilities() API exported which return a list of traits it implements.

[Ethindp]

So, something like:

trait Module {
fn get_module_info()->ModuleInfo;
fn module_init()->i32;
fn module_cleanup();
}

And then add extra functions to that trait per module?
If so, how would I then make it possible to export those in such a way as to dynamically figure out what functions are available? Or am I overcomplicating the problem?
Edit: Just re-read your post and realized what you were trying to say. So, expanding on the above trate, we then have something like:

trait Module {
fn get_module_info()->ModuleInfo;
fn module_init()->i32;
fn module_cleanup();
fn capabilities()->[&str]; // maybe?
}

How would I solve the other problem of knowing what the parameters to those functions would be? Or should I implement an RPC server in the application and then the modules could call out to the RPC server and the modules can dynamically update the application?

[prataprc]

Something like following, will it work ?

// type describing its capabilities.
trait Capabilities {
    fn name(&self) -> String;
    // return the list of traits Self implements.
    fn capable_of(&self) -> Vec<String>;
    fn new_trait1(&self) -> Option<Box<dyn Trait1>>;
    fn new_trait2(&self) -> Option<Box<dyn Trait2>>;
}

trait Module {
    fn get_module_info() -> ModuleInfo;
    fn module_init() -> i32;
    fn module_cleanup();
    // return a vector of capability objects, each object shall
    // describe a type implementing one or more traits.
    fn capabilities(&self) -> Vec<Box<dyn Capabilities>>;
}

trait Trait1 {
    fn handle1(&self, a: i32) -> String;
}

trait Trait2 {
    fn handle2(&self, a: i32) -> String;
}