'module wide' shared function parameters, TLS, dynamic scope emulation


#1

Imagine if you could add common function parameters at the module level, almost as a means of generalizing what OOP achieves with ‘this’ (an extra parameter added to a bunch of function signatures, automatically placed into the scopes… now imagine if that was generalised so that you could nest them, i’ve often wanted that actually…).

it might look something like this,

mod foo<T>(x:Something) {  
                        // 'x' is a shared module level 'parameter' or 'variable', 
                        // -intermediate between global/local
                        // -usable by all functions declared inside..
    fn bar(y:Y) { .. blah blah x.. blah blah y..}... 
    fn baz(z:Z) { .. blah blah x blah blah z ..}.. 
} 
// .. equivalent to ..
fn foo::bar<T>(y:Y, x:Something){..}
fn foo::baz<T>(z:Z, x:Something){...}

For motivation ,see the conflict between ‘no globals / pass everything from above’ (traditionally the camp i’ve been in) versus those who resist it (… i.e. ‘why pass a parameter manually thats repeated many times’) .

e.g. I like ‘jonathan blows’ JAI language goals but he’s actually a ‘globals apologist’ who rejects rusts demand that globals are unsafe{} … whereas one of the rust-restrictions that I actually like.

Also consider the potential low level benefit of having commonly used values cached in registers;

i started to think about this after posting this, perhaps something more general like this could be a better solution to 'TLS"-esque use-cases.

also consider in addition to something like this Module wide type-parameters, thoughts

There are many details here that I don’t have a concrete suggestion for. any thoughts generally ? would this be too difficult to do… or would it merely be a nice generalisation of things we can already do, e.g. with nested functions and lambdas.

overall part of my thinking is “we lose some of what C++ classes can do, but we gain another point - the module - that we could use for control”

Think also of effects systems . My preference for ‘passing system pointers’ around is about clearly marking side effects, e.g. “rendering functions take a rendering object pointer, update functions don’t” etc etc. I’ve had conflict with others who dislike this idea (“passing it all the time is annoying…”) and admittedly, it is more verbose.

Could something like this already be achieved through procedural macros;

I guess regular macros could already wrap ‘a bunch of function declarations and stuff an extra parameter in’ fairly easily (maybe even rolling ‘dynamic-scope’-esque stacks to support setting them at various points the calligraph?) but it’s really the ‘passing of those through callers’ (i could roll versions of the functions that retreive their parameters from TLS… but i’d want those to be efficiently shared as locals instead of being retrieved each time)

i worry a bit about wrapping such large blocks of code that way aswell, e.g. the effect on error messages / giving unexpected behaviour. it also bugs me making an extra nesting level, but could a macro be applied to an entire file? my_macro!{ mod foo; } // bring in 'foo' and wrap it's entirety all in 'my_macro'

i’m not sure what calling this stuff would look like. would it have to behave exactly like an effects system, i.e. tracing the calligraph of functions that use hidden parameters, then inject passing between all of those; how would you pass in the initial value.

are there any other approaches to this sort of thing (e.g. FP currying, accumulating actions and stuffing extra parameters in later). would the module-based idea be building on existing rust structures more.

would something like this just be too unusual to consider… or would it be quite natural alongside the way existing tools like “global variables”, "TLS’ already work (and what people already see in OOP languages)

another link, I recall ‘long_void’ also made something like ‘dynamic scope’ in his ‘rust-flavoured scripting language’ https://github.com/PistonDevelopers/dyon

Part of the reason I got thinking this way is the loss of overloading, e.g. in the ‘manual passing’ idea , the ‘common system pointer’ basically becomes part of the functions name and resolving… it’s mere use distinguishes those functions from others. So again it’s more thinking along the lines of ‘if modules, traits etc are the rusty-way of thinking, can we leverage them even more?’

actually using it… there’d have to be some way of injecting the initial values and it would have to figure out how to pass the value between the initial call and consumers (foo::bar(), foo::baz() above) . e.g. lets say a() is outside foo() but traverses some big datastructure to figure out calls to foo. you’d need to say something like foo(x){/* set the parameter ‘x’ for subsequent calls to foo*/ … a() /* x actually gets passed to a(), and in turn to calls it makes to foo::bar(…), foo::baz(…) */ }

(see also https://en.wikipedia.org/wiki/Dependency_injection … i’m personally not keen on talk of ‘setting on the client’, i prefer tempraries passed when needed…)


Module wide type-parameters, thoughts
Module wide type-parameters, thoughts
Type parameters in modules
#2

You can also use the current library for this: https://github.com/pistondevelopers/current.

I’ve used it in some projects and it works pretty well.


#3

This is an interesting solution, however I think you’re going the wrong way about solving it. You’d essentially be using a static mut, but just removing the need to wrap all accesses in unsafe.

The way I see it, your problem is you want to access a lot of various bits of state and want to make it more convenient. Some people like to use global’s for this, however this is usually frowned upon for the same reasons people don’t like singletons. The typical OOP way of doing things is to pass everything in so you can do DI and make the system easier to change. However then you need to thread a bunch of stuff through your entire program, which requires more typin and is annoying.

Instead of making globals easier to use, I’d say that this is OOP trying to tell you that your architecture is too coupled. Just like people say “listen to your tests” and when things are hard to test, you should rethink your architecture, if you have loads of state which is used everywhere maybe your architecture is too coupled and needs a rethink? If you think of things from that perspective, creating the idea of module-local globals isn’t really the solution you want.

That said, it’s nice for me to say all that but it’s not overly helpful and doesn’t do much to solve the initial problem. Besides all the technical issues (global variable accessed without unsafe introducing safety issues, additional complexity, potentially unwanted features which need to be supported forever, etc), I’d try to encapsulate all that mutable state into a single object to make the typing less. It’s not great, but I feel like that’s going to be the least bad solution to this problem in the long run.


#4

don’t forget that it could be more like customised/optimized Thread-Local-Storage aswell- you could pass a different one to a thread (the ability to have a different ‘rendering-system-interface’ per thread is a big part of the motivation to ‘pass from above’)

again thats a slight misunderstanding of what i’d proposing: it’s more like a scope intermediate between global and local - I reference the ideas of TLS and dynamic scope; I do not propose a safety violation here
(Admittedly ‘system pointers’ sound like things which are used in unsafe ways, but those parts would still be wrapped in unsafe; and the use of the variable is still constrained by the same borrowing/mutabiilty rules)

well the classic case i think of already has a clear division (‘update - gameworld’ - ‘render - d3d/gl etc…).I have always liked the split of a parameter saying which one is mutable or not touched; there’s a clear split into 2 major halves’ of the application straight away

I guess what you’re saying is ‘stick with the idea of passing from above’ … but then that gets me back to the idea of overloads doing the ‘namespacing’ (rather than heirachical namespacing) - e.g. individual functions getting their names mangled with the system pointers that they use; hence why I ask if there’s a way to run more with the different tools Rust gives us.


#5

thanks, does look like it’s what I had in mind;

flicking through the implementation I do notice thread-local-storage , and key-values / ‘hash map’; (as well as a ‘dynamic scope’ idea, nesting them achieved through the guards i think?)

so basically what I’m invisaging is some serious compiler features (or hopefully procedural macros…) to achieve something similar with less runtime overhead

I dont object so much to ‘needing getters/setters’ to access the current values - it’s more the idea of overhead that triggers me. (but let me read more and see at what granularity the TLS stuff is actually used, does the library allow streamlining that for you?)

Perhaps there is a middle ground within the capability of rusts macros building on whats there

e.g.

 mod_with_current_val!{ 
    render_system=>                      // the module name
    (rs:&mut RenderingSystem)=>  // the parameter passed everywhere
    // functions ..
    fn foo(y:Y) { ...} // can use 'rs'    //e.g. render_sytem::foo(x,rs)   , etc
    fn bar(z:Z) { ...} // can use 'rs'
}

use_mod_with_current_val!{  
    my_game_renderer=>
    (rs:&mut RenderingSystem)=> 

   fn render_my_entity(e:&MyEntity) { // the macro injected 'rs' pointers..
       call!(draw_prims=>...args..) // will get 'rs' injected
       // or you could manually pass
   }

}

for the ultimate friction-reduction it would want to figure out where to inject ‘rs’ into function calls.

another take,

if it is doable as library code, and use of TLS, could it be achieved through an LLVM->LLVM transformation outside of rust - like an optimisation to ‘any nested use of tls’

… e.g. tell an LLVM based tool “here are the getters/setters for current vals - can you cache the current vals as local/passed parameters where possible…”

tool sees
fn render_system::foo(x) {
    rs=get_current_rs()
    ... use rs..
}

tool generates
fn render_system::foo_with_rs(x,rs) {
    // cut-paste of 'foo' with rs cached
    // when seeing calls to any other functions which also have cached rs versions, call those instead
}

#6

[quote=“Michael-F-Bryan, post:3, topic:11996, full:true”] I’d try to encapsulate all that mutable state into a single object to make the typing less. It’s not great, but I feel like that’s going to be the least bad solution to this problem in the long run.
[/quote]

well one nice simple thing rust does do for us is allows grouping values very easily , so that could be less painful than in c++…

 fn render_my_ent(gr:(&GameState,&mut RenderSystem), e:Entity) {
     // easy to get 'gamestate, renderstate' individually, or pass both down..
     // saved us rolling a named class for each combination as we'd have to in C++..
     // although we might want to typedef it..
 }

simple tuples are one of the things that continue to draw me to rust. (the way you could easily restructure it or pass it as one)


#7

I was thinking about function composition, but that might be too hard to do.
Performance wise I haven’t needed anything faster than the “current” library, yet.
You can always rewrite the code to use arguments afterwards.


#8

What about a custom attribute which you can put on top of a module and it’ll expand to some sort of mutable lazy static TLS thing?

#[module-local = "foo: HashMap<u32, String>"]
mod bar {
  fn use_foo() -> Option<&str> {
    foo.get(5)
  }
}

#9

could such an attribute could work at the file level aswell? I’m not entirely certain where the boundaries in capability of attributes/‘procedural macros’ are (do attributes only apply to following ‘items’?)
It’s one big drawback with putting it in the module declaration as in my original suggestion… it wouldn’t make much sense to have to ‘add parameters to the module’ in the ‘mod foo;’ statement bringing that module in elsewhere.

I suppose it’s possible the ‘current’ library is the best that is currently possible and we really would have to do some heavy work (‘effects system’ etc) to improve on it


#10

Yes, it will work:

// bar.rs
#![module_local = "foo: HashMap<u32, String>"]
fn use_foo() -> Option<&str> {
    foo.get()
}

// lib.rs
mod bar;