How to pass created variable to static handler


#1

I have a struct that should be inited once with some value and then it can be freely referenced (or even shared across threads) without problems. The problem is that I want to use it in a handler of hyper callback. Here is my sample code:

fn main() {
	let matches = App::new("Bot")
		.arg(
			Arg::with_name("token")
				.short("t")
				.long("token")
				.help("Sets the bot token to use")
				.takes_value(true)
				.required(true),
		)
		.arg(
			Arg::with_name("address")
				.short("a")
				.long("address")
				.help("Sets the address where webhook sends updates")
				.takes_value(true)
				.required(true),
		)
		.get_matches();

	let bot_token = matches.value_of("token").unwrap();
	let address = matches.value_of("address").unwrap();
	run(&bot_token, &address);
}

fn run(bot_token: &str, listening_address: &str) {
	let addr: SocketAddr = listening_address.parse().unwrap();
	let telegram_client = TelegramClient::new(bot_token.into());

	let server = Server::bind(&addr)
		.serve(move || service_fn(|x| echo(x, &telegram_client)))
		.map_err(|e| error!("server error: {}", e));

	info!("Listening on http://{}", addr);
	rt::run(server);
}

fn echo(
	req: Request<Body>,
	telegram_client: &TelegramClient,
) -> impl Future<Item = Response<Body>, Error = hyper::Error> + Send {
...

The problem here is that hyper require callback to be static:

error[E0621]: explicit lifetime required in the type of `telegram_client`
  --> src\main.rs:82:6
   |
81 |     telegram_client: &TelegramClient,
   |     --------------- consider changing the type of `telegram_client` to `&'static telegram_client::TelegramClient`
82 | ) -> impl Future<Item = Response<Body>, Error = hyper::Error> + Send {
   |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ lifetime `'static` required

If I follow this advice and change telegram_client lifetime to static I get an error that closure may outlive run function.

How should I avoid it? I probably could use Rc, but I’d like to avoid it when I’m sure there never will be any mutable reference while multiple readonly references should be resolved easily by common borrow checker.

Entire code may be found here with other relevant files in the same branch. It’s also possible to just clone the repo and see exactly what’s happening there. I’m playing with different approaches (such as Arc) so the code may differ a bit. But in case you clone it you could try your ideas on real code and see if it works.


#2

When you get 'static requirement, the compiler is trying to hint it wants owned values, and not references. You can’t make a 'static-living variable, unless you do hacky hacks involving leaking heap memory.

The borrow checker doesn’t acknowledge existence of single-threaded programs, and doesn’t treat main as special, so as far as Rust is concerned, the server you’re creating may be multithreaded, and the threads may outlive main, so borrow checker will never let you use the short-lived temporary variables from main in the server.

Use owned values (String, not &str, and possibly TelegramClient not &TelegramClient). If you need multiple references, Arc may be unavoidable.


#3

Yep, I actually wanted to create one instance and then share it across all threads, because it’s safe to share readonly state.

here is an MRE which could be easier to read

extern crate futures; // 0.1.1
extern crate hyper; // 0.12

use std::net::SocketAddr;
use hyper::service::service_fn;
use hyper::{Body, Request, Response, Server};
use futures::prelude::*;

fn main() {
    let token = std::env::args().next().unwrap();
    run(&token, "127.0.0.1:8080");
}

fn run(bot_token: &str, listening_address: &str) {
    let addr: SocketAddr = listening_address.parse().unwrap();

    let server = Server::bind(&addr)
        .serve(move || service_fn(|x| echo(x, bot_token)));
}

fn echo(
    req: Request<Body>,
    bot_token: &str
) -> impl Future<Item = Response<Body>, Error = hyper::Error> + Send {
    futures::future::ok(Response::new(Body::empty()))
}

#4

It’s not safe from Rust’s perspective, because something like this could happen:

fn main() {
   let token = …;
   run(&token);
   drop(token); // inserted by the compiler

   // another thread wakes up now and runs:
   service_fn(); // unsafe now
}

main is just like any foo function. To the borrow checker other threads are allowed to run after main() exits.

Your code is equivalent to:

fn foo() {
    let token = std::env::args().next().unwrap();
    run(&token, "127.0.0.1:8080");
}

fn run(bot_token: &str, listening_address: &str) {
    let addr: SocketAddr = listening_address.parse().unwrap();

    let server = Server::bind(&addr)
        .serve(move || service_fn(|x| echo(x, bot_token)));
}

fn echo(
    req: Request<Body>,
    bot_token: &str
) -> impl Future<Item = Response<Body>, Error = hyper::Error> + Send {
    futures::future::ok(Response::new(Body::empty()))
}

where foo() is unsafe, because its temporary token variable is passed to a thread that will outlive the call to foo().


#5

So I’m forced to use Arc or stuff like sync::Once, right?


#6

Yes. lazy_static! might work too.


#7

If your value’s safety doesn’t depend on its Drop implementation, and if you’re fine with your value living until your program terminates, there’s also the option to put it into a Box and then call Box::leak() on it.

Your value will never be dropped in that case, but in return, you get a &'static mut reference to it. This can be reborrowed to a &'static reference, which you can freely pass around.

Keep in mind, however, that this
a) requires moving your value to the heap, and
b) creates a memory leak.
It is easy to accidentally call this function in a loop and thus grow your program’s memory beyond all limits. So this function should definitely be used sparingly.


#8

Yes, I’m ok with Box::leak and I used it but it didn’t work out this issue.

All in all I’ve ended with just an Arc, I don’t see a big performance issue with it makes type system happy. However, I wonder why Box::leak didn’t fix the issue.


#9

Do you have a reproducer or some code to show?


#10

Well, here is a commit where I had it: https://github.com/Pzixel/boyan_detector_bot/blob/ed0160070bfabe42a2417383399911ff8057586b/src/main.rs#L67

It’s not an MRE and I probably should create one, but I cannot do it right now.


#11

It’s probably because you turned it into a &'static mut rather than just &'static. A mutable reference moves (or reborrows), so you can’t share it across service_fn invocations.


#12

AFAIK &mut automatically downgrades to & when possible so I don’t think it’s an issue


#13

Not if it gets moved first (or requires to be moved); you can think of it like Box itself. What was the compilation error that you had?


#14
error[E0373]: closure may outlive the current function, but it borrows `telegram_client`, which is owned by the current function
  --> src\main.rs:73:16
   |
73 |         .serve(|| service_fn(|x| echo(x, telegram_client)))
   |                ^^                        --------------- `telegram_client` is borrowed here
   |                |
   |                may outlive borrowed value `telegram_client`
help: to force the closure to take ownership of `telegram_client` (and any other referenced variables), use the `move` keyword
   |
73 |         .serve(move || service_fn(|x| echo(x, telegram_client)))
   |                ^^^^^^^

error[E0373]: closure may outlive the current function, but it borrows `telegram_client`, which is owned by the current function
  --> src\main.rs:84:53
   |
84 |     let result = req.into_body().concat2().and_then(|chunk| {
   |                                                     ^^^^^^^ may outlive borrowed value `telegram_client`
...
90 |                     telegram_client
   |                     --------------- `telegram_client` is borrowed here
help: to force the closure to take ownership of `telegram_client` (and any other referenced variables), use the `move` keyword
   |
84 |     let result = req.into_body().concat2().and_then(move |chunk| {
   |                                                     ^^^^^^^^^^^^

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0373`.
error: Could not compile `boyan_detector_bot`.

#15

This is how closures desugar. You end up with the inner closure “borrowing” the &'static reference. You can think of it like:

struct __Closure<'a> {
    telegram_client: &'a &'static TelegramClient,
}

If you add the move keyword like the error suggests, you’ll get what you’re after:

struct __Closure {
    telegram_client: &'static TelegramClient,
}

But you need to make sure you don’t move a &'static mut - either get rid of the mut or borrow it as immutable first, and then move that borrow into the closure (then you’ll have a &'static &'static mut ..., which essentially locks it to be immutable for the rest of your program, so you may as well just drop the mut to begin with).


#16

Okay, let’s say I change it to

let telegram_client = Box::new(TelegramClient::new(bot_token.into()));
let telegram_client: &'static mut _ = Box::leak(telegram_client);
let telegram_client: &'static _ = telegram_client;

let server = Server::bind(&addr)
	.serve(move || service_fn(|x| echo(x, telegram_client)))
	.map_err(|e| error!("server error: {}", e));

Then I still get an error:

error[E0597]: `*telegram_client` does not live long enough
  --> src\main.rs:74:47
   |
74 |         .serve(move || service_fn(|x| echo(x, telegram_client)))
   |                                   ---         ^^^^^^^^^^^^^^^ borrowed value does not live long enough
   |                                   |
   |                                   capture occurs here
   |
   = note: borrowed value must be valid for the static lifetime...
note: ...but borrowed value is only valid for the lifetime  as defined on the body at 74:16
  --> src\main.rs:74:16
   |
74 |         .serve(move || service_fn(|x| echo(x, telegram_client)))
   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0373]: closure may outlive the current function, but it borrows `telegram_client`, which is owned by the current function
  --> src\main.rs:85:53
   |
85 |     let result = req.into_body().concat2().and_then(|chunk| {
   |                                                     ^^^^^^^ may outlive borrowed value `telegram_client`
...
91 |                     telegram_client
   |                     --------------- `telegram_client` is borrowed here
help: to force the closure to take ownership of `telegram_client` (and any other referenced variables), use the `move` keyword
   |
85 |     let result = req.into_body().concat2().and_then(move |chunk| {
   |                                                     ^^^^^^^^^^^^

error: aborting due to 2 previous errors

Some errors occurred: E0373, E0597.
For more information about an error, try `rustc --explain E0373`.
error: Could not compile `boyan_detector_bot`.

#17

It looks like you have other code trying to access telegram_client without moving it. If you can show the rest of the code or provide a repro, it would help.

But, let me ask again - why do you have &'static mut as the type the box leaks into? Why isn’t that &'static _?


#18

It looks like you have other code trying to access telegram_client without moving it. If you can show the rest of the code or provide a repro, it would help.

This is the complete code sample, the rest is different callbacks and so on. You can clone the whole branch and play with it if you wish: https://github.com/Pzixel/boyan_detector_bot/tree/rustlang_example

But, let me ask again - why do you have &'static mut as the type the box leaks into? Why isn’t that &static _?

Because it’s the return value of Box::leak

pub fn leak<'a>(b: Box<T>) -> &'a mut T


#19

That expresses that you have unique access to it but you can assign to a shared alias, which is what you want.


#20

Okay, I take it. However, it doesn’t affect the situation in any way because if you watch carefully I trim the mut modifier on the next line.