Tokio strange issue

1

Hi all,

I have playing with tokio and few libraries and got the strange issue ...

Cargo.toml

[package]
name = "SomeApp"
version = "0.1.0"
authors = ["strange <strange@gmail.com>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "0.2", features = ["full"] }
reqwest = { version = "0.10", features = ["json"] }
scraper = "0.11.0"
url = "2.1.1"
futures = "0.3.4"

... and main.rs

#![feature(try_trait)]

use tokio::prelude::*;
use scraper::{Html, Selector};
use reqwest::Url;
use url::ParseError;
use futures::future::{BoxFuture, FutureExt};
use futures::Future;
use std::panic;

fn handle_url(url: Url) -> BoxFuture<'static, Result<(), Box<dyn std::error::Error + Send + Sync>>> {
    async move {
        // let url = Url::parse(url).unwrap();
        println!("Url path is {}", url);
        let resp = reqwest::get(url.clone()).await?.text().await?;
        println!("resp is {:#?}", resp);
        let selector = Selector::parse("a").unwrap();
        let document = Html::parse_document(&resp);
        handle_url(Url::parse("https://docs.rs/tokio/").unwrap()).await;
        Ok(())
    }.boxed()
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let url = Url::parse("https://docs.rs/tokio/0.2.19/tokio/signal/index.html").unwrap();
    tokio::spawn(handle_url(url)).await;
    Ok(())
}

I got some strange issue:

error: future cannot be sent between threads safely
  --> src\main.rs:40:7
   |
40 |     }.boxed()
   |       ^^^^^ future created by async block is not `Send`
   |
   = help: within `tendril::tendril::NonAtomic`, the trait `std::marker::Sync` is not implemented for `std::cell::Cell<u
size>`
note: future is not `Send` as this value is used across an await
  --> src\main.rs:19:9
   |
18 |         let document = Html::parse_document(&resp);
   |             -------- has type `scraper::html::Html` which is not `Send`
19 |         handle_url(Url::parse("asdsafas").unwrap()).await;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ await occurs here, with `document` maybe used later
...
40 |     }.boxed()
   |     - `document` is later dropped here

error: aborting due to previous error; 4 warnings emitted

Could somebody explain it ?
I cannot get how document can be dropped before it will be awaited handle_url(Url::parse("tokio - Rust").unwrap()).await; ...

Does it mean that at line handle_url(Url::parse("tokio - Rust").unwrap()).await; task could be switched to different thread ?!

2

Yes, tokio::spawn may run the spawned task on multiple threads. You can use LocalSet and spawn_local to run a task on a single thread, which will remove the Send requirement.

2 Likes
3

I have got the issue ...

It is pretty simple !!

Html, Selector};
use reqwest::Url;
use url::ParseError;
use futures::future::{BoxFuture, FutureExt};
use futures::Future;
use std::panic;

fn handle_url(url: Url) -> BoxFuture<'static, Result<(), Box<dyn std::error::Error + Send + Sync>>> {
    async move {
        // let url = Url::parse(url).unwrap();
        println!("Url path is {}", url);
        let resp = reqwest::get(url.clone()).await?.text().await?;
        println!("resp is {:#?}", resp);
        let selector = Selector::parse("a").unwrap();
        {
            let document = Html::parse_document(&resp);
            // Do some processing
        }
        handle_url(Url::parse("https://docs.rs/tokio/").unwrap()).await;
        Ok(())
    }.boxed()
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let url = Url::parse("https://docs.rs/tokio/0.2.19/tokio/signal/index.html").unwrap();
    tokio::spawn(handle_url(url)).await;
    Ok(())
}

Document object is not thread-safe and at the line handle_url(Url::parse("tokio - Rust").unwrap()).await; could be created task that will be run on another thread and it requires all types that are used to be Send and Sync

4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.