Web scraper won't run? unresolved import

use scraper::Html;
use scraper::Selector;
use reqwest::blocking;

fn main() {
    let response = blocking::get(
        "https://examplewebsite.com",
    )
    .unwrap()
    .text()
    .unwrap();

    let document = Html::parse_document(&response);

    let title_selector = Selector::parse("div.content>div").unwrap();

    let titles = document.select(&title_selector).map(|x| x.inner_html());

    titles
        .zip(1..25)
        .for_each(|(item, number)| println!("{}. {}", number, item));
}

So, I go into my src folder with my main.rs file and then in the terminal I type cargo check, and no errors, then i do cargo build and no errors. But, when I do rustc main.rs to make it into a program so I can launch it with ./main.rs it gives me the 3 errors pertaining to my scraper and reqwest cargos. I added them to my cargo.toml file, but I still get the error. The error says "unresolved import 'scraper'", also the same with 'reqwest' too. Then it has a red error question saying ^^^^^^^ maybe a missing crate 'scraper'? and then the help says "help: consider adding 'extern crate scraper' to use the 'scraper' crate. I don't knonw how to solve this and am still a beginner in programming. Right now I am trying to make a web scraper so I can use it for my manga program. Any help is appreciated.

Use cargo build and cargo build --release to compile debug and optimized binaries (respectively), instead of trying to use rustc directly. The results of a successful compilation will be in a subdirectory under the target directory.

1 Like

It was successful and did fix the debug and unoptimisation, but I went into the target folder and found debug, release, CACHEDIR.TAG, and .rustc_info.json. I went into the release folder to try and launch it since I saw an execultable, but it gave me this error when I launch it from the terminal:

./Project_Name

thread 'main' panicked at src/main.rs:9:6:
called Result::unwrap() on an Err value: reqwest::Error { kind: Builder, source: RelativeUrlWithoutBase }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

The unwrap method's job is to panic when the expression returns an error. What you're seeing above is such a panic. The error is shown above. Look at the source line it mentions. The error message tells you what is wrong. Then look at the doc for the method you're calling before the unwrap. It should tell you why this error is returned.

2 Likes

It looks like .get only accepts absolute URLs meaning I just forgot the https:// at the front of the website. Their is no more errors, and now I just need to figure out how to print the html inside my terminal. When I do ./Project_Name inside the release folder nothing happens and no errors occur. It just sends me back to the release folder. Thanks so much for everyone's help!

1 Like

It's possible that titles is empty. Try inserting a dbg!(&titles) after the line let titles = ...

That made it print. Thanks so much! I really like how rust can point me to a problem and makes it so much easier to troubleshoot and fix. Cargo is a godsend. It printed the HTML like I wanted, so I think everything is wrapped up. Thanks again!