Imitation of the user on the web site

How to do to connect to the site, click on the link on the site and get some information (on a new page) by writing it to a variable?

The big question is whether you want to support JavaScript (i.e. links created by JS or links that don't do anything without JS).

  • If you need JS, then you need a full web browser and remote control of the browser via WebDriver or similar mechanism. There's selenium-rs, but WebDriver isn't Rust specific and you'll find more mature libraries for this in Node.js land.

  • If you don't need JS, then you can download HTML with a crate like reqwest, parse the HTML with html5ever and extract the link yourself.

1 Like