Rustube getting stuck and then deleting downloaded file

I am just learning streams and async rust so this might be a stupid mistake in my code. So I am trying to use rustube to download a youtube video , as a fail safe if the asset (video) is not there. However..

  1. When I use video.download(...) on the main thread the progress gets stuck at 100%. and I use progress_bar.finish_with_message("Finished downloading asset!") to tell that the video is finished downloading but it gets stuck at 100% but doesn't show message even after an hour.
  2. When I use the same code on a new thread using tokio::spawn and tokio::join! ,progress bar goes till 100% and displays message "Finished downloading asset!". But after that it deletes the .webm that it has created. This leaves with no file downloaded. And no , I don't delete the file after download.

This is my Cargo.toml

[dependencies]
#....

# For progress bar
indicatif = "0.17.6"

# For downloading neccessary youtube videos
rustube = { version = "0.6.0" , features = ["callback"]}

And my main.rs

#[tokio::main]
async fn main() {
    assets::download_background_videos().await;
}

Where my asset.rs is

use tokio::{
    spawn,
    try_join
};

use indicatif::{ProgressBar,ProgressStyle};

use rustube::{
    Video,
    Id,
    Callback,
};

use crate::AssetsError;

const VIDEOS_DIR : &str = "assets/videos";

pub(in crate) async fn download_background_videos() -> Result<(),AssetsError> {
     // just prints checking assets and if assets is already present

     // Using main thread (1.)
     check_download_vid_with_id("Pt5_GSKIWQM").await?;
    // And the one using new thread (2.)
     let mc = spawn(check_download_vid_with_id("Pt5_GSKIWQM"));

     try_join!(mc)

    // prints Finished checking for assets!
    Ok(())
}

// Function that rly downloads stuff
async fn check_download_vid_with_id(id : &str) -> Result<(),AssetsError> {
    let _dir = format!("{VIDEOS_DIR}/{id}.mp4");
    let path = Path::new(&_dir);
        
    match path.exists()  {
        true => Ok(()),
        false => {
            let _id = Id::from_str(id).unwrap();
            let _video = Video::from_id(_id.into_owned()).await.unwrap();
            
            match _video.best_video() {
                None => Err(AssetsError::VideoStreamUnavailable),
                Some(stream) => {
                    let _content_length = stream.content_length().await.unwrap();
                    let progress_bar = ProgressBar::new(_content_length);
                        
                    progress_bar.set_style(ProgressStyle::with_template("[{elapsed_precise}] {bar:40.cyan/blue} {percent}% {msg}").unwrap());

                   let callback = {
                        let _progress_bar = progress_bar.clone();

                        Callback::new()
                            .connect_on_progress_closure_slow(move |arg| {
                                _progress_bar.inc(arg.current_chunk as u64);
                                _progress_bar.set_length(_progress_bar.length().unwrap()/*(arg.content_length.unwrap() - _progress_bar.length().unwrap()).abs()*/);
                            }) 
                            .connect_on_complete_closure(move |_| progress_bar.finish_with_message("Finished downloading asset!") )
                    };

                    stream.download_to_dir_with_callback(VIDEOS_DIR,callback).await.map(|_| Ok(()))?
                }
            }
        }
    }
}

AssetsError is just a custom error type I have made using thiserror crate and it is as follows :

use thiserror::Error;

#[derive(Error,Debug)]
pub(in crate) enum AssetsError {
    #[error("Failed to create asset directory : {}",.0)]
    CreatingDirectory(#[from] std::io::Error),

    #[error("Failed to find video stream , maybe check for internet connection")]
    VideoStreamUnavailable,

    #[error("Failed to download video due to , {}",.0)]
    DownloadError(#[from] rustube::Error),
}

So I still do not get why it doesn't downloads the file and gets stuck in the first place? Any help is appriciated

I see that rustube uses the log crate. Maybe if you hook up a logger and set the logging level to debug you'll get some useful info.

I have never used a logger in rust so how do i set it up as i have added
env_logger = "0.10.0" to my Cargo.toml
and added env_logger::init(); to my main function and have tried running command
cargo run RUST_LOG=debug but that doesn't show any logs and also have tried settings in Cargo.toml

[env]
RUST_LOG="debug"

I have used env_logger add this code

env_logger::init();
    log::debug!("this is a debug {}", "message");
    log:: error!("this is printed by default");

    if log::log_enabled!(log::Level::Info) {
        let x = 3 * 4; // expensive computation
        log::info!("the answer was: {}", x);
    }

and have ran

  • cargo run RUST_LOG=main
  • cargo run RUST_LOG=debug
  • cargo run RUST_LOG=info
  • cargo run RUST_LOG=warn
  • cargo run RUST_LOG=error
    I have even tried set RUST_LOG=main (I am on Windows 10 btw)
    But in all only the error log shows

You need to set environment variable, not to pass it as parameter. In Windows Terminal, this is set RUST_LOG=whatever (don't forget to set it to empty string afterwards though).

1 Like

As I have written above

Setting that also for some reason doesn't show anything besides errors logs.

Also after examining the rustube crate , in the internal_download function I noticed that it only deletes the file if some error as occurred , what error it does not really tell.

Why do you use RUST_LOG=main though? You don't want to log information in the main module, you want to log information provided by the dependency, so it's more logical to set the level (i.e. debug or info) for all modules at once.

1 Like

I did this as in env_logger it stattes

And enable all logging:

$ RUST_LOG=main ./main
[2017-11-09T02:12:24Z DEBUG main] this is a debug message
[2017-11-09T02:12:24Z ERROR main] this is printed by default
[2017-11-09T02:12:24Z INFO main] the answer was: 12

This looks like a misformulation on the crate's side - "all logging" here seems to mean "all logging for module main".

So I type set RUST_LOG=rustube::debug or set RUST_LOG=debug

Well, do you want to see all logs from the module rustube::debug? That's what the first syntax says. Or do you want to see all logs up to debug level from all modules? That's the second one.

Well I run the flag and all I got was

[....] ... download_to : assets/videos/VwJaIa_Eyds.mp4
[...] ... start downloading VwJaIa_Eyds
[...] ... try to download VwJaIa_Eyds using sequenced download // repeated a couple of times
[...] ... failed to download : VwJaIa_Eyds 

I wish I could be more helpful but thats all that I got

You may have to file an issue with Rustube.

It wouldn't surprise me if Rustube can't keep up with Google's blocks against this.

1 Like

Ok thanks , till that is solved I will just manually download the assets.

It might be better to run yt-dlp or youtube-dl as a subprocess. I know that at least yt-dlp can keep up with Google's blocking.

Ok thanks the info , I'll try it soon as possible and get back to this thread