Corrupt FTP Transfers

1

Hello All,

I am currently working on a program to recursively step through the directories of an FTP server, which then clones the chosen directory to the local computer.

The program runs fine and identifies all files and folders with the correct structure etc. However my issue is that when some files are transferred, they are corrupted e.g. jpeg files with only the top third of the image intact and the rest all jumbled garbage (it looks to me like an incomplete transfer or an issue with writing of the contents to disk). I have copied the files from the server using Filezilla and they are fully intact with no corruption.

I am using the "ftp-client" crate from crates.io and it seems to do everything I need. Below is the portion of the code which retrieves the file and writes it to my local PC. I am wondering if I either need to confirm that the file is fully received from the ftp server before writing to the disk or do I need to somehow give the program enough time to write the contents locally to disk before moving on to the next file? Are there any buffers at play here in the background that need to be flushed out before the program moves to the next file?

Appreciate if anyone has any tips on file transfers or on what may be causing the corruption.
Thanks,
djme.

            //retrieve the file from the current working directory
            let remote_file_path = String::from("./") + &components[8];     //e.g. "./file.text"
            let remote_file = client.retrieve_file(&remote_file_path).unwrap();

            //write the file to disk
            let mut local_file = File::create(&local_file_path).unwrap();
            local_file.write_all(&remote_file).unwrap();
2

I am not sure, but this might be a bug in the ftp-client crate. I had a very quick look at this crate and it does not seem to set the TYPE flag. This would mean that binary data such as jpegs are being downloaded as text and some of the bytes are being corrupted.

My search:

More information on the flag:
https://cr.yp.to/ftp/type.html#type

3

Hi newbie101,
Thanks for your reply.
Your answer seems to make sense based off what I am seeing.

Is there a way I can set the type in my program to transfer all files as binary (I think it should be ok to transfer all files, including text files as binary files)? In the meantime I might have a look on crates.io to see if there are any other ftp crates that work better for me.

4

I made a PR to add the TYPE commands to switch between ascii and binary:
https://github.com/aaneto/ftp-client/pull/4/files

If you would like, you can use the branch in Cargo with:

[dependencies]
ftp-client = { git = "https://github.com/alienscience/ftp-client", branch = "transfer-type" }

And then add the following command to your code before downloading anything:

client.binary().unwrap();
5

Wow! Thank you newbie101!

I followed your instructions and the download a jpeg file as a test. The file was downloaded perfectly and is fully intact. Your knowledge, speed of reply and helpfulness is very impressive.

I am very grateful to you for taking the time to root cause and fix this issue. I'll see if I can add an issue to the original git repository to have the issue fixed there ('ll reference your helpful answer here).

Thanks again,
djme.

6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.