How to Copy Symlinks

Hi, I'm developing a TCP file transfer program.

my algorithm is

learn name of the file from sender
create a file that has same name as client's file on the receiver
read bytes from sender and send it
receive bytes and write to a file on the receiver side

but problem is I can't do the same for symlinks.
Let's say if I have a.symlink (or just a as a symlink without extention).
and when I just create a. symlink on the otherside and send all data that client side has, it does not work

I just want to make symlinks like copied to usb stick then transfer to another computer.
No need to make them work in other environment or something.
Just I want to make them stay as they are like usb stick example.

here is my repo: GitHub - Tahinli/rust-tcp-file-transfer: TCP File Transfer Server and Client in Rust

I'm not quite sure I understood your problem, but you could try and use std::fs::symlink_metadata to query, whether the file at the path is a symlink and if it is, use std::fs::read_link to get the path of the real file you can then read and send to your receiver. Here a sketch:

use std::fs;
use std::path::PathBuf;

fn main() {
    let mut path = PathBuf::from("/some/file/path.txt");
    
    let meta = fs::symlink_metadata(&path).unwrap();
    
    if meta.is_symlink() {
        path = fs::read_link(path).unwrap();
    }
    
    let bytes = fs::read(path).unwrap();
    
    // here send bytes to receiver
}

Edit: I haven't checked if the functions in fs actually follow symlinks automatically. I.e. when you try fs::read("symlink"), will it read the contents of the file the symlink points to? #37204 suggests it maybe does.

Sorry for bad explanation by me.

I want copy symlink itself. If I just detect what symlink points to, I have to use platform specific functions to create new symlink at the receiver side.
std::os::unix::fs::symlink or std::os::windows::fs::symlink_file.

and I think this brakes my scenario.

I'm just building rsync like program but over network and cross platform.
I just want to make my files like copied with usb stick.

I hope I could explain this time.

How does it break your scenario exactly? If you want to support receivers on both windows and unix platforms, you can use conditional compilation to abstract over the platform-specific functions:

use std::io::Result;
use std::path::Path;

#[cfg(windows)]
fn symlink<P: AsRef<Path>, Q: AsRef<Path>>(original: P, link: Q) -> Result<()> {
    std::os::windows::fs::symlink_file(original, link)
}

#[cfg(unix)]
fn symlink<P: AsRef<Path>, Q: AsRef<Path>>(original: P, link: Q) -> Result<()> {
    std::os::unix::fs::symlink(original, link)
}

fn main() {
    symlink("a", "a.symlink").unwrap();
}
2 Likes

It breaks because, let's say one person has a folder in Linux remote server and this folder has symlinks inside. If this folder is wanted to be downloaded and transfered to another Linux local server that doesn't have network access. So folder should be transferred manually with some external drive.This will be a problem If this person using windows as a personal computer.

I know it's a specific scenario but I want to build my program like a backup tool. I don't want to change any data or it's type. They should stay same after transfer. If symlink works or not it's user's concern. My responsibility is just transferring exact file.

In what sense is a symlink on Linux created with std::os::unix::fs::symlink() not the same exact file as a symlink on Windows created with std::os::windows::fs::symlink_file() and the same arguments? If your position is that a Linux symlink can never be the same as a Windows symlink, then you've shot yourself in the foot.

EDIT: Or is your problem that you think your program should just transfer the bytes inside a file and that's all you should need to transfer a symlink? Unfortunately, that's not how it works. Your protocol is going to have to gain a way to pass the information "There is a symlink at /some/path pointing to a <directory|non-directory> at /some/other/path" between client & server, and then the receiver of the data will have to create the symlink using the appropriate OS-specific function instead of just writing bytes to a file.

3 Likes

It' might be as you said, I was wrong I think, I just know that they just have different type and my information can be wrong.

Yeah normally my algorithm is same as you wrote in edit part, and yes it's not working for symlinks.
Even zip archive breaks symlinks. But tar.gz keep symlinks still recognizable by OS. I thought I can handle this situation somehow, but I think I'm wrong. Thanks for your explanation.

I will try what @jofas mentioned about platform specific compiling. I hope it solves.

Note that creating symlinks on Windows also requires elevated privileges by default

How can that even work?

Linux have one type of symlinks: the one which may point to everything.

Windows, on the other hand, have three different types: junctions and two types of symlinks (one which points to directory and another one which points to file).

Forget Rust, try to first imagine what you plan to do with that mess on the conceptual side.

1 Like

Not if you enabled developer's mode.

TAR is Unix tool. It doesn't try to deal with three different types of symlinks on Windows. And untar programs may cheat: they may scan the whole archive and find out whether target is file or directory then call CreateSymbolicLink with appropriate flags.

Okay let's forget Rust, can you explain to me how mounting drive over network works ?

Today if you mount a folder that has symlinks or any other type of file over network with (nfs, cifs , so on).
You get your file without any problem. That is what I want to build.

No need to adapt linux symlink to windows or vice versa. I just want exact thing.

There should be a way to do this.

That's why I gave usb stick example. If I copy my linux symlink to a usb stick then transfer to windows machine, it won't work. But if I get this file to another linux, it will work.

By saying it will work or not -> I meant OS will understand that it's symlink. If the file doesn't exist where symlink point to. It won't work but it still be a symlink.

NFS is an open protocol. You can read the specification here:

https://www.rfc-editor.org/rfc/rfc7530

and take inspiration from that, maybe?

1 Like

Which protocol are we are talking about, which client, which server? All these things do matter.

Yes, but it took a lot of time to add enough kludges to make it all work. Mostly. If you are lucky.

Then you would have to study how that was handled by different tools, I guess.

Of course there's need to do that! Windows may only work with Windows symlinks and Linux may only work with Linux symlinks.

Then looking on how different NFS clients implement it would help, I guess.

Of course there are way to do this! You may emulate Windows semantic on top of Unix one (like NFS clients do). Or you may emulate Linux semantic on top of Windows one (like CIFS clients do).

It's all possible. Maybe there's even crate which implemented some of that mess (although that's not guaranteed).

What's just not exist is a simple way to handle all that!

Let me repeat one more time: there are no such thing as “simply a symlink” on Windows. There are three main types of symlinks, but there are more.

There are symlinks in the POSIX for Windows NT, there are NTFS Links, there are CygWin symlinks and there are, probably, more than I don't know about.

Copying these even just on your one, single, Windows system works when done with some apps and not when done with other apps.

So if you want to do that — prepare to conduct a lot of experiments and then incorporate lots of patches over time when people would find random warts and you would fix them.

It may or may not work, depending on many factors. NTFS-3G creates yet another type of symlink, called Interix Symlink. If you would use SFU then these would work in Windows, too (not sure if they would work in native Windows programs or just in SFU programs).

And that's not guaranteed, either. This would depend on what version of NTFS driver would you use.

Not all symlinks are interpreted at OS level. Even among these three “main” ones there are a difference: if you are looking at a remote server, junctions are processed at the server and directory symbolic links are processed at the client.

That means that when you are mounting your drive (in your question above) you see symbolic links to directory as, well, symbolic links, while junctions would look like simple folder which just happen to contain the exact same files.

The world of Windows symlinks, sure, is very exciting and vast. But it's very far from “simple”.

1 Like

I don't want it. I'm just building kind of archiving tool over network. But okay.

Sorry if I explained in a bad way. Because there is some misunderstanding situation.

I just want this scenario:

a.symlink in my computer that points to a.txt in my desktop folder.
I will transfer a.symlink to windows machine with my program.
Then take this symlink from windows and copy to usb stick.
Then copy same symlink from usb stick to my computer again.
This symlink will work again in my computer.

I don't want to make windows to understand my file not even usb stick.
I don't want to check if symlink points to valid file or existed one.

10 minutes ago I archived my symlink in linux with 7z method. Sent my friend who has windows.
He extracted 7z with Winrar. This symlink didn't work in Windows and that is what I wanted.
Then he just archived this symlink with rar format and sent to me again.
This symlink was still pointing to my file and it was working.

This is what I want, I've been trying to implement that if I can do the same without using archiving methods.

Sorry for being stubborn, I'm just trying to understand even if it seems like I can't.

Nobody tells you that you are stupid, don't worry. It's just situation with symlinks on Windows is horrible, awful, mess and thus it's hard to understand what you actually want without “zooming out” far enough.

That's nice description of the scenario, but note that use of Winrar is critical here. If, instead of using Winrar, you would use CygWin's version of 7z to unpack the archive and then would use Winrar to archive the result then after unpacking that archive your symlink would only work in CygWin but wouldn't work in native Windows apps!

Except if you would use winsymlinks option, in which case it would work. But only if you use to unpack original 7z archive with it, if you would use it to unpack Winrar-created archive, then it's too late, now you have to unpack with Winrar and then archive with CygWin's versions of 7z!

Similarly here:

In that scenario Internix symlink or Cygwin symlink would survive, but native NTFS symlink would be lost!

Different scenario, different tools, different outcomes. The sad truth is that all types of Windows symlinks may be destroyed depending on which apps and which options you are using to transfer them (but some other Windows symlinks would survive these transformations)!

I understand that. You want some kind of “simple soluton”, but that one just simply doesn't exist.

From the description it sounds like you have already discovered the best approach:

You need to decide whether you want to only support symlinks-to-file or want to support symlinks-to-directory.

On Linux there are no difference. On WIndows these are distinct. This approach would would support “official Windows symlinks” introduced in Windows Vista and would ignore all other types.

That's not perfect solution (it doesn't support junctions and Internix or Cygwin symlinks). But that's what 7z and Winrar are doing and thus, probably, enough for your needs.

There are no common method which would handle both Unix and Windows, but you can easily implement it in your program in some helper module: just make two functions which would call symlink on Windows and symlink_dir or symlink_file on Windows and that should be enough for your needs.

The fact that there are just one symlink (which may point to file or directory) in Unix while there two “official” symlinks on Windows (and half-dozen “old” kinds) is just something you have to deal with (in any language).

I think for your needs supporting two “official” symlinks on Windows and ignoring everything else is enough. Even if “official” symlinks don't always work (in particular they wouldn't survive trip over FAT-formatted USB stick).

2 Likes

I feel the misconception here is that, a symlink is not a regular file contains some magic data that specific OS remembers, it's a special file that can't be emulated with regular file. You just can't treat it like a regular file when copying it to a foreign system.

5 Likes

I understand, thank you all.

I will mark this as a solution.

Note that this is not true for Internix and Cygwin symlinks. These are special files with magic content and attributes hidden and system. That's why they survive trip via FAT USB stick.

Sometimes you can. But then you need an app which would interpret these and symlinks.

Most Windows app don't do that, most CygWin apps do.

The problem here lies with the fact that “native” Windows symlinks arrived in Windows Vista, that's year 2007, while Windows NT existed since 1993.

That meant that in these 14 years people invented many kludgy ways to represent symlinks and these are still with us, to some extent.

1 Like