File-sharing server/client idea

#1

I’m thinking of coding up a special-purpose file-sharing server/client, since I don’t know of any existing software that does what I want. Here is my idea:

The whole system consists of 1 server and any number of clients. The objective of the system is to “share” a directory among the clients. The word “share” is used in a loose sense, as only 1 client at a time is allowed to access (read or modify) the directory at a time. The clients explicitly acquire and release a lock to this directory by communicating with the server. When the server grants a client the lock, the client’s directory is synchronized to match the server’s. When the client requests to release its lock, the server’s directory is synchronized to match that client’s, before the lock is released. When a client has the lock, the directory is available to them locally. The synchronizations use a delta-transfer algorithm to efficiently transfer the changes.

Any thoughts on the idea? Anything out there that does this already, perhaps some version control system?

#2

Sounds like an interesting idea, but may I ask why do you have the restriction of only 1 client at a time?

#3

The restriction on 1 user at a time is because the files I’m dealing with are unmergeable, so if 2 people edit the files, someone’s progress will be lost.

#4

Would something like Syncthing suit your needs?

I understand you’re trying to prevent conflicts, but I’m not sure what you can do to prevent your clients from making their local copy of the directory’s content, release the lock, then later request the lock again and try to commit their changes even though the master directory might have been changed by another client in the meantime.

#5

If you’re going to implement it, I would suggest looking into UDT. There are bindings for it here: https://github.com/eminence/udt-rs.

#6

I don’t think Syncthing has the features I need, except delta transfer.

As for the problem of clients being able to subvert the system: that’s not an issue for me. The system is to be used in a setting where I trust the clients not to do that. The server could also keep a backup of the files each time it sends them off, in case something goes wrong.

#7

The UDT page mentions “UDT is designed for extremely high speed networks…”. I have 6 Mbps download, 3 Mbps upload. I don’t think UDT would help me.

Currently, I plan to use the librsync library to handle the delta transfer. I’m still looking around for a networking library, as I want something higher-level than just using TcpStream.

#8

If you’ve ever used ZMQ, there’s a protocol called FileMQ built on top of it: https://rfc.zeromq.org/spec:35/FILEMQ/ . You’d have to implement the protocol in Rust.

#9

That page is hilarious. I haven’t used ZMQ, so I won’t be going this route.

I’m leaning towards using tokio, even though it’s in its early stages.