But what if you don't have a service, but a command that just performs a job on some directory and wants to avoid concurrent access (from the same program)? Does that mean every program which performs such operations (and that would be many) should/would (have to) depend on DBUS?
To make it an example: Would you propose to make cargo require a running DBUS server?
No, I mentioned that before why it is ok for Cargo to use it since it's not much of a big deal if your cargo cache gets corrupted because you can just blow it away and start again. The cargo outputs are basically temporary files anyway. The article also mentions that case:
Or to turn this around, the only case where it is kind of safe to use file locking is in trivial applications where portability is not key and by using BSD locking on a file system where you can rely that it is local and on files inaccessible to others. Of course, that doesn't leave much, except for private files in /tmp for trivial user applications.
It is different for persistent services on a server where the file contents are important, it is a security problem if the data gets wiped out or corrupted. I mean, Cargo could also optionally use dbus or some other OS specific method if it was desired to make it robust, has anyone checked what it does on Windows to prevent concurrent access?
So IIRC those locks in Windows are mandatory so they can actually be used for security purposes. It is not the same as BSD locks which are strictly advisory. Mandatory file locking in Linux is not portable and is also basically broken and should not be used.
And I forgot to mention, Cargo also has a different case where it is ok to run multiple Cargo instances at the same time, but not on different directories. The question in OP is only about running one instance at the same time.
But if you can access the data directory and the lock file is in that directory, wouldn't it be possible to corrupt the data anyway? So I'm not sure if there is a real disadvantage here?
Disregarding that, I still see that flock makes trouble on Solaris. I (personally) could live with that (more than living with not running on all the BSDs).
I'm also not sure if the NFS issue is still existing. Note that your linked post is over 10 years old, so some info might be outdated. Since flock and the other mechanism are integrated nowadays (edit: at least on FreeBSD, see also NOTES in the FreeBSD man page), I believe that flock should work on NFS too, meanwhile. But I don't know about that.
The flock(), fcntl(2), and lockf(3) locks are compatible. Processes using different locking interfaces can cooperate over the same file safely. However, only one of such interfaces should be used within the same process.
No, with a mandatory lock you can lock the whole directory.
That quote you have refers to the FreeBSD implementation, AFAIK other OSes can have completely different behavior since flock(2) is a non-standard API. I don't know why this discussion has derailed so far into talks about BSD when the OP specifically only asked about Linux. The blog may be out of date but the BSD man pages are also completely irrelevant to the situation on Linux.
But DBUS doesn't let you make a mandatory lock either, does it?
Because I said that it is wise to program platform independent, where possible. You may consider that as derailing. I find it a reasonable note for other readers of this thread.
Anyway, if a reader of this post only cares for Linux, they may ignore all my comments and info and do whatever is done on Linux and make a platform-dependent solution.
No, dbus name ownership is also just advisory, but they do avoid any issues with locks having weird behavior between filesystems and operating systems. BSD locks are not platform independent at all either. If you really need to isolate a directory on Linux/BSD then you probably want containers/jails.
This simply cannot be done in a platform independent way, the most you can do is a make a crate that attempts to abstract over the various different ways an OS does it. But I have not seen a crate that does that anywhere.
Strangely, the blog post you linked here says:
Some OSes even find it funny to make flock() and fcntl(F_SET_LK) control the same locks. Linux treats them independently -- except for the cases where it doesn't: on Linux NFS they are transparently converted to POSIX locks, too now. What a chaos!
But then in the Linux manpage on flock, it says:
flock() does not lock files over NFS. Use fcntl(2) instead: that does work over NFS, given a sufficiently recent version of Linux and a server which supports locking.
Sounds a bit contradictory, but not sure if I misunderstand something.
Anyway, I acknowledge, that flock is not completely portable, e.g. regarding Solaris.
P.S.: I would also like to see a crate which provides platform-independent locking.
Sorry, I just realized I spoke incorrectly before. The Cargo implementation on windows also doesn't lock the directory, it just locks the same file, and AFAIK windows doesn't support mandatory directory locking. So that would also just be advisory... It probably is not easy for most filesystems to even implement mandatory directory locking either.
For this reason, I think it is even more important to use process isolation if you need security. There are a some crates for "portable" advisory file locking like fd-lock that seem to use those same methods as cargo, i.e. they should never be used for security purposes.
The manpages on linux.die.net are out of date, I really wish those would not show up in search engines anymore. The pages on man7.org tend to be better updated, see there:
Since Linux 2.6.12, NFS clients support flock() locks by emulating them as fcntl(2) byte-range locks on the entire file. This means that fcntl(2) and flock() locks do interact with one another over NFS. It also means that in order to place an exclusive lock, the file must be opened for writing.
I suppose then these lines in the cargo source are obsolete and should/could be removed?
if is_on_nfs_mount(path) {
return Ok(());
}
There are significant buggy server implementations in the wild. Still needs to be there.
Summarizing so far, there is the following crates:
pidfile-rs (BSD file locks, runs with Linux and BSD)fd-lock (file locking, aiming to be platform-independent)zbus (crate to connect to DBUS, which may be used to communicate with an already running instance of the program)There are actually a ton of other crates for advisory locks, I probably can't list them all or tell the difference easily...
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.