Wunschkonzert, Ponyhof und Abenteuerspielplatz

I forgot to mention another central problem in my blog story about file locking on Linux:

Different machines have access to different features of the same file system. Here's an example: let's say you have two machines in your home LAN. You want them to share their $HOME directory, so that you (or your family) can use either machine and have access to all your (or their) data. So you export /home on one machine via NFS and mount it from the other machine.

So far so good. But what happens to file locking now? Programs on the first machine see a fully-featured ext3 or ext4 file system, where all kinds of locking works (even though the API might suck as mentioned in the earlier blog story). But what about the other machine? If you set up lockd properly then POSIX locking will work on both. If you didn't one machine can use POSIX locking properly, the other cannot. And it gets even worse: as mentioned recent NFS implementations on Linux transparently convert client-side BSD locking into POSIX locking on the server side. Now, if the same application uses BSD locking on both the client and the server side from two instances they will end up with two orthogonal locks and although both sides think they have properly acquired a lock (and they actually did) they will overwrite each other's data, because those two locks are independent. (And one wonders why the NFS developers implemented this brokenness nonetheless...).

This basically means that locking cannot be used unless it is verified that everyone accessing a file system can make use of the same file system feature set. If you use file locking on a file system you should do so only if you are sufficiently sure that nobody using a broken or weird NFS implementation might want to access and lock those files as well. And practically that is impossible. Even if fpathconf() was improved so that it could inform the caller whether it can successfully apply a file lock to a file, this would still not give any hint if the same is true for everybody else accessing the file. But that is essential when speaking of advisory (i.e. cooperative) file locking.

And no, this isn't easy to fix. So again, the recommendation: forget about file locking on Linux, it's nothing more than a useless toy.

Also read Jeremy Allison's (Samba) take on POSIX file locking. It's an interesting read.

posted at: 19:49 | path: /projects | permanent link to this entry | 4 comments

Posted by Erick at Tue Jun 29 07:06:59 2010
I used to admin a network consisting of Linux servers and a combination of Linux, Irix and Windows clients. The Linux and Windows boxes accessed smb shares while the Irix clients accessed NFS exports. The file locking was an impossible to manage nightmare. It was a constant problem and was insane to deal with. It got to the point that people wanted to abandon Linux on the server and client, use all Windows and keep the Irix boxes only because the engineering software couldn't be relicensed for Windows clients affordably.

Once you get systemd done, which I commend you - that's a killer piece of software! - then apply your mad skills to fixing locking everywhere. Please. :-)

Posted by Erick at Tue Jun 29 07:10:19 2010
LOL - I didn't see you posted Jeremy's article on locking. Good stuff! You two should put your brains together and fix it. Please. :-)

BTW- The only reason ppl wanted to get rid of Linux, as mentioned in my previous post, was because of the file locking issues. That was the only reason. That's sad.

Posted by Adam Klobukowski at Tue Jun 29 08:11:36 2010
And now you're going to break our file locking? ;) Maybe time for systemlockd?

Posted by Gaetano at Fri Jul 9 10:45:09 2010
Annecdotical evidence: file locking over nfs is very much in use by web server farms, where a lot of frontends share the same data.
As developers of eZPpublish, we started out using flock() calls to manage rw access to shared files, but in the end just gave up (and that's counting we at least avoid the pitfall described in this post: one process having access to the same files mounted as local filesystem instead of nfs!) and switched to a mixed configuration: nfs+database. File metadata are kept in a table in the db, actual file data in the share.
A lot more code to maintain, and perfs suffer a bit - but i hear other document biggish management systems went the same route...

Leave a Comment:

It should be obvious but in case it isn't: the opinions reflected here are my own. They are not the views of my employer, or Ronald McDonald, or anyone else.

Please note that I take the liberty to delete any comments posted here that I deem inappropriate, off-topic, or insulting. And I excercise this liberty quite agressively. So yes, if you comment here, I might censor you. If you don't want to be censored your are welcome to comment on your own blog instead.

レナート Wunschkonzert, Ponyhof und Abenteuerspielplatz ﻟﻴﻨﺎﺭﺕ

Mon, 28 Jun 2010

Addendum on the Brokenness of File Locking