1        Mandatory File Locking For The Linux Operating System
   3                Andy Walker <>
   5                           15 April 1996
   6                     (Updated September 2007)
   80. Why you should avoid mandatory locking
  11The Linux implementation is prey to a number of difficult-to-fix race
  12conditions which in practice make it not dependable:
  18          lock was held.
  21          read has actually completed, and the reading process may see
  23          to it.
  24        - Similar races make the claimed mutual exclusion between lock
  25          and mmap similarly unreliable.
  271. What is  mandatory locking?
  30Mandatory locking is kernel enforced file locking, as opposed to the more usual
  31cooperative file locking used to guarantee sequential access to files among
  32processes. File locks are applied using the flock() and fcntl() system calls
  34normally a process' responsibility to check for locks on a file it wishes to
  35update, before applying its own lock, updating the file and unlocking it again.
  36The most commonly used example of this (and in the case of sendmail, the most
  37troublesome) is access to a user's mailbox. The mail user agent and the mail
  38transfer agent must guard against updating the mailbox at the same time, and
  39prevent reading the mailbox while it is being updated.
  41In a perfect world all processes would use and honour a cooperative, or
  43a lot of poorly written code out there.
  45In trying to address this problem, the designers of System V UNIX came up
  46with a "mandatory" locking scheme, whereby the operating system kernel would
  47block attempts by a process to write to a file that another process holds a
  51The System V mandatory locking scheme was intended to have as little impact as
  52possible on existing user code. The scheme is based on marking individual files
  53as candidates for mandatory locking, and using the existing fcntl()/lockf()
  54interface for applying locks just as if they were normal, advisory locks.
  56Note 1: In saying "file" in the paragraphs above I am actually not telling
  57the whole truth. System V locking is based on fcntl(). The granularity of
  59to entire files, so the mandatory locking rules also have byte level
  62Note 2: POSIX.1 does not specify any scheme for mandatory locking, despite
  63borrowing the fcntl() locking scheme from System V. The mandatory locking
  64scheme is defined by the System V Interface Definition (SVID) Version 3.
  662. Marking a file for mandatory locking
  69A file is marked as a candidate for mandatory locking by setting the group-id
  70bit in its file mode but removing the group-execute bit. This is an otherwise
  71meaningless combination, and was chosen by the System V implementors so as not
  72to break existing user programs.
  74Note that the group-id bit is usually automatically cleared by the kernel when
  75a setgid file is written to. This is a security measure. The kernel has been
  76modified to recognize the special case of a mandatory lock candidate and to
  77refrain from clearing this bit. Similarly the kernel has been modified not
  78to run mandatory lock candidates with setgid privileges.
  803. Available implementations
  83I have considered the implementations of mandatory locking available with
  84SunOS 4.1.x, Solaris 2.x and HP-UX 9.x.
  86Generally I have tried to make the most sense out of the behaviour exhibited
  87by these three reference systems. There are many anomalies.
  89All the reference systems reject all calls to open() for a file on which
  90another process has outstanding mandatory locks. This is in direct
  91contravention of SVID 3, which states that only calls to open() with the
  92O_TRUNC flag set should be rejected. The Linux implementation follows the SVID
  93definition, which is the "Right Thing", since only calls with O_TRUNC can
  94modify the contents of the file.
  97just mandatory locks. That would appear to contravene POSIX.1.
  99mmap() is another interesting case. All the operating systems mentioned
 101also disallows advisory locks for such a file. SVID actually specifies the
 102paranoid HP-UX behaviour.
 104In my opinion only MAP_SHARED mappings should be immune from locking, and then
 105only from mandatory locks - that is what is currently implemented.
 107SunOS is so hopeless that it doesn't even honour the O_NONBLOCK flag for
 108mandatory locks, so reads and writes to locked files always block when they
 109should return EAGAIN.
 111I'm afraid that this is such an esoteric area that the semantics described
 112below are just as valid as any others, so long as the main points seem to
 1154. Semantics
 127   status EAGAIN.
 136   error status EAGAIN.
 1385. Attempts to apply a mandatory lock to a file that is memory mapped and
 140   EAGAIN.
 144   EAGAIN.
 1465. Which system calls are affected?
 152for the purposes of mandatory locking.
 154The affected region is usually defined as stretching from the current position
 155for the total number of bytes read or written. For the truncate calls it is
 156defined as the bytes of a file removed or added (we must also consider bytes
 158range of bytes.)
 160Note 3: I may have overlooked some system calls that need mandatory lock
 162better still fix the system calls yourself and submit a patch to me or Linus.
 1646. Warning!
 167Not even root can override a mandatory lock, so runaway processes can wreak
 168havoc if they lock crucial files. The way around it is to change the file
 169permissions (remove the setgid bit) before trying to read or write to it.
 170Of course, that might be a bit tricky if the system is hung :-(