edge case posix file lock deadlock detectionmailto:kernelnewbies@kernelnewbies.org2019-08-16T16:22:35ZTheodore Duboistblodt@icloud.comedge case posix file lock deadlock detection2019-01-31T16:06:52Zurn:uuid:8d7b9b7f-598d-ce5a-e8d2-30a5bc2c7742
I'm having trouble figuring out how the kernel handles a particular case in deadlock detection on posix file locks. Here's the scenario:

PID 1: locks byte 2
PID 3: locks byte 0
PID 2: locks byte 10
PID 1: locks byte 10
PID 2: locks bytes 0-2 inclusive

The last step fails with EDEADLK, but I'm not sure how that is detected. The specified range conflicts with two different locks, and the first loop in posix_lock_inode would find whichever one comes first in the linked list, and pass that to the deadlock detector.

If the lock on byte 2 comes first in the list, a cycle would be found between the lock on byte 2 and the lock on byte 10. But if the lock on byte 0 comes first, the deadlock detector would return NULL from what_owner_is_waiting_for on that lock since PID 3 has no other locks, and immediately succeed.

What am I missing?


Kernelnewbies mailing list