title: IB/ipoib: Fix mcast list locking
description: In the Linux kernel, the following vulnerability has been resolved:
IB/ipoib: Fix mcast list locking
Releasing the `priv->lock` while iterating the `priv->multicast_list` in
`ipoib_mcast_join_task()` opens a window for `ipoib_mcast_dev_flush()` to
remove the items while in the middle of iteration. If the mcast is removed
while the lock was dropped, the for loop spins forever resulting in a hard
lockup (as was reported on RHEL 4.18.0-372.75.1.el8_6 kernel):
Task A (kworker/u72:2 below)       | Task B (kworker/u72:0 below)
-----------------------------------+-----------------------------------
ipoib_mcast_join_task(work)        | ipoib_ib_dev_flush_light(work)
spin_lock_irq(&priv->lock)       | __ipoib_ib_dev_flush(priv, ...)
list_for_each_entry(mcast,       | ipoib_mcast_dev_flush(dev = priv->dev)
&priv->multicast_list, list) |
ipoib_mcast_join(dev, mcast)   |
spin_unlock_irq(&priv->lock) |
|   spin_lock_irqsave(&priv->lock, flags)
|   list_for_each_entry_safe(mcast, tmcast,
|                  &priv->multicast_list, list)
|     list_del(&mcast->list);
|     list_add_tail(&mcast->list, &remove_list)
|   spin_unlock_irqrestore(&priv->lock, flags)
spin_lock_irq(&priv->lock)   |
|   ipoib_mcast_remove_list(&remove_list)
(Here, `mcast` is no longer on the  |     list_for_each_entry_safe(mcast, tmcast,
`priv->multicast_list` and we keep |                            remove_list, list)
spinning on the `remove_list` of   |  >>>  wait_for_completion(&mcast->done)
the other thread which is blocked  |
and the list is still valid on     |
it's stack.)
Fix this by keeping the lock held and changing to GFP_ATOMIC to prevent
eventual sleeps.
Unfortunately we could not reproduce the lockup and confirm this fix but
based on the code review I think this fix should address such lockups.
crash> bc 31
PID: 747      TASK: ff1c6a1a007e8000  CPU: 31   COMMAND: "kworker/u72:2"
--
[exception RIP: ipoib_mcast_join_task+0x1b1]
RIP: ffffffffc0944ac1  RSP: ff646f199a8c7e00  RFLAGS: 00000002
RAX: 0000000000000000  RBX: ff1c6a1a04dc82f8  RCX: 0000000000000000
work (&priv->mcast_task{,.work})
RDX: ff1c6a192d60ac68  RSI: 0000000000000286  RDI: ff1c6a1a04dc8000
&mcast->list
RBP: ff646f199a8c7e90   R8: ff1c699980019420   R9: ff1c6a1920c9a000
R10: ff646f199a8c7e00  R11: ff1c6a191a7d9800  R12: ff1c6a192d60ac00
mcast
R13: ff1c6a1d82200000  R14: ff1c6a1a04dc8000  R15: ff1c6a1a04dc82d8
dev                    priv (&priv->lock)     &priv->multicast_list (aka head)
ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
--- <NMI exception stack> ---
#5 [ff646f199a8c7e00] ipoib_mcast_join_task+0x1b1 at ffffffffc0944ac1 [ib_ipoib]
#6 [ff646f199a8c7e98] process_one_work+0x1a7 at ffffffff9bf10967
crash> rx ff646f199a8c7e68
ff646f199a8c7e68:  ff1c6a1a04dc82f8 <<< work = &priv->mcast_task.work
crash> list -hO ipoib_dev_priv.multicast_list ff1c6a1a04dc8000
(empty)
crash> ipoib_dev_priv.mcast_task.work.func,mcast_mutex.owner.counter ff1c6a1a04dc8000
mcast_task.work.func = 0xffffffffc0944910 <ipoib_mcast_join_task>,
mcast_mutex.owner.counter = 0xff1c69998efec000
crash> b 8
PID: 8        TASK: ff1c69998efec000  CPU: 33   COMMAND: "kworker/u72:0"
--
#3 [ff646f1980153d50] wait_for_completion+0x96 at ffffffff9c7d7646
#4 [ff646f1980153d90] ipoib_mcast_remove_list+0x56 at ffffffffc0944dc6 [ib_ipoib]
#5 [ff646f1980153de8] ipoib_mcast_dev_flush+0x1a7 at ffffffffc09455a7 [ib_ipoib]
#6 [ff646f1980153e58] __ipoib_ib_dev_flush+0x1a4 at ffffffffc09431a4 [ib_ipoib]
#7 [ff
---truncated---
affected:
{
product: Linux
vendor: Linux
defaultStatus: unaffected
repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
versions:
version: 1da177e4c3f4, lessThan: 4c8922ae8eb8, status: affected, versionType: git
version: 1da177e4c3f4, lessThan: 615e3adc2042, status: affected, versionType: git
version: 1da177e4c3f4, lessThan: ac2630fd3c90, status: affected, versionType: git
version: 1da177e4c3f4, lessThan: ed790bd0903e, status: affected, versionType: git
version: 1da177e4c3f4, lessThan: 5108a2dc2db5, status: affected, versionType: git
version: 1da177e4c3f4, lessThan: 342258fb46d6, status: affected, versionType: git
version: 1da177e4c3f4, lessThan: 7c7bd4d561e9, status: affected, versionType: git
version: 1da177e4c3f4, lessThan: 4f973e211b3b, status: affected, versionType: git
}
{
product: Linux
vendor: Linux
defaultStatus: affected
repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
versions:
version: 4.19.307, lessThanOrEqual: 4.19.*, status: unaffected, versionType: custom
version: 5.4.269, lessThanOrEqual: 5.4.*, status: unaffected, versionType: custom
version: 5.10.210, lessThanOrEqual: 5.10.*, status: unaffected, versionType: custom
version: 5.15.149, lessThanOrEqual: 5.15.*, status: unaffected, versionType: custom
version: 6.1.77, lessThanOrEqual: 6.1.*, status: unaffected, versionType: custom
version: 6.6.16, lessThanOrEqual: 6.6.*, status: unaffected, versionType: custom
version: 6.7.4, lessThanOrEqual: 6.7.*, status: unaffected, versionType: custom
version: 6.8, lessThanOrEqual: *, status: unaffected, versionType: original_commit_for_fix
}
