Back to home page

OSCL-LXR

 
 

    


0001 .. _up_doc:
0002 
0003 RCU on Uniprocessor Systems
0004 ===========================
0005 
0006 A common misconception is that, on UP systems, the call_rcu() primitive
0007 may immediately invoke its function.  The basis of this misconception
0008 is that since there is only one CPU, it should not be necessary to
0009 wait for anything else to get done, since there are no other CPUs for
0010 anything else to be happening on.  Although this approach will *sort of*
0011 work a surprising amount of the time, it is a very bad idea in general.
0012 This document presents three examples that demonstrate exactly how bad
0013 an idea this is.
0014 
0015 Example 1: softirq Suicide
0016 --------------------------
0017 
0018 Suppose that an RCU-based algorithm scans a linked list containing
0019 elements A, B, and C in process context, and can delete elements from
0020 this same list in softirq context.  Suppose that the process-context scan
0021 is referencing element B when it is interrupted by softirq processing,
0022 which deletes element B, and then invokes call_rcu() to free element B
0023 after a grace period.
0024 
0025 Now, if call_rcu() were to directly invoke its arguments, then upon return
0026 from softirq, the list scan would find itself referencing a newly freed
0027 element B.  This situation can greatly decrease the life expectancy of
0028 your kernel.
0029 
0030 This same problem can occur if call_rcu() is invoked from a hardware
0031 interrupt handler.
0032 
0033 Example 2: Function-Call Fatality
0034 ---------------------------------
0035 
0036 Of course, one could avert the suicide described in the preceding example
0037 by having call_rcu() directly invoke its arguments only if it was called
0038 from process context.  However, this can fail in a similar manner.
0039 
0040 Suppose that an RCU-based algorithm again scans a linked list containing
0041 elements A, B, and C in process contexts, but that it invokes a function
0042 on each element as it is scanned.  Suppose further that this function
0043 deletes element B from the list, then passes it to call_rcu() for deferred
0044 freeing.  This may be a bit unconventional, but it is perfectly legal
0045 RCU usage, since call_rcu() must wait for a grace period to elapse.
0046 Therefore, in this case, allowing call_rcu() to immediately invoke
0047 its arguments would cause it to fail to make the fundamental guarantee
0048 underlying RCU, namely that call_rcu() defers invoking its arguments until
0049 all RCU read-side critical sections currently executing have completed.
0050 
0051 Quick Quiz #1:
0052         Why is it *not* legal to invoke synchronize_rcu() in this case?
0053 
0054 :ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
0055 
0056 Example 3: Death by Deadlock
0057 ----------------------------
0058 
0059 Suppose that call_rcu() is invoked while holding a lock, and that the
0060 callback function must acquire this same lock.  In this case, if
0061 call_rcu() were to directly invoke the callback, the result would
0062 be self-deadlock.
0063 
0064 In some cases, it would possible to restructure to code so that
0065 the call_rcu() is delayed until after the lock is released.  However,
0066 there are cases where this can be quite ugly:
0067 
0068 1.      If a number of items need to be passed to call_rcu() within
0069         the same critical section, then the code would need to create
0070         a list of them, then traverse the list once the lock was
0071         released.
0072 
0073 2.      In some cases, the lock will be held across some kernel API,
0074         so that delaying the call_rcu() until the lock is released
0075         requires that the data item be passed up via a common API.
0076         It is far better to guarantee that callbacks are invoked
0077         with no locks held than to have to modify such APIs to allow
0078         arbitrary data items to be passed back up through them.
0079 
0080 If call_rcu() directly invokes the callback, painful locking restrictions
0081 or API changes would be required.
0082 
0083 Quick Quiz #2:
0084         What locking restriction must RCU callbacks respect?
0085 
0086 :ref:`Answers to Quick Quiz <answer_quick_quiz_up>`
0087 
0088 Summary
0089 -------
0090 
0091 Permitting call_rcu() to immediately invoke its arguments breaks RCU,
0092 even on a UP system.  So do not do it!  Even on a UP system, the RCU
0093 infrastructure *must* respect grace periods, and *must* invoke callbacks
0094 from a known environment in which no locks are held.
0095 
0096 Note that it *is* safe for synchronize_rcu() to return immediately on
0097 UP systems, including PREEMPT SMP builds running on UP systems.
0098 
0099 Quick Quiz #3:
0100         Why can't synchronize_rcu() return immediately on UP systems running
0101         preemptable RCU?
0102 
0103 .. _answer_quick_quiz_up:
0104 
0105 Answer to Quick Quiz #1:
0106         Why is it *not* legal to invoke synchronize_rcu() in this case?
0107 
0108         Because the calling function is scanning an RCU-protected linked
0109         list, and is therefore within an RCU read-side critical section.
0110         Therefore, the called function has been invoked within an RCU
0111         read-side critical section, and is not permitted to block.
0112 
0113 Answer to Quick Quiz #2:
0114         What locking restriction must RCU callbacks respect?
0115 
0116         Any lock that is acquired within an RCU callback must be acquired
0117         elsewhere using an _bh variant of the spinlock primitive.
0118         For example, if "mylock" is acquired by an RCU callback, then
0119         a process-context acquisition of this lock must use something
0120         like spin_lock_bh() to acquire the lock.  Please note that
0121         it is also OK to use _irq variants of spinlocks, for example,
0122         spin_lock_irqsave().
0123 
0124         If the process-context code were to simply use spin_lock(),
0125         then, since RCU callbacks can be invoked from softirq context,
0126         the callback might be called from a softirq that interrupted
0127         the process-context critical section.  This would result in
0128         self-deadlock.
0129 
0130         This restriction might seem gratuitous, since very few RCU
0131         callbacks acquire locks directly.  However, a great many RCU
0132         callbacks do acquire locks *indirectly*, for example, via
0133         the kfree() primitive.
0134 
0135 Answer to Quick Quiz #3:
0136         Why can't synchronize_rcu() return immediately on UP systems
0137         running preemptable RCU?
0138 
0139         Because some other task might have been preempted in the middle
0140         of an RCU read-side critical section.  If synchronize_rcu()
0141         simply immediately returned, it would prematurely signal the
0142         end of the grace period, which would come as a nasty shock to
0143         that other thread when it started running again.