)]}' { "log": [ { "commit": "696a02cc16b182dd78b1f395ae336f449cc90f11", "tree": "5411af06d52a3f567e6b3fe592128d3fc0306be1", "parents": [ "c3ce910b1456a45fa88959af3735bd6b285e54af" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Feb 16 21:59:33 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:42:30 2012 -0800" }, "message": "rcu: Hold off RCU_FAST_NO_HZ after timer posted\n\nThis commit handles workloads that transition quickly between idle and\nnon-idle, and where the CPU\u0027s callbacks cannot be invoked, but where\nRCU does not have anything immediate for the CPU to do. Without this\npatch, the RCU_FAST_NO_HZ code can be invoked repeatedly on each entry\nto idle. The commit sets the per-CPU rcu_dyntick_holdoff variable to\nhold off further attempts for a tick.\n\nReported-by: \"Abou Gazala, Neven M\" \u003cneven.m.abou.gazala@intel.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "c3ce910b1456a45fa88959af3735bd6b285e54af", "tree": "d0e74991e637dddbeb3a8ab70ecd983325133893", "parents": [ "8a2ecf474d3ee8dd5d001490349e422cec52f39f" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Feb 14 10:12:54 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:42:11 2012 -0800" }, "message": "rcu: Eliminate softirq-mediated RCU_FAST_NO_HZ idle-entry loop\n\nIf a softirq is pending, the current CPU has RCU callbacks pending,\nand RCU does not immediately need anything from this CPU, then the\ncurrent code resets the RCU_FAST_NO_HZ state machine. This means that\nupon exit from the subsequent softirq handler, RCU_FAST_NO_HZ will\ntry really hard to force RCU into dyntick-idle mode. And if the same\nconditions hold after a few tries (determined by RCU_IDLE_OPT_FLUSHES),\nthe same situation can repeat, possibly endlessly. This scenario is\nnot particularly good for battery lifetime.\n\nThis commit therefore suppresses the early exit from the RCU_FAST_NO_HZ\nstate machine in the case where there is a softirq pending. This change\nforces the state machine to retain its memory, and to enter holdoff if\nthis condition persists.\n\nReported-by: \"Abou Gazala, Neven M\" \u003cneven.m.abou.gazala@intel.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "236fefafe5d3d34b78ed2ccf5510909716112326", "tree": "9ff5ec129e59667341aa07120720b6e4a5ced4d9", "parents": [ "2036d94a7b61ca5032ce90f2bda06afec0fe713e" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jan 31 14:00:41 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:06:08 2012 -0800" }, "message": "rcu: Call out dangers of expedited RCU primitives\n\nThe expedited RCU primitives can be quite useful, but they have some\nhigh costs as well. This commit updates and creates docbook comments\ncalling out the costs, and updates the RCU documentation as well.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "2036d94a7b61ca5032ce90f2bda06afec0fe713e", "tree": "fc9f4b5ba1058e5a2fdf9ccd187766c90f5ae036", "parents": [ "c5fdcec927ee31fc96e92339c3a83ac6e0725289" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Jan 30 17:02:47 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:06:07 2012 -0800" }, "message": "rcu: Rework detection of use of RCU by offline CPUs\n\nBecause newly offlined CPUs continue executing after completing the\nCPU_DYING notifiers, they legitimately enter the scheduler and use\nRCU while appearing to be offline. This calls for a more sophisticated\napproach as follows:\n\n1.\tRCU marks the CPU online during the CPU_UP_PREPARE phase.\n\n2.\tRCU marks the CPU offline during the CPU_DEAD phase.\n\n3.\tDiagnostics regarding use of read-side RCU by offline CPUs use\n\tRCU\u0027s accounting rather than the cpu_online_map. (Note that\n\t__call_rcu() still uses cpu_online_map to detect illegal\n\tinvocations within CPU_DYING notifiers.)\n\n4.\tOffline CPUs are prevented from hanging the system by\n\tforce_quiescent_state(), which pays attention to cpu_online_map.\n\tSome additional work (in a later commit) will be needed to\n\tguarantee that force_quiescent_state() waits a full jiffy before\n\tassuming that a CPU is offline, for example, when called from\n\tidle entry. (This commit also makes the one-jiffy wait\n\texplicit, since the old-style implicit wait can now be defeated\n\tby RCU_FAST_NO_HZ and by rcutorture.)\n\nThis approach avoids the false positives encountered when attempting to\nuse more exact classification of CPU online/offline state.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "c0cfbbb0d4fca14b828a7635a59784adfba8989d", "tree": "a6201565b275d63b17df2a256e9b30c91f94cb6d", "parents": [ "3d3b7db0a22085cfc05c3318b9874f7fb8266d18" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Jan 23 17:23:35 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:06:05 2012 -0800" }, "message": "rcu: No interrupt disabling for rcu_prepare_for_idle()\n\nThe rcu_prepare_for_idle() function is always called with interrupts\ndisabled, so there is no reason to disable interrupts again within\nrcu_prepare_for_idle(). Therefore, this commit removes all of the\ninterrupt disabling, also removing a latent disabling-unbalance bug.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "3d3b7db0a22085cfc05c3318b9874f7fb8266d18", "tree": "7f6a080ca64f8a07dd058c2771b983c602fd1a65", "parents": [ "c0d6d01bffdce19fa19baad6cb8cc3eed7bfd6f5" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Jan 23 17:05:46 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:06:04 2012 -0800" }, "message": "rcu: Move synchronize_sched_expedited() to rcutree.c\n\nNow that TREE_RCU and TREE_PREEMPT_RCU no longer do anything different\nfor the single-CPU case, there is no need for multiple definitions of\nsynchronize_sched_expedited(). It is no longer in any sense a plug-in,\nso move it from kernel/rcutree_plugin.h to kernel/rcutree.c.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "c0d6d01bffdce19fa19baad6cb8cc3eed7bfd6f5", "tree": "ea4c20ddc87b94d7bef2402fe6dbe3186be263c2", "parents": [ "24cd7fd0eaa0d9f5e197ff77a83b006a86696068" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Jan 23 12:41:26 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:06:03 2012 -0800" }, "message": "rcu: Check for illegal use of RCU from offlined CPUs\n\nAlthough it is legal to use RCU during early boot, it is anything\nbut legal to use RCU at runtime from an offlined CPU. After all, RCU\nexplicitly ignores offlined CPUs. This commit therefore adds checks\nfor runtime use of RCU from offlined CPUs.\n\nThese checks are not perfect, in particular, they can be subverted\nthrough use of things like rcu_dereference_raw(). Note that it is not\npossible to put checks in rcu_read_lock() and friends due to the fact\nthat these primitives are used in code that might be used under either\nRCU or lock-based protection, which means that checking rcu_read_lock()\ngets you fat piles of false positives.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "a858af2875fb291d0f4b0a4419fefbf03c2379c0", "tree": "5183796cbf50ec0cd3bd5c741db15705376dacbd", "parents": [ "13cfcca0e4e2d4cee1d0183c049eb34e54ac976e" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Jan 16 13:29:10 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:03:49 2012 -0800" }, "message": "rcu: Print scheduling-clock information on RCU CPU stall-warning messages\n\nThere have been situations where RCU CPU stall warnings were caused by\nissues in scheduling-clock timer initialization. To make it easier to\ntrack these down, this commit causes the RCU CPU stall-warning messages\nto print out the number of scheduling-clock interrupts taken in the\ncurrent grace period for each stalled CPU.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "27565d64a4e564e72c22d8c91a3cfcb9442383e8", "tree": "699319f63f6c1cddc04ce85e77b3718361bbe03f", "parents": [ "5e1ee6e1016763812018bf5c5e966992821dc47e" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Jan 12 19:35:08 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:03:47 2012 -0800" }, "message": "rcu: Remove #ifdef CONFIG_SMP from TREE_RCU\n\nNow that both TINY_RCU and TINY_PREEMPT_RCU have been in place for awhile,\nit is time to remove UP support from TREE_RCU, which is what this commit\ndoes.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "30fbcc90b02187c55c57ff0ecf57cecbd487d694", "tree": "866824e1972d900ceda045c93a3958c74ffbf1e9", "parents": [ "1aa03f1188f7b0b85df2de602b33ee7b6fab8e00" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Jan 12 11:01:14 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:03:44 2012 -0800" }, "message": "rcu: Clean up straggling rcu_preempt_needs_cpu() name\n\nThe recent updates to RCU_CPU_FAST_NO_HZ have an rcu_needs_cpu() that\ndoes more than just check for callbacks, so get the name for\nrcu_preempt_needs_cpu() consistent with that change, now calling it\nrcu_preempt_cpu_has_callbacks().\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "778d250a29224795e6320b58928bafa6b6104a06", "tree": "51eaa45a16731b22797ed88d34584c8b6d55cc01", "parents": [ "091541bbdb9906349481a504e7d8e7fa89f6b6bb" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Jan 10 14:13:24 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:03:36 2012 -0800" }, "message": "rcu: Limit lazy-callback duration\n\nCurrently, a given CPU is permitted to remain in dyntick-idle mode\nindefinitely if it has only lazy RCU callbacks queued. This is vulnerable\nto corner cases in NUMA systems, so limit the time to six seconds by\ndefault. (Currently controlled by a cpp macro.)\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e5601400081651060a59bd1f45f2821bb8e97f95", "tree": "391e7e0b08d8b642eca260950c7f052832411aa9", "parents": [ "ae1f18e480c83d15539fb234a2dc5f0aa04cb119" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sat Jan 07 11:03:57 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:03:34 2012 -0800" }, "message": "rcu: Simplify offline processing\n\nMove -\u003eqsmaskinit and blkd_tasks[] manipulation to the CPU_DYING\nnotifier. This simplifies the code by eliminating a potential\ndeadlock and by reducing the responsibilities of force_quiescent_state().\nAlso rename functions to make their connection to the CPU-hotplug\nstages explicit.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "486e259340fc4c60474f2c14703e3b3634bb58ca", "tree": "70a58702194588fa0773463523f72c682785d040", "parents": [ "0bb7b59d6e2b8440cd7097097dd4bbfc4d76ed07" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Fri Jan 06 14:11:30 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:03:25 2012 -0800" }, "message": "rcu: Avoid waking up CPUs having only kfree_rcu() callbacks\n\nWhen CONFIG_RCU_FAST_NO_HZ is enabled, RCU will allow a given CPU to\nenter dyntick-idle mode even if it still has RCU callbacks queued.\nRCU avoids system hangs in this case by scheduling a timer for several\njiffies in the future. However, if all of the callbacks on that CPU\nare from kfree_rcu(), there is no reason to wake the CPU up, as it is\nnot a problem to defer freeing of memory.\n\nThis commit therefore tracks the number of callbacks on a given CPU\nthat are from kfree_rcu(), and avoids scheduling the timer if all of\na given CPU\u0027s callbacks are from kfree_rcu().\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "fe15d706cfc1cb321dbe2329b04b5ca185edff60", "tree": "bec14ab52167f4f0210e8c207c93922c8270402f", "parents": [ "18fec7d8758dd416904da205375e6fa667defc80" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jan 04 13:30:33 2012 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 21 09:03:23 2012 -0800" }, "message": "rcu: Add lockdep-RCU checks for simple self-deadlock\n\nIt is illegal to have a grace period within a same-flavor RCU read-side\ncritical section, so this commit adds lockdep-RCU checks to splat when\nsuch abuse is encountered. This commit does not detect more elaborate\nRCU deadlock situations. These situations might be a job for lockdep\nenhancements.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "4f89b336fd1edf0c88875d0b7fcdc288c7de903d", "tree": "8cadb7d4c326ad01ce7f0da55db8f03af17917e6", "parents": [ "70321d447aa1a7cc2d60db16234f43c5a65630e7" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Fri Dec 09 14:43:47 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:33:19 2011 -0800" }, "message": "rcu: Apply ACCESS_ONCE() to rcu_boost() return value\n\nBoth TINY_RCU\u0027s and TREE_RCU\u0027s implementations of rcu_boost() access\nthe -\u003eboost_tasks and -\u003eexp_tasks fields without preventing concurrent\nchanges to these fields. This commit therefore applies ACCESS_ONCE in\norder to prevent compiler mischief.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "70321d447aa1a7cc2d60db16234f43c5a65630e7", "tree": "a97749ef35045fc41585d121f595cb0644b319c2", "parents": [ "d493011a376f9df3b8b3da1102509b343b1a4ef2" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 09 14:00:06 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:33:18 2011 -0800" }, "message": "Revert \"rcu: Permit rt_mutex_unlock() with irqs disabled\"\n\nThis reverts commit 5342e269b2b58ee0b0b4168a94087faaa60d0567.\n\nThe approach taken in this patch was deemed too abusive to mutexes,\nand thus too likely to result in maintenance problems in the future.\nInstead, we will disallow RCU read-side critical sections that partially\noverlap with interrupt-disbled code segments.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "f23f7fa1c8effca19b52b98fc71016109d21db59", "tree": "e669b1b0cd96820949dd9deee1e5113fd667fb36", "parents": [ "dff1672d9199fffddb58fa7970ccf59005fc35f3" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Nov 30 15:41:14 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:10 2011 -0800" }, "message": "rcu: Adaptive dyntick-idle preparation\n\nIf there are other CPUs active at a given point in time, then there is a\nlimit to what a given CPU can do to advance the current RCU grace period.\nBeyond this limit, attempting to force the RCU grace period forward will\ndo nothing but consume energy burning CPU cycles.\n\nTherefore, this commit takes an adaptive approach to RCU_FAST_NO_HZ\npreparations for idle. It pushes the RCU core state machine for\ntwo cycles unconditionally, and then it will push from zero to three\nadditional cycles, but only as long as the RCU core has work for this\nCPU to do immediately. The rcu_pending() function is used to check\nwhether the RCU core has such work.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "dff1672d9199fffddb58fa7970ccf59005fc35f3", "tree": "a9740f65dfc203d183cb06ce0faeed0de5cef58b", "parents": [ "facc4e159672b4ed10aa18147bfa187b013c9505" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 29 15:57:13 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:09 2011 -0800" }, "message": "rcu: Keep invoking callbacks if CPU otherwise idle\n\nThe rcu_do_batch() function that invokes callbacks for TREE_RCU and\nTREE_PREEMPT_RCU normally throttles callback invocation to avoid degrading\nscheduling latency. However, as long as the CPU would otherwise be idle,\nthere is no downside to continuing to invoke any callbacks that have passed\nthrough their grace periods. In fact, processing such callbacks in a\ntimely manner has the benefit of increasing the probability that the\nCPU can enter the power-saving dyntick-idle mode.\n\nTherefore, this commit allows callback invocation to continue beyond the\npreset limit as long as the scheduler does not have some other task to\nrun and as long as context is that of the idle task or the relevant\nRCU kthread.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "7cb92499000e3c86dae653077b1465458a039ef6", "tree": "ebc982a5cc562b4fe0cb8f20541f45a5506a0b5f", "parents": [ "3842a0832a1d6eb0b31421f8810a813135967512" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Nov 28 12:28:34 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:07 2011 -0800" }, "message": "rcu: Permit dyntick-idle with callbacks pending\n\nThe current implementation of RCU_FAST_NO_HZ prevents CPUs from entering\ndyntick-idle state if they have RCU callbacks pending. Unfortunately,\nthis has the side-effect of often preventing them from entering this\nstate, especially if at least one other CPU is not in dyntick-idle state.\nHowever, the resulting per-tick wakeup is wasteful in many cases: if the\nCPU has already fully responded to the current RCU grace period, there\nwill be nothing for it to do until this grace period ends, which will\nfrequently take several jiffies.\n\nThis commit therefore permits a CPU that has done everything that the\ncurrent grace period has asked of it (rcu_pending() \u003d\u003d 0) even if it\nstill as RCU callbacks pending. However, such a CPU posts a timer to\nwake it up several jiffies later (6 jiffies, based on experience with\ngrace-period lengths). This wakeup is required to handle situations\nthat can result in all CPUs being in dyntick-idle mode, thus failing\nto ever complete the current grace period. If a CPU wakes up before\nthe timer goes off, then it cancels that timer, thus avoiding spurious\nwakeups.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "3ad0decf98d97b9039d8ed47cee287366b929cdf", "tree": "3d6c02ceb148b55b9632422346978bf369e790e9", "parents": [ "f535a607c13c7b674e0788ca5765779aa74a01c3" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 22 21:08:13 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:03 2011 -0800" }, "message": "rcu: Reduce latency of rcu_prepare_for_idle()\n\nRe-enable interrupts across calls to quiescent-state functions and\nalso across force_quiescent_state() to reduce latency.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "f535a607c13c7b674e0788ca5765779aa74a01c3", "tree": "433a07d16ff11d9f67e7991831cebf3cadfd9939", "parents": [ "84ad00cb61f1cb21f0b63bc6f7dc254399eb3830" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 22 20:43:02 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:02 2011 -0800" }, "message": "rcu: Eliminate RCU_FAST_NO_HZ grace-period hang\n\nWith the new implementation of RCU_FAST_NO_HZ, it was possible to hang\nRCU grace periods as follows:\n\no\tCPU 0 attempts to go idle, cycles several times through the\n\trcu_prepare_for_idle() loop, then goes dyntick-idle when\n\tRCU needs nothing more from it, while still having at least\n\ton RCU callback pending.\n\no\tCPU 1 goes idle with no callbacks.\n\nBoth CPUs can then stay in dyntick-idle mode indefinitely, preventing\nthe RCU grace period from ever completing, possibly hanging the system.\n\nThis commit therefore prevents CPUs that have RCU callbacks from entering\ndyntick-idle mode. This approach also eliminates the need for the\nend-of-grace-period IPIs used previously.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "84ad00cb61f1cb21f0b63bc6f7dc254399eb3830", "tree": "7f275af0caaead54a5e8847afa308f5e2b33997a", "parents": [ "3084f2f80cc8a1fd66233722d88beac0fe85e26f" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 22 17:46:19 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:01 2011 -0800" }, "message": "rcu: Avoid needlessly IPIing CPUs at GP end\n\nIf a CPU enters dyntick-idle mode with callbacks pending, it will need\nan IPI at the end of the grace period. However, if it exits dyntick-idle\nmode before the grace period ends, it will be needlessly IPIed at the\nend of the grace period.\n\nTherefore, this commit clears the per-CPU rcu_awake_at_gp_end flag\nwhen a CPU determines that it does not need it. This in turn requires\ndisabling interrupts across much of rcu_prepare_for_idle() in order to\navoid having nested interrupts clearing this state out from under us.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "3084f2f80cc8a1fd66233722d88beac0fe85e26f", "tree": "bce77805235278599eda0572d5d36b11419681d7", "parents": [ "433cdddcd9ac5558068edd7f8d4707a70f7710f5" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 22 17:07:11 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:00 2011 -0800" }, "message": "rcu: Go dyntick-idle more quickly if CPU has serviced current grace period\n\nThe earlier version would attempt to push callbacks through five times\nbefore going into dyntick-idle mode if callbacks remained, but the CPU\nhad done all that it needed to do for the current RCU grace periods.\nThis is wasteful: In most cases, once the CPU has done all that it\nneeds to for the current RCU grace periods, it will make no further\nprogress on the callbacks no matter how many times it loops through\nthe RCU core processing and the idle-entry code.\n\nThis commit therefore goes to dyntick-idle mode whenever the current\nCPU has done all it can for the current grace period.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "433cdddcd9ac5558068edd7f8d4707a70f7710f5", "tree": "7f3686eb64c9ee3a239e5920df588c70837c5637", "parents": [ "045fb9315a2129023d70a0eecf0942e18fca4fcd" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 22 14:58:03 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:00 2011 -0800" }, "message": "rcu: Add tracing for RCU_FAST_NO_HZ\n\nThis commit adds trace_rcu_prep_idle(), which is invoked from\nrcu_prepare_for_idle() and rcu_wake_cpu() to trace attempts on\nthe part of RCU to force CPUs into dyntick-idle mode.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "aea1b35e29e658d42d7ba2237f3aa7f93e18509d", "tree": "0221b1c12b5c3e3ed7f2bb2ffc957b09891bcb51", "parents": [ "0989cb46783188ea7346ba6490be0046b9b7a725" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Nov 02 06:54:54 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:43 2011 -0800" }, "message": "rcu: Allow dyntick-idle mode for CPUs with callbacks\n\nCurrently, RCU does not permit a CPU to enter dyntick-idle mode if that\nCPU has any RCU callbacks queued. This means that workloads for which\neach CPU wakes up and does some RCU updates every few ticks will never\nenter dyntick-idle mode. This can result in significant unnecessary power\nconsumption, so this patch permits a given to enter dyntick-idle mode if\nit has callbacks, but only if that same CPU has completed all current\nwork for the RCU core. We determine use rcu_pending() to determine\nwhether a given CPU has completed all current work for the RCU core.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "a0f8eefb127f5be07628954f310a7fc8c82b2fc3", "tree": "58fa4263824fa3b907b9ef70b1255fcf7c3c791b", "parents": [ "b40d293eb36ba40cd428b6d178db911174689702" ], "author": { "name": "Thomas Gleixner", "email": "tglx@linutronix.de", "time": "Thu Nov 03 12:08:17 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:29 2011 -0800" }, "message": "rcu: Remove redundant return from rcu_report_exp_rnp()\n\nEmpty void functions do not need \"return\", so this commit removes it\nfrom rcu_report_exp_rnp().\n\nSigned-off-by: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "b40d293eb36ba40cd428b6d178db911174689702", "tree": "78109a13c0bf86608f3caaea547fd9e948aee743", "parents": [ "34240697d619c439c55f21989680024dcb604aab" ], "author": { "name": "Thomas Gleixner", "email": "tglx@linutronix.de", "time": "Sat Oct 22 07:12:34 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:28 2011 -0800" }, "message": "rcu: Omit self-awaken when setting up expedited grace period\n\nWhen setting up an expedited grace period, if there were no readers, the\ntask will awaken itself. This commit removes this useless self-awakening.\n\nSigned-off-by: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "7077714ec4940a6c5b1189c3afb4f47bf49ad877", "tree": "c8713205a1e6ce6973822bb76ea5e7e1774444ee", "parents": [ "389abd48efe1ceacb141b2fd151263b1bc432dbc" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Sep 22 13:18:44 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:22 2011 -0800" }, "message": "rcu: Make synchronize_sched_expedited() better at work sharing\n\nWhen synchronize_sched_expedited() takes its second and subsequent\nsnapshots of sync_sched_expedited_started, it subtracts 1. This\nmeans that the concurrent caller of synchronize_sched_expedited()\nthat incremented to that value sees our successful completion, it\nwill not be able to take advantage of it. This restriction is\npointless, given that our full expedited grace period would have\nhappened after the other guy started, and thus should be able to\nserve as a proxy for the other guy successfully executing\ntry_stop_cpus().\n\nThis commit therefore removes the subtraction of 1.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "389abd48efe1ceacb141b2fd151263b1bc432dbc", "tree": "f61a46a05c11b3d6ecd10b6beb73824bd5e88388", "parents": [ "af446b702c58b700cc5fa99f6edc78b99e55b995" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 21 14:41:37 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:21 2011 -0800" }, "message": "rcu: Avoid RCU-preempt expedited grace-period botch\n\nBecause rcu_read_unlock_special() samples rcu_preempted_readers_exp(rnp)\nafter dropping rnp-\u003elock, the following sequence of events is possible:\n\n1.\tTask A exits its RCU read-side critical section, and removes\n\titself from the -\u003eblkd_tasks list, releases rnp-\u003elock, and is\n\tthen preempted. Task B remains on the -\u003eblkd_tasks list, and\n\tblocks the current expedited grace period.\n\n2.\tTask B exits from its RCU read-side critical section and removes\n\titself from the -\u003eblkd_tasks list. Because it is the last task\n\tblocking the current expedited grace period, it ends that\n\texpedited grace period.\n\n3.\tTask A resumes, and samples rcu_preempted_readers_exp(rnp) which\n\tof course indicates that nothing is blocking the nonexistent\n\texpedited grace period. Task A is again preempted.\n\n4.\tSome other CPU starts an expedited grace period. There are several\n\ttasks blocking this expedited grace period queued on the\n\tsame rcu_node structure that Task A was using in step 1 above.\n\n5.\tTask A examines its state and incorrectly concludes that it was\n\tthe last task blocking the expedited grace period on the current\n\trcu_node structure. It therefore reports completion up the\n\trcu_node tree.\n\n6.\tThe expedited grace period can then incorrectly complete before\n\tthe tasks blocked on this same rcu_node structure exit their\n\tRCU read-side critical sections. Arbitrarily bad things happen.\n\nThis commit therefore takes a snapshot of rcu_preempted_readers_exp(rnp)\nprior to dropping the lock, so that only the last task thinks that it is\nthe last task, thus avoiding the failure scenario laid out above.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "e90c53d3e238dd0b7b02964370e8fece1778df96", "tree": "23e5b8623c2c37bd8a4656c66266209872f80701", "parents": [ "5b61b0baa9e80289c53413e573befc5790a04ac7" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat Aug 20 18:29:32 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:48 2011 -0700" }, "message": "rcu: Remove rcu_needs_cpu_flush() to avoid false quiescent states\n\nThe purpose of rcu_needs_cpu_flush() was to iterate on pushing the\ncurrent grace period in order to help the current CPU enter dyntick-idle\nmode. However, this can result in failures if the CPU starts entering\ndyntick-idle mode, but then backs out. In this case, the call to\nrcu_pending() from rcu_needs_cpu_flush() might end up announcing a\nnon-existing quiescent state.\n\nThis commit therefore removes rcu_needs_cpu_flush() in favor of letting\nthe dyntick-idle machinery at the end of the softirq handler push the\nloop along via its call to rcu_pending().\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "5b61b0baa9e80289c53413e573befc5790a04ac7", "tree": "5f3c4b002089b6d6e92fafaba42ab4a42ef1f1f9", "parents": [ "ab8f11e5f6655861ad4758a7da76b2fc0e0dcc98" ], "author": { "name": "Mike Galbraith", "email": "efault@gmx.de", "time": "Fri Aug 19 11:39:11 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:47 2011 -0700" }, "message": "rcu: Wire up RCU_BOOST_PRIO for rcutree\n\nRCU boost threads start life at RCU_BOOST_PRIO, while others remain\nat RCU_KTHREAD_PRIO. While here, change thread names to match other\nkthreads, and adjust rcu_yield() to not override the priority set by\nthe user. This last change sets the stage for runtime changes to\npriority in the -rt tree.\n\nSigned-off-by: Mike Galbraith \u003cefault@gmx.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "5342e269b2b58ee0b0b4168a94087faaa60d0567", "tree": "77467d64948328e506f06eaf5a719de7fade4b83", "parents": [ "06ae115a1d551cd952d80df06eaf8b5153351875" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Aug 16 17:46:46 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:43 2011 -0700" }, "message": "rcu: Permit rt_mutex_unlock() with irqs disabled\n\nCreate a separate lockdep class for the rt_mutex used for RCU priority\nboosting and enable use of rt_mutex_lock() with irqs disabled. This\nprevents RCU priority boosting from falling prey to deadlocks when\nsomeone begins an RCU read-side critical section in preemptible state,\nbut releases it with an irq-disabled lock held.\n\nUnfortunately, the scheduler\u0027s runqueue and priority-inheritance locks\nstill must either completely enclose or be completely enclosed by any\noverlapping RCU read-side critical section.\n\nThis version removes a redundant local_irq_restore() noted by\nYong Zhang.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "9bc8b5586f94be6391458074ecbba8827ba8ba9d", "tree": "323804bb7180337c266f88519cea6b1f42114ead", "parents": [ "037067a1b6f9a70f862f3ed9d59fe28b7cd55ac4" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat Aug 13 13:31:47 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:41 2011 -0700" }, "message": "rcu: Suppress NMI backtraces when stall ends before dump\n\nIt is possible for an RCU CPU stall to end just as it is detected, in\nwhich case the current code will uselessly dump all CPU\u0027s stacks.\nThis commit therefore checks for this condition and refrains from\nsending needless NMIs.\n\nAnd yes, the stall might also end just after we checked all CPUs and\ntasks, but in that case we would at least have given some clue as\nto which CPU/task was at fault.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "82e78d80fc392ac7e98326bc8beeb8a679913ffd", "tree": "4999d3c03ed711056474c3a31512e7d3cf253b87", "parents": [ "5c51dd7349d4bb26f845f17f85daa168f5fa03f2" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Aug 04 07:55:34 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:39 2011 -0700" }, "message": "rcu: Simplify unboosting checks\n\nCommit 7765be (Fix RCU_BOOST race handling current-\u003ercu_read_unlock_special)\nintroduced a new -\u003ercu_boosted field in the task structure. This is\nredundant because the existing -\u003ercu_boost_mutex will be non-NULL at\nany time that -\u003ercu_boosted is nonzero. Therefore, this commit removes\n-\u003ercu_boosted and tests -\u003ercu_boost_mutex instead.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "6206ab9bab620fc0fbbed30ce20d145b0b3d1840", "tree": "b57912ba9fff6f2deba89c0178d3122bbd1fda6e", "parents": [ "6846c0c54074d47927c90eab4a805115e1ae3292" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Aug 01 06:22:11 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:35 2011 -0700" }, "message": "rcu: Move __rcu_read_unlock()\u0027s barrier() within if-statement\n\nWe only need to constrain the compiler if we are actually exiting\nthe top-level RCU read-side critical section. This commit therefore\nmoves the first barrier() cal in __rcu_read_unlock() to inside the\n\"if\" statement, thus avoiding needless register flushes for inner\nrcu_read_unlock() calls.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e4cc1f22b2f4e9b0207a8cdb63e56dcf99e82d35", "tree": "ec3fad6587f0a83e764a571394bdfb85a84edf30", "parents": [ "d4c08f2ac311a360230eef7e5395b0ec8d8f0670" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Jun 27 00:17:43 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:22 2011 -0700" }, "message": "rcu: Simplify quiescent-state accounting\n\nThere is often a delay between the time that a CPU passes through a\nquiescent state and the time that this quiescent state is reported to the\nRCU core. It is quite possible that the grace period ended before the\nquiescent state could be reported, for example, some other CPU might have\ndeduced that this CPU passed through dyntick-idle mode. It is critically\nimportant that quiescent state be counted only against the grace period\nthat was in effect at the time that the quiescent state was detected.\n\nPreviously, this was handled by recording the number of the last grace\nperiod to complete when passing through a quiescent state. The RCU\ncore then checks this number against the current value, and rejects\nthe quiescent state if there is a mismatch. However, one additional\npossibility must be accounted for, namely that the quiescent state was\nrecorded after the prior grace period completed but before the current\ngrace period started. In this case, the RCU core must reject the\nquiescent state, but the recorded number will match. This is handled\nwhen the CPU becomes aware of a new grace period -- at that point,\nit invalidates any prior quiescent state.\n\nThis works, but is a bit indirect. The new approach records the current\ngrace period, and the RCU core checks to see (1) that this is still the\ncurrent grace period and (2) that this grace period has not yet ended.\nThis approach simplifies reasoning about correctness, and this commit\nchanges over to this new approach.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "d4c08f2ac311a360230eef7e5395b0ec8d8f0670", "tree": "06e425b8153e076fbe43b037cf4497ac50afddb7", "parents": [ "965a002b4f1a458c5dcb334ec29f48a0046faa25" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sat Jun 25 06:36:56 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:21 2011 -0700" }, "message": "rcu: Add grace-period, quiescent-state, and call_rcu trace events\n\nAdd trace events to record grace-period start and end, quiescent states,\nCPUs noticing grace-period start and end, grace-period initialization,\ncall_rcu() invocation, tasks blocking in RCU read-side critical sections,\ntasks exiting those same critical sections, force_quiescent_state()\ndetection of dyntick-idle and offline CPUs, CPUs entering and leaving\ndyntick-idle mode (except from NMIs), CPUs coming online and going\noffline, and CPUs being kicked for staying in dyntick-idle mode for too\nlong (as in many weeks, even on 32-bit systems).\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n\nrcu: Add the rcu flavor to callback trace events\n\nThe earlier trace events for registering RCU callbacks and for invoking\nthem did not include the RCU flavor (rcu_bh, rcu_preempt, or rcu_sched).\nThis commit adds the RCU flavor to those trace events.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "385680a9487d2f85382ad6d74e2a15837e47bfd9", "tree": "68809ef9cdfeee43b7c08152bfdbc45d2c0d6489", "parents": [ "eab0993c7ba5c7d9b3613d6037e0f31f0ccbe181" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Jun 21 22:43:26 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:19 2011 -0700" }, "message": "rcu: Add event-trace markers to TREE_RCU kthreads\n\nAdd event-trace markers to TREE_RCU kthreads to allow including these\nkthread\u0027s CPU time in the utilization calculations.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e0f23060adfa3f27beaa7918eff70258b88471b6", "tree": "f0e8ed4ad97a1fc4d3c2c5af639a63063ae52e32", "parents": [ "72fe701b70e6ced35d734b676c13efbc8fc769a9" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Jun 21 01:29:39 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:16 2011 -0700" }, "message": "rcu: Update comments to reflect softirqs vs. kthreads\n\nWe now have kthreads only for flavors of RCU that support boosting,\nso update the now-misleading comments accordingly.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e99033c5c160f1f247c665923a66acec693a967c", "tree": "c5b0863ad4b673134070872a159055237186f0fe", "parents": [ "300df91ca9358f7f09298eec9503c12b32054ef7" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Jun 21 00:13:44 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:14 2011 -0700" }, "message": "rcu: Put names into TINY_RCU structures under RCU_TRACE\n\nIn order to allow event tracing to distinguish between flavors of\nRCU, we need those names in the relevant RCU data structures. TINY_RCU\nhas avoided them for memory-footprint reasons, so add them only if\nCONFIG_RCU_TRACE\u003dy.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "2c42818e962e2858334bf45bfc56662b3752df34", "tree": "192364123c9aeeab282c53168e51eddece9d8be4", "parents": [ "f039d1f1884b2fe9c13d28f59d8330f0b0518fc4" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 22:14:36 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:36:42 2011 -0700" }, "message": "rcu: Abstract common code for RCU grace-period-wait primitives\n\nPull the code that waits for an RCU grace period into a single function,\nwhich is then called by synchronize_rcu() and friends in the case of\nTREE_RCU and TREE_PREEMPT_RCU, and from rcu_barrier() and friends in\nthe case of TINY_RCU and TINY_PREEMPT_RCU.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "1eb521210a8c9823038abe4ddfe8c69e713ec17d", "tree": "f9fc70df7c25f80def12aeb231475405043eb1ed", "parents": [ "1f288094807861ec1e48c428d2c49ccf7aaf3767" ], "author": { "name": "Shaohua Li", "email": "shaohua.li@intel.com", "time": "Thu Jun 16 16:02:54 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:36:34 2011 -0700" }, "message": "rcu: Avoid unnecessary self-wakeup of per-CPU kthreads\n\nThere are a number of cases where the RCU can find additional work\nfor the per-CPU kthread within the context of that per-CPU kthread.\nIn such cases, the per-CPU kthread is already running, so attempting\nto wake itself up does nothing except waste CPU cycles. This commit\ntherefore checks to see if it is in the per-CPU kthread context,\nomitting the wakeup in this case.\n\nSigned-off-by: Shaohua Li \u003cshaohua.li@intel.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "1f288094807861ec1e48c428d2c49ccf7aaf3767", "tree": "f8f1217bcf0a720d6ae17304b1bcb3ddbecb42be", "parents": [ "a102a9ece5489e1718cd7543aa079082450ac3a2" ], "author": { "name": "Eric Dumazet", "email": "eric.dumazet@gmail.com", "time": "Thu Jun 16 15:53:18 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:36:33 2011 -0700" }, "message": "rcu: Use kthread_create_on_node()\n\nCommit a26ac2455ffc (move TREE_RCU from softirq to kthread) added\nper-CPU kthreads. However, kthread creation uses kthread_create(), which\ncan put the kthread\u0027s stack and task struct on the wrong NUMA node.\nTherefore, use kthread_create_on_node() instead of kthread_create()\nso that the stacks and task structs are placed on the correct NUMA node.\n\nA similar change was carried out in commit 94dcf29a11b3 (kthread:\nuse kthread_create_on_node()).\n\nAlso change rcutorture\u0027s priority-boost-test kthread creation.\n\nSigned-off-by: Eric Dumazet \u003ceric.dumazet@gmail.com\u003e\nCC: Tejun Heo \u003ctj@kernel.org\u003e\nCC: Rusty Russell \u003crusty@rustcorp.com.au\u003e\nCC: Andrew Morton \u003cakpm@linux-foundation.org\u003e\nCC: Andi Kleen \u003cak@linux.intel.com\u003e\nCC: Ingo Molnar \u003cmingo@elte.hu\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "ec433f0c51527426989ea8a38a856d810d739414", "tree": "001550d7b18b3f2b5326da3f9dfd893e31c8dc1c", "parents": [ "c5d753a55ac92e09816d410cd17093813f1a904b" ], "author": { "name": "Peter Zijlstra", "email": "a.p.zijlstra@chello.nl", "time": "Tue Jul 19 15:32:00 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jul 20 10:50:12 2011 -0700" }, "message": "softirq,rcu: Inform RCU of irq_exit() activity\n\nThe rcu_read_unlock_special() function relies on in_irq() to exclude\nscheduler activity from interrupt level. This fails because exit_irq()\ncan invoke the scheduler after clearing the preempt_count() bits that\nin_irq() uses to determine that it is at interrupt level. This situation\ncan result in failures as follows:\n\n $task\t\t\tIRQ\t\tSoftIRQ\n\n rcu_read_lock()\n\n /* do stuff */\n\n \u003cpreempt\u003e |\u003d UNLOCK_BLOCKED\n\n rcu_read_unlock()\n --t-\u003ercu_read_lock_nesting\n\n\t\t\tirq_enter();\n\t\t\t/* do stuff, don\u0027t use RCU */\n\t\t\tirq_exit();\n\t\t\t sub_preempt_count(IRQ_EXIT_OFFSET);\n\t\t\t invoke_softirq()\n\n\t\t\t\t\tttwu();\n\t\t\t\t\t spin_lock_irq(\u0026pi-\u003elock)\n\t\t\t\t\t rcu_read_lock();\n\t\t\t\t\t /* do stuff */\n\t\t\t\t\t rcu_read_unlock();\n\t\t\t\t\t rcu_read_unlock_special()\n\t\t\t\t\t rcu_report_exp_rnp()\n\t\t\t\t\t ttwu()\n\t\t\t\t\t spin_lock_irq(\u0026pi-\u003elock) /* deadlock */\n\n rcu_read_unlock_special(t);\n\nEd can simply trigger this \u0027easy\u0027 because invoke_softirq() immediately\ndoes a ttwu() of ksoftirqd/# instead of doing the in-place softirq stuff\nfirst, but even without that the above happens.\n\nCure this by also excluding softirqs from the\nrcu_read_unlock_special() handler and ensuring the force_irqthreads\nksoftirqd/# wakeup is done from full softirq context.\n\n[ Alternatively, delaying the -\u003ercu_read_lock_nesting decrement\n until after the special handling would make the thing more robust\n in the face of interrupts as well. And there is a separate patch\n for that. ]\n\nCc: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nReported-and-tested-by: Ed Tomlinson \u003cedt@aei.ca\u003e\nSigned-off-by: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "10f39bb1b2c1923ca73e70cb13aeee0e9b822d8f", "tree": "2e8349c57b1f0563a27be49410f413b643ded1a9", "parents": [ "be0e1e21ef707be4d16ea6a96ac9997463e4b8d2" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sun Jul 17 21:14:35 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jul 20 10:50:11 2011 -0700" }, "message": "rcu: protect __rcu_read_unlock() against scheduler-using irq handlers\n\nThe addition of RCU read-side critical sections within runqueue and\npriority-inheritance lock critical sections introduced some deadlock\ncycles, for example, involving interrupts from __rcu_read_unlock()\nwhere the interrupt handlers call wake_up(). This situation can cause\nthe instance of __rcu_read_unlock() invoked from interrupt to do some\nof the processing that would otherwise have been carried out by the\ntask-level instance of __rcu_read_unlock(). When the interrupt-level\ninstance of __rcu_read_unlock() is called with a scheduler lock held\nfrom interrupt-entry/exit situations where in_irq() returns false,\ndeadlock can result.\n\nThis commit resolves these deadlocks by using negative values of\nthe per-task -\u003ercu_read_lock_nesting counter to indicate that an\ninstance of __rcu_read_unlock() is in flight, which in turn prevents\ninstances from interrupt handlers from doing any special processing.\nThis patch is inspired by Steven Rostedt\u0027s earlier patch that similarly\nmade __rcu_read_unlock() guard against interrupt-mediated recursion\n(see https://lkml.org/lkml/2011/7/15/326), but this commit refines\nSteven\u0027s approach to avoid the need for preemption disabling on the\n__rcu_read_unlock() fastpath and to also avoid the need for manipulating\na separate per-CPU variable.\n\nThis patch avoids need for preempt_disable() by instead using negative\nvalues of the per-task -\u003ercu_read_lock_nesting counter. Note that nested\nrcu_read_lock()/rcu_read_unlock() pairs are still permitted, but they will\nnever see -\u003ercu_read_lock_nesting go to zero, and will therefore never\ninvoke rcu_read_unlock_special(), thus preventing them from seeing the\nRCU_READ_UNLOCK_BLOCKED bit should it be set in -\u003ercu_read_unlock_special.\nThis patch also adds a check for -\u003ercu_read_unlock_special being negative\nin rcu_check_callbacks(), thus preventing the RCU_READ_UNLOCK_NEED_QS\nbit from being set should a scheduling-clock interrupt occur while\n__rcu_read_unlock() is exiting from an outermost RCU read-side critical\nsection.\n\nOf course, __rcu_read_unlock() can be preempted during the time that\n-\u003ercu_read_lock_nesting is negative. This could result in the setting\nof the RCU_READ_UNLOCK_BLOCKED bit after __rcu_read_unlock() checks it,\nand would also result it this task being queued on the corresponding\nrcu_node structure\u0027s blkd_tasks list. Therefore, some later RCU read-side\ncritical section would enter rcu_read_unlock_special() to clean up --\nwhich could result in deadlock if that critical section happened to be in\nthe scheduler where the runqueue or priority-inheritance locks were held.\n\nThis situation is dealt with by making rcu_preempt_note_context_switch()\ncheck for negative -\u003ercu_read_lock_nesting, thus refraining from\nqueuing the task (and from setting RCU_READ_UNLOCK_BLOCKED) if we are\nalready exiting from the outermost RCU read-side critical section (in\nother words, we really are no longer actually in that RCU read-side\ncritical section). In addition, rcu_preempt_note_context_switch()\ninvokes rcu_read_unlock_special() to carry out the cleanup in this case,\nwhich clears out the -\u003ercu_read_unlock_special bits and dequeues the task\n(if necessary), in turn avoiding needless delay of the current RCU grace\nperiod and needless RCU priority boosting.\n\nIt is still illegal to call rcu_read_unlock() while holding a scheduler\nlock if the prior RCU read-side critical section has ever had either\npreemption or irqs enabled. However, the common use case is legal,\nnamely where then entire RCU read-side critical section executes with\nirqs disabled, for example, when the scheduler lock is held across the\nentire lifetime of the RCU read-side critical section.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "be0e1e21ef707be4d16ea6a96ac9997463e4b8d2", "tree": "fe53c0731f5c3a6e56e2358eee9b8f17c3af83f4", "parents": [ "7765be2fec0f476fcd61812d5f9406b04c765020" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat May 21 05:57:18 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jul 19 21:38:53 2011 -0700" }, "message": "rcu: Streamline code produced by __rcu_read_unlock()\n\nGiven some common flag combinations, particularly -Os, gcc will inline\nrcu_read_unlock_special() despite its being in an unlikely() clause.\nUse noinline to prohibit this misoptimization.\n\nIn addition, move the second barrier() in __rcu_read_unlock() so that\nit is not on the common-case code path. This will allow the compiler to\ngenerate better code for the common-case path through __rcu_read_unlock().\n\nSuggested-by: Linus Torvalds \u003ctorvalds@linux-foundation.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nAcked-by: Mathieu Desnoyers \u003cmathieu.desnoyers@efficios.com\u003e\n" }, { "commit": "7765be2fec0f476fcd61812d5f9406b04c765020", "tree": "e89ccb3fbd5655090b05f608f8e1d39924f23a9a", "parents": [ "131906b0062ddde7f85bbe67754983c754648bd8" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Jul 14 12:24:11 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jul 19 21:38:52 2011 -0700" }, "message": "rcu: Fix RCU_BOOST race handling current-\u003ercu_read_unlock_special\n\nThe RCU_BOOST commits for TREE_PREEMPT_RCU introduced an other-task\nwrite to a new RCU_READ_UNLOCK_BOOSTED bit in the task_struct structure\u0027s\n-\u003ercu_read_unlock_special field, but, as noted by Steven Rostedt, without\ncorrectly synchronizing all accesses to -\u003ercu_read_unlock_special.\nThis could result in bits in -\u003ercu_read_unlock_special being spuriously\nset and cleared due to conflicting accesses, which in turn could result\nin deadlocks between the rcu_node structure\u0027s -\u003elock and the scheduler\u0027s\nrq and pi locks. These deadlocks would result from RCU incorrectly\nbelieving that the just-ended RCU read-side critical section had been\npreempted and/or boosted. If that RCU read-side critical section was\nexecuted with either rq or pi locks held, RCU\u0027s ensuing (incorrect)\ncalls to the scheduler would cause the scheduler to attempt to once\nagain acquire the rq and pi locks, resulting in deadlock. More complex\ndeadlock cycles are also possible, involving multiple rq and pi locks\nas well as locks from multiple rcu_node structures.\n\nThis commit fixes synchronization by creating -\u003ercu_boosted field in\ntask_struct that is accessed and modified only when holding the -\u003elock\nin the rcu_node structure on which the task is queued (on that rcu_node\nstructure\u0027s -\u003eblkd_tasks list). This results in tasks accessing only\ntheir own current-\u003ercu_read_unlock_special fields, making unsynchronized\naccess once again legal, and keeping the rcu_read_unlock() fastpath free\nof atomic instructions and memory barriers.\n\nThe reason that the rcu_read_unlock() fastpath does not need to access\nthe new current-\u003ercu_boosted field is that this new field cannot\nbe non-zero unless the RCU_READ_UNLOCK_BLOCKED bit is set in the\ncurrent-\u003ercu_read_unlock_special field. Therefore, rcu_read_unlock()\nneed only test current-\u003ercu_read_unlock_special: if that is zero, then\ncurrent-\u003ercu_boosted must also be zero.\n\nThis bug does not affect TINY_PREEMPT_RCU because this implementation\nof RCU accesses current-\u003ercu_read_unlock_special with irqs disabled,\nthus preventing races on the !SMP systems that TINY_PREEMPT_RCU runs on.\n\nMaybe-reported-by: Dave Jones \u003cdavej@redhat.com\u003e\nMaybe-reported-by: Sergey Senozhatsky \u003csergey.senozhatsky@gmail.com\u003e\nReported-by: Steven Rostedt \u003crostedt@goodmis.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Steven Rostedt \u003crostedt@goodmis.org\u003e\n" }, { "commit": "131906b0062ddde7f85bbe67754983c754648bd8", "tree": "1b92eacb707befb294b4d9985f10aa44715cba16", "parents": [ "b0d304172f49061b4ff78f9e2b02719ac69c8a7e" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Jul 17 02:05:49 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jul 19 21:38:51 2011 -0700" }, "message": "rcu: decrease rcu_report_exp_rnp coupling with scheduler\n\nPREEMPT_RCU read-side critical sections blocking an expedited grace\nperiod invoke rcu_report_exp_rnp(). When the last such critical section\nhas completed, rcu_report_exp_rnp() invokes the scheduler to wake up the\ntask that invoked synchronize_rcu_expedited() -- needlessly holding the\nroot rcu_node structure\u0027s lock while doing so, thus needlessly providing\na way for RCU and the scheduler to deadlock.\n\nThis commit therefore releases the root rcu_node structure\u0027s lock before\ncalling wake_up().\n\nReported-by: Ed Tomlinson \u003cedt@aei.ca\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "b0d304172f49061b4ff78f9e2b02719ac69c8a7e", "tree": "882123e565ced895910490514f6e676c708b5001", "parents": [ "620917de59eeb934b9f8cf35cc2d95c1ac8ed0fc" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sun Jul 10 15:57:35 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jul 13 08:17:56 2011 -0700" }, "message": "rcu: Prevent RCU callbacks from executing before scheduler initialized\n\nUnder some rare but real combinations of configuration parameters, RCU\ncallbacks are posted during early boot that use kernel facilities that\nare not yet initialized. Therefore, when these callbacks are invoked,\nhard hangs and crashes ensue. This commit therefore prevents RCU\ncallbacks from being invoked until after the scheduler is fully up and\nrunning, as in after multiple tasks have been spawned.\n\nIt might well turn out that a better approach is to identify the specific\nRCU callbacks that are causing this problem, but that discussion will\nwait until such time as someone really needs an RCU callback to be invoked\n(as opposed to merely registered) during early boot.\n\nReported-by: julie Sullivan \u003ckernelmail.jms@gmail.com\u003e\nReported-by: RKK \u003ckulkarni.ravi4@gmail.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Konrad Rzeszutek Wilk \u003ckonrad.wilk@oracle.com\u003e\nTested-by: julie Sullivan \u003ckernelmail.jms@gmail.com\u003e\nTested-by: RKK \u003ckulkarni.ravi4@gmail.com\u003e\n" }, { "commit": "f8b7fc6b514f34a51875dd48dff70d4d17a54f38", "tree": "dc6583851e3536516761b0fd9b07f5a3a1743ebb", "parents": [ "a46e0899eec7a3069bcadd45dfba7bf67c6ed016" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Jun 16 08:26:32 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Jun 16 16:12:05 2011 -0700" }, "message": "rcu: Move RCU_BOOST #ifdefs to header file\n\nThe commit \"use softirq instead of kthreads except when RCU_BOOST\u003dy\"\njust applied #ifdef in place. This commit is a cleanup that moves\nthe newly #ifdef\u0027ed code to the header file kernel/rcutree_plugin.h.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "a46e0899eec7a3069bcadd45dfba7bf67c6ed016", "tree": "78158b4056fe1365d5086f66769abdc3ef2643c3", "parents": [ "09223371deac67d08ca0b70bd18787920284c967" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jun 15 15:47:09 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jun 15 23:07:21 2011 -0700" }, "message": "rcu: use softirq instead of kthreads except when RCU_BOOST\u003dy\n\nThis patch #ifdefs RCU kthreads out of the kernel unless RCU_BOOST\u003dy,\nthus eliminating context-switch overhead if RCU priority boosting has\nnot been configured.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "09223371deac67d08ca0b70bd18787920284c967", "tree": "0ad0354a93ac209c8cfd2b79d03d0462aa3c5c58", "parents": [ "9a432736904d386cda28b987b38ba14dae960ecc" ], "author": { "name": "Shaohua Li", "email": "shaohua.li@intel.com", "time": "Tue Jun 14 13:26:25 2011 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jun 14 15:25:39 2011 -0700" }, "message": "rcu: Use softirq to address performance regression\n\nCommit a26ac2455ffcf3(rcu: move TREE_RCU from softirq to kthread)\nintroduced performance regression. In an AIM7 test, this commit degraded\nperformance by about 40%.\n\nThe commit runs rcu callbacks in a kthread instead of softirq. We observed\nhigh rate of context switch which is caused by this. Out test system has\n64 CPUs and HZ is 1000, so we saw more than 64k context switch per second\nwhich is caused by RCU\u0027s per-CPU kthread. A trace showed that most of\nthe time the RCU per-CPU kthread doesn\u0027t actually handle any callbacks,\nbut instead just does a very small amount of work handling grace periods.\nThis means that RCU\u0027s per-CPU kthreads are making the scheduler do quite\na bit of work in order to allow a very small amount of RCU-related\nprocessing to be done.\n\nAlex Shi\u0027s analysis determined that this slowdown is due to lock\ncontention within the scheduler. Unfortunately, as Peter Zijlstra points\nout, the scheduler\u0027s real-time semantics require global action, which\nmeans that this contention is inherent in real-time scheduling. (Yes,\nperhaps someone will come up with a workaround -- otherwise, -rt is not\ngoing to do well on large SMP systems -- but this patch will work around\nthis issue in the meantime. And \"the meantime\" might well be forever.)\n\nThis patch therefore re-introduces softirq processing to RCU, but only\nfor core RCU work. RCU callbacks are still executed in kthread context,\nso that only a small amount of RCU work runs in softirq context in the\ncommon case. This should minimize ksoftirqd execution, allowing us to\nskip boosting of ksoftirqd for CONFIG_RCU_BOOST\u003dy kernels.\n\nSigned-off-by: Shaohua Li \u003cshaohua.li@intel.com\u003e\nTested-by: \"Alex,Shi\" \u003calex.shi@intel.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "9a432736904d386cda28b987b38ba14dae960ecc", "tree": "20294203f9c6132d79737e9ab54c746546afe7d4", "parents": [ "59c5f46fbe01a00eedf54a23789634438bb80603" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 30 20:38:55 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jun 14 15:25:15 2011 -0700" }, "message": "rcu: Simplify curing of load woes\n\nMake the functions creating the kthreads wake them up. Leverage the\nfact that the per-node and boost kthreads can run anywhere, thus\ndispensing with the need to wake them up once the incoming CPU has\ngone fully online.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Daniel J Blueman \u003cdaniel.blueman@gmail.com\u003e\n" }, { "commit": "d72bce0e67e8afc6eb959f656013cbb577426f1e", "tree": "9c93d4df9aa895d6f2f555e0cf50e7ae5ebaded4", "parents": [ "55922c9d1b84b89cb946c777fddccb3247e7df2c" ], "author": { "name": "Peter Zijlstra", "email": "peterz@infradead.org", "time": "Mon May 30 13:34:51 2011 +0200" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Tue May 31 10:01:48 2011 +0200" }, "message": "rcu: Cure load woes\n\nCommit cc3ce5176d83 (rcu: Start RCU kthreads in TASK_INTERRUPTIBLE\nstate) fudges a sleeping task\u0027 state, resulting in the scheduler seeing\na TASK_UNINTERRUPTIBLE task going to sleep, but a TASK_INTERRUPTIBLE\ntask waking up. The result is unbalanced load calculation.\n\nThe problem that patch tried to address is that the RCU threads could\nstay in UNINTERRUPTIBLE state for quite a while and triggering the hung\ntask detector due to on-demand wake-ups.\n\nCure the problem differently by always giving the tasks at least one\nwake-up once the CPU is fully up and running, this will kick them out of\nthe initial UNINTERRUPTIBLE state and into the regular INTERRUPTIBLE\nwait state.\n\n[ The alternative would be teaching kthread_create() to start threads as\n INTERRUPTIBLE but that needs a tad more thought. ]\n\nReported-by: Damien Wyart \u003cdamien.wyart@free.fr\u003e\nSigned-off-by: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nAcked-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nLink: http://lkml.kernel.org/r/1306755291.1200.2872.camel@twins\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "cc3ce5176d83cd8ae1134f86e208ea758d6cb78e", "tree": "a28ada4c1e5fd8c3e8210ca1a1df950af8a704ff", "parents": [ "08bca60a6912ad225254250c0a9c3a05b4152cfa" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed May 25 13:42:06 2011 -0700" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat May 28 17:41:56 2011 +0200" }, "message": "rcu: Start RCU kthreads in TASK_INTERRUPTIBLE state\n\nUpon creation, kthreads are in TASK_UNINTERRUPTIBLE state, which can\nresult in softlockup warnings. Because some of RCU\u0027s kthreads can\nlegitimately be idle indefinitely, start them in TASK_INTERRUPTIBLE\nstate in order to avoid those warnings.\n\nSuggested-by: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Yinghai Lu \u003cyinghai@kernel.org\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "08bca60a6912ad225254250c0a9c3a05b4152cfa", "tree": "760b20e6eaaa02412fcecb6bc5c3598b6bc0fdce", "parents": [ "8826f3b0397562eee6f8785d548be9dfdb169100" ], "author": { "name": "Peter Zijlstra", "email": "a.p.zijlstra@chello.nl", "time": "Fri May 20 16:06:29 2011 -0700" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat May 28 17:41:52 2011 +0200" }, "message": "rcu: Remove waitqueue usage for cpu, node, and boost kthreads\n\nIt is not necessary to use waitqueues for the RCU kthreads because\nwe always know exactly which thread is to be awakened. In addition,\nwake_up() only issues an actual wakeup when there is a thread waiting on\nthe queue, which was why there was an extra explicit wake_up_process()\nto get the RCU kthreads started.\n\nEliminating the waitqueues (and wake_up()) in favor of wake_up_process()\neliminates the need for the initial wake_up_process() and also shrinks\nthe data structure size a bit. The wakeup logic is placed in a new\nrcu_wait() macro.\n\nSigned-off-by: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "23b5c8fa01b723c70a20d6e4ef4ff54c7656d6e1", "tree": "d03faad5e19848b35a019793b9a1cbc0bb68a708", "parents": [ "4305ce7894dd38b0633bfc8978437320119223bd" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Sep 07 10:38:22 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 09:42:23 2011 -0700" }, "message": "rcu: Decrease memory-barrier usage based on semi-formal proof\n\n(Note: this was reverted, and is now being re-applied in pieces, with\nthis being the fifth and final piece. See below for the reason that\nit is now felt to be safe to re-apply this.)\n\nCommit d09b62d fixed grace-period synchronization, but left some smp_mb()\ninvocations in rcu_process_callbacks() that are no longer needed, but\nsheer paranoia prevented them from being removed. This commit removes\nthem and provides a proof of correctness in their absence. It also adds\na memory barrier to rcu_report_qs_rsp() immediately before the update to\nrsp-\u003ecompleted in order to handle the theoretical possibility that the\ncompiler or CPU might move massive quantities of code into a lock-based\ncritical section. This also proves that the sheer paranoia was not\nentirely unjustified, at least from a theoretical point of view.\n\nIn addition, the old dyntick-idle synchronization depended on the fact\nthat grace periods were many milliseconds in duration, so that it could\nbe assumed that no dyntick-idle CPU could reorder a memory reference\nacross an entire grace period. Unfortunately for this design, the\naddition of expedited grace periods breaks this assumption, which has\nthe unfortunate side-effect of requiring atomic operations in the\nfunctions that track dyntick-idle state for RCU. (There is some hope\nthat the algorithms used in user-level RCU might be applied here, but\nsome work is required to handle the NMIs that user-space applications\ncan happily ignore. For the short term, better safe than sorry.)\n\nThis proof assumes that neither compiler nor CPU will allow a lock\nacquisition and release to be reordered, as doing so can result in\ndeadlock. The proof is as follows:\n\n1.\tA given CPU declares a quiescent state under the protection of\n\tits leaf rcu_node\u0027s lock.\n\n2.\tIf there is more than one level of rcu_node hierarchy, the\n\tlast CPU to declare a quiescent state will also acquire the\n\t-\u003elock of the next rcu_node up in the hierarchy, but only\n\tafter releasing the lower level\u0027s lock. The acquisition of this\n\tlock clearly cannot occur prior to the acquisition of the leaf\n\tnode\u0027s lock.\n\n3.\tStep 2 repeats until we reach the root rcu_node structure.\n\tPlease note again that only one lock is held at a time through\n\tthis process. The acquisition of the root rcu_node\u0027s -\u003elock\n\tmust occur after the release of that of the leaf rcu_node.\n\n4.\tAt this point, we set the -\u003ecompleted field in the rcu_state\n\tstructure in rcu_report_qs_rsp(). However, if the rcu_node\n\thierarchy contains only one rcu_node, then in theory the code\n\tpreceding the quiescent state could leak into the critical\n\tsection. We therefore precede the update of -\u003ecompleted with a\n\tmemory barrier. All CPUs will therefore agree that any updates\n\tpreceding any report of a quiescent state will have happened\n\tbefore the update of -\u003ecompleted.\n\n5.\tRegardless of whether a new grace period is needed, rcu_start_gp()\n\twill propagate the new value of -\u003ecompleted to all of the leaf\n\trcu_node structures, under the protection of each rcu_node\u0027s -\u003elock.\n\tIf a new grace period is needed immediately, this propagation\n\twill occur in the same critical section that -\u003ecompleted was\n\tset in, but courtesy of the memory barrier in #4 above, is still\n\tseen to follow any pre-quiescent-state activity.\n\n6.\tWhen a given CPU invokes __rcu_process_gp_end(), it becomes\n\taware of the end of the old grace period and therefore makes\n\tany RCU callbacks that were waiting on that grace period eligible\n\tfor invocation.\n\n\tIf this CPU is the same one that detected the end of the grace\n\tperiod, and if there is but a single rcu_node in the hierarchy,\n\twe will still be in the single critical section. In this case,\n\tthe memory barrier in step #4 guarantees that all callbacks will\n\tbe seen to execute after each CPU\u0027s quiescent state.\n\n\tOn the other hand, if this is a different CPU, it will acquire\n\tthe leaf rcu_node\u0027s -\u003elock, and will again be serialized after\n\teach CPU\u0027s quiescent state for the old grace period.\n\nOn the strength of this proof, this commit therefore removes the memory\nbarriers from rcu_process_callbacks() and adds one to rcu_report_qs_rsp().\nThe effect is to reduce the number of memory barriers by one and to\nreduce the frequency of execution from about once per scheduling tick\nper CPU to once per grace period.\n\nThis was reverted do to hangs found during testing by Yinghai Lu and\nIngo Molnar. Frederic Weisbecker supplied Yinghai with tracing that\nlocated the underlying problem, and Frederic also provided the fix.\n\nThe underlying problem was that the HARDIRQ_ENTER() macro from\nlib/locking-selftest.c invoked irq_enter(), which in turn invokes\nrcu_irq_enter(), but HARDIRQ_EXIT() invoked __irq_exit(), which\ndoes not invoke rcu_irq_exit(). This situation resulted in calls\nto rcu_irq_enter() that were not balanced by the required calls to\nrcu_irq_exit(). Therefore, after these locking selftests completed,\nRCU\u0027s dyntick-idle nesting count was a large number (for example,\n72), which caused RCU to to conclude that the affected CPU was not in\ndyntick-idle mode when in fact it was.\n\nRCU would therefore incorrectly wait for this dyntick-idle CPU, resulting\nin hangs.\n\nIn contrast, with Frederic\u0027s patch, which replaces the irq_enter()\nin HARDIRQ_ENTER() with an __irq_enter(), these tests don\u0027t ever call\neither rcu_irq_enter() or rcu_irq_exit(), which works because the CPU\nrunning the test is already marked as not being in dyntick-idle mode.\nThis means that the rcu_irq_enter() and rcu_irq_exit() calls and RCU\nthen has no problem working out which CPUs are in dyntick-idle mode and\nwhich are not.\n\nThe reason that the imbalance was not noticed before the barrier patch\nwas applied is that the old implementation of rcu_enter_nohz() ignored\nthe nesting depth. This could still result in delays, but much shorter\nones. Whenever there was a delay, RCU would IPI the CPU with the\nunbalanced nesting level, which would eventually result in rcu_enter_nohz()\nbeing called, which in turn would force RCU to see that the CPU was in\ndyntick-idle mode.\n\nThe reason that very few people noticed the problem is that the mismatched\nirq_enter() vs. __irq_exit() occured only when the kernel was built with\nCONFIG_DEBUG_LOCKING_API_SELFTESTS.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "80d02085d99039b3b7f3a73c8896226b0cb1ba07", "tree": "c310902423ecb00effadcb59c60cbf118d4037cb", "parents": [ "11c476f31a0fabc6e604da5b09a6590b57c3fb20" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 12 01:08:07 2011 -0700" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu May 19 23:25:29 2011 +0200" }, "message": "Revert \"rcu: Decrease memory-barrier usage based on semi-formal proof\"\n\nThis reverts commit e59fb3120becfb36b22ddb8bd27d065d3cdca499.\n\nThis reversion was due to (extreme) boot-time slowdowns on SPARC seen by\nYinghai Lu and on x86 by Ingo\n.\nThis is a non-trivial reversion due to intervening commits.\n\nConflicts:\n\n\tDocumentation/RCU/trace.txt\n\tkernel/rcutree.c\n\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "1217ed1ba5c67393293dfb0f03c353b118dadeb4", "tree": "a765356c8418e134de85fd05d9fe6eda41de859c", "parents": [ "29ce831000081dd757d3116bf774aafffc4b6b20" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed May 04 21:43:49 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat May 07 22:50:45 2011 -0700" }, "message": "rcu: permit rcu_read_unlock() to be called while holding runqueue locks\n\nAvoid calling into the scheduler while holding core RCU locks. This\nallows rcu_read_unlock() to be called while holding the runqueue locks,\nbut only as long as there was no chance of the RCU read-side critical\nsection having been preempted. (Otherwise, if RCU priority boosting\nis enabled, rcu_read_unlock() might call into the scheduler in order to\nunboost itself, which might allows self-deadlock on the runqueue locks\nwithin the scheduler.)\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "6cc68793e380bb51f447d8d02af873b7bc01f222", "tree": "f2b66ace23acc0a31a08f136ad25659d6b51c4b1", "parents": [ "13491a0ee1ef862b6c842132b6eb9c5e721af5ad" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Mar 02 13:15:15 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:59 2011 -0700" }, "message": "rcu: fix spelling\n\nThe \"preemptible\" spelling is preferable. May as well fix it.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "13491a0ee1ef862b6c842132b6eb9c5e721af5ad", "tree": "198d4dc1d56c4d6357d93522cc2ff3676cb78021", "parents": [ "7e8b4c72344e0d904b0e3fa9fd2eb116f04b3d41" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Fri Feb 25 11:37:59 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:58 2011 -0700" }, "message": "rcu: call __rcu_read_unlock() in exit_rcu for tree RCU\n\nUsing __rcu_read_lock() in place of rcu_read_lock() leaves any debug\nstate as it really should be, namely with the lock still held.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "a9f4793d8900dc5dc09b3951bdcd4731290e06fe", "tree": "8e3247ed961e8255b4d7e1c1b4dd13ffb361d4d4", "parents": [ "90e6ac3657fd3b0446d585082000af3cf46439a7" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon May 02 03:46:10 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:56 2011 -0700" }, "message": "rcu: fix tracing bug thinko on boost-balk attribution\n\nThe rcu_initiate_boost_trace() function mis-attributed refusals to\ninitiate RCU priority boosting that were in fact due to its not yet\nbeing time to boost. This patch fixes the faulty comparison.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "d71df90eadfc35aa549ff9a850842673febca71f", "tree": "dd275a3f4848784bc6625574341c11befc1b3823", "parents": [ "0ac3d136b2e3cdf1161178223bc5da14a06241d0" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Mar 29 17:48:28 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:56 2011 -0700" }, "message": "rcu: add tracing for RCU\u0027s kthread run states.\n\nAdd tracing to help debugging situations when RCU\u0027s kthreads are not\nrunning but are supposed to be.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "0ea1f2ebeb217d38770aebf91c4ecaa8e01b3305", "tree": "ef558d2eb804ca033a5166d24f5b4be7cf87727f", "parents": [ "67b98dba474f293c389fc2b7254dcf7c0492e3bd" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Feb 22 13:42:43 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:55 2011 -0700" }, "message": "rcu: Add boosting to TREE_PREEMPT_RCU tracing\n\nIncludes total number of tasks boosted, number boosted on behalf of each\nof normal and expedited grace periods, and statistics on attempts to\ninitiate boosting that failed for various reasons.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "0f962a5e7277c34987b77dc82fc9aefcedc95e27", "tree": "b265d93dcc8f2669f8fb9b353344b9f6cc354728", "parents": [ "27f4d28057adf98750cf863c40baefb12f5b6d21" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Apr 14 12:13:53 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:55 2011 -0700" }, "message": "rcu: Force per-rcu_node kthreads off of the outgoing CPU\n\nThe scheduler has had some heartburn in the past when too many real-time\nkthreads were affinitied to the outgoing CPU. So, this commit lightens\nthe load by forcing the per-rcu_node and the boost kthreads off of the\noutgoing CPU. Note that RCU\u0027s per-CPU kthread remains on the outgoing\nCPU until the bitter end, as it must in order to preserve correctness.\n\nAlso avoid disabling hardirqs across calls to set_cpus_allowed_ptr(),\ngiven that this function can block.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "27f4d28057adf98750cf863c40baefb12f5b6d21", "tree": "a66584d2cb68cebda327313f78b8f4eb602ed4ef", "parents": [ "a26ac2455ffcf3be5c6ef92bc6df7182700f2114" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Feb 07 12:47:15 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:55 2011 -0700" }, "message": "rcu: priority boosting for TREE_PREEMPT_RCU\n\nAdd priority boosting for TREE_PREEMPT_RCU, similar to that for\nTINY_PREEMPT_RCU. This is enabled by the default-off RCU_BOOST\nkernel parameter. The priority to which to boost preempted\nRCU readers is controlled by the RCU_BOOST_PRIO kernel parameter\n(defaulting to real-time priority 1) and the time to wait before\nboosting the readers who are blocking a given grace period is\ncontrolled by the RCU_BOOST_DELAY kernel parameter (defaulting to\n500 milliseconds).\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "a26ac2455ffcf3be5c6ef92bc6df7182700f2114", "tree": "601697c7c3fd152a1b3f29f43d3a028fc8fefd5f", "parents": [ "12f5f524cafef3ab689929b118f2dfb8bf2be321" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Jan 12 14:10:23 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: move TREE_RCU from softirq to kthread\n\nIf RCU priority boosting is to be meaningful, callback invocation must\nbe boosted in addition to preempted RCU readers. Otherwise, in presence\nof CPU real-time threads, the grace period ends, but the callbacks don\u0027t\nget invoked. If the callbacks don\u0027t get invoked, the associated memory\ndoesn\u0027t get freed, so the system is still subject to OOM.\n\nBut it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit\nmoves the callback invocations to a kthread, which can be boosted easily.\n\nAlso add comments and properly synchronized all accesses to\nrcu_cpu_kthread_task, as suggested by Lai Jiangshan.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "12f5f524cafef3ab689929b118f2dfb8bf2be321", "tree": "639473556b6edf9b79e0a18d5ba58f80eea76519", "parents": [ "e59fb3120becfb36b22ddb8bd27d065d3cdca499" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Nov 29 21:56:39 2010 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: merge TREE_PREEPT_RCU blocked_tasks[] lists\n\nCombine the current TREE_PREEMPT_RCU -\u003eblocked_tasks[] lists in the\nrcu_node structure into a single -\u003eblkd_tasks list with -\u003egp_tasks\nand -\u003eexp_tasks tail pointers. This is in preparation for RCU priority\nboosting, which will add a third dimension to the combinatorial explosion\nin the -\u003eblocked_tasks[] case, but simply a third pointer in the new\n-\u003eblkd_tasks case.\n\nAlso update documentation to reflect blocked_tasks[] merge\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "e59fb3120becfb36b22ddb8bd27d065d3cdca499", "tree": "37eaadfe112b64caae943fc7469274bc96553d92", "parents": [ "a00e0d714fbded07a7a2254391ce9ed5a5cb9d82" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Sep 07 10:38:22 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: Decrease memory-barrier usage based on semi-formal proof\n\nCommit d09b62d fixed grace-period synchronization, but left some smp_mb()\ninvocations in rcu_process_callbacks() that are no longer needed, but\nsheer paranoia prevented them from being removed. This commit removes\nthem and provides a proof of correctness in their absence. It also adds\na memory barrier to rcu_report_qs_rsp() immediately before the update to\nrsp-\u003ecompleted in order to handle the theoretical possibility that the\ncompiler or CPU might move massive quantities of code into a lock-based\ncritical section. This also proves that the sheer paranoia was not\nentirely unjustified, at least from a theoretical point of view.\n\nIn addition, the old dyntick-idle synchronization depended on the fact\nthat grace periods were many milliseconds in duration, so that it could\nbe assumed that no dyntick-idle CPU could reorder a memory reference\nacross an entire grace period. Unfortunately for this design, the\naddition of expedited grace periods breaks this assumption, which has\nthe unfortunate side-effect of requiring atomic operations in the\nfunctions that track dyntick-idle state for RCU. (There is some hope\nthat the algorithms used in user-level RCU might be applied here, but\nsome work is required to handle the NMIs that user-space applications\ncan happily ignore. For the short term, better safe than sorry.)\n\nThis proof assumes that neither compiler nor CPU will allow a lock\nacquisition and release to be reordered, as doing so can result in\ndeadlock. The proof is as follows:\n\n1.\tA given CPU declares a quiescent state under the protection of\n\tits leaf rcu_node\u0027s lock.\n\n2.\tIf there is more than one level of rcu_node hierarchy, the\n\tlast CPU to declare a quiescent state will also acquire the\n\t-\u003elock of the next rcu_node up in the hierarchy, but only\n\tafter releasing the lower level\u0027s lock. The acquisition of this\n\tlock clearly cannot occur prior to the acquisition of the leaf\n\tnode\u0027s lock.\n\n3.\tStep 2 repeats until we reach the root rcu_node structure.\n\tPlease note again that only one lock is held at a time through\n\tthis process. The acquisition of the root rcu_node\u0027s -\u003elock\n\tmust occur after the release of that of the leaf rcu_node.\n\n4.\tAt this point, we set the -\u003ecompleted field in the rcu_state\n\tstructure in rcu_report_qs_rsp(). However, if the rcu_node\n\thierarchy contains only one rcu_node, then in theory the code\n\tpreceding the quiescent state could leak into the critical\n\tsection. We therefore precede the update of -\u003ecompleted with a\n\tmemory barrier. All CPUs will therefore agree that any updates\n\tpreceding any report of a quiescent state will have happened\n\tbefore the update of -\u003ecompleted.\n\n5.\tRegardless of whether a new grace period is needed, rcu_start_gp()\n\twill propagate the new value of -\u003ecompleted to all of the leaf\n\trcu_node structures, under the protection of each rcu_node\u0027s -\u003elock.\n\tIf a new grace period is needed immediately, this propagation\n\twill occur in the same critical section that -\u003ecompleted was\n\tset in, but courtesy of the memory barrier in #4 above, is still\n\tseen to follow any pre-quiescent-state activity.\n\n6.\tWhen a given CPU invokes __rcu_process_gp_end(), it becomes\n\taware of the end of the old grace period and therefore makes\n\tany RCU callbacks that were waiting on that grace period eligible\n\tfor invocation.\n\n\tIf this CPU is the same one that detected the end of the grace\n\tperiod, and if there is but a single rcu_node in the hierarchy,\n\twe will still be in the single critical section. In this case,\n\tthe memory barrier in step #4 guarantees that all callbacks will\n\tbe seen to execute after each CPU\u0027s quiescent state.\n\n\tOn the other hand, if this is a different CPU, it will acquire\n\tthe leaf rcu_node\u0027s -\u003elock, and will again be serialized after\n\teach CPU\u0027s quiescent state for the old grace period.\n\nOn the strength of this proof, this commit therefore removes the memory\nbarriers from rcu_process_callbacks() and adds one to rcu_report_qs_rsp().\nThe effect is to reduce the number of memory barriers by one and to\nreduce the frequency of execution from about once per scheduling tick\nper CPU to once per grace period.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "a00e0d714fbded07a7a2254391ce9ed5a5cb9d82", "tree": "7c213dbf1e2561655d07749b6d689027460fb8cc", "parents": [ "0ee5623f9a6e52df90a78bd21179f8ab370e102e" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 08 17:14:39 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: Remove conditional compilation for RCU CPU stall warnings\n\nThe RCU CPU stall warnings can now be controlled using the\nrcu_cpu_stall_suppress boot-time parameter or via the same parameter\nfrom sysfs. There is therefore no longer any reason to have\nkernel config parameters for this feature. This commit therefore\nremoves the RCU_CPU_STALL_DETECTOR and RCU_CPU_STALL_DETECTOR_RUNNABLE\nkernel config parameters. The RCU_CPU_STALL_TIMEOUT parameter remains\nto allow the timeout to be tuned and the RCU_CPU_STALL_VERBOSE parameter\nremains to allow task-stall information to be suppressed if desired.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "e27fc9641e8ddc8146f8e01f06e5eba2469698de", "tree": "5cff4c23cb113b901718d9c0f4a6e3081f3173bd", "parents": [ "46fdb0937f26124700fc9fc80da4776330cc00d3" ], "author": { "name": "Tejun Heo", "email": "tj@kernel.org", "time": "Mon Nov 22 21:36:11 2010 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 17 12:34:08 2010 -0800" }, "message": "rcu: increase synchronize_sched_expedited() batching\n\nThe fix in commit #6a0cc49 requires more than three concurrent instances\nof synchronize_sched_expedited() before batching is possible. This\npatch uses a ticket-counter-like approach that is also not unrelated to\nLai Jiangshan\u0027s Ring RCU to allow sharing of expedited grace periods even\nwhen there are only two concurrent instances of synchronize_sched_expedited().\n\nThis commit builds on Tejun\u0027s original posting, which may be found at\nhttp://lkml.org/lkml/2010/11/9/204, adding memory barriers, avoiding\noverflow of signed integers (other than via atomic_t), and fixing the\ndetection of batching.\n\nSigned-off-by: Tejun Heo \u003ctj@kernel.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "db3a8920995484e5e9a0abaf3bad2c7311b163db", "tree": "e183de92a908466dd5568df151507b2331ff29a5", "parents": [ "2d999e03b7c8305b4385dd20992e4ed3e827177b" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Oct 25 07:39:22 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Nov 29 22:02:00 2010 -0800" }, "message": "rcu: fix race condition in synchronize_sched_expedited()\n\nThe new (early 2010) implementation of synchronize_sched_expedited() uses\ntry_stop_cpu() to force a context switch on every CPU. It also permits\nconcurrent calls to synchronize_sched_expedited() to share a single call\nto try_stop_cpu() through use of an atomically incremented\nsynchronize_sched_expedited_count variable. Unfortunately, this is\nsubject to failure as follows:\n\no\tTask A invokes synchronize_sched_expedited(), try_stop_cpus()\n\tsucceeds, but Task A is preempted before getting to the atomic\n\tincrement of synchronize_sched_expedited_count.\n\no\tTask B also invokes synchronize_sched_expedited(), with exactly\n\tthe same outcome as Task A.\n\no\tTask C also invokes synchronize_sched_expedited(), again with\n\texactly the same outcome as Tasks A and B.\n\no\tTask D also invokes synchronize_sched_expedited(), but only\n\tgets as far as acquiring the mutex within try_stop_cpus()\n\tbefore being preempted, interrupted, or otherwise delayed.\n\no\tTask E also invokes synchronize_sched_expedited(), but only\n\tgets to the snapshotting of synchronize_sched_expedited_count.\n\no\tTasks A, B, and C all increment synchronize_sched_expedited_count.\n\no\tTask E fails to get the mutex, so checks the new value\n\tof synchronize_sched_expedited_count. It finds that the\n\tvalue has increased, so (wrongly) assumes that its work\n\thas been done, returning despite there having been no\n\texpedited grace period since it began.\n\nThe solution is to have the lowest-numbered CPU atomically increment\nthe synchronize_sched_expedited_count variable within the\nsynchronize_sched_expedited_cpu_stop() function, which is under\nthe protection of the mutex acquired by try_stop_cpus(). However, this\nalso requires that piggybacking tasks wait for three rather than two\ninstances of try_stop_cpu(), because we cannot control the order in\nwhich the per-CPU callback function occur.\n\nCc: Tejun Heo \u003ctj@kernel.org\u003e\nCc: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "2d999e03b7c8305b4385dd20992e4ed3e827177b", "tree": "3402eeb03a6cc7ac8013974983f09e46f5ecd562", "parents": [ "29494be71afe2a16ad04e344306a620d7cc22d06" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Oct 20 12:06:18 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Nov 29 22:01:59 2010 -0800" }, "message": "rcu: update documentation/comments for Lai\u0027s adoption patch\n\nLai\u0027s RCU-callback immediate-adoption patch changes the RCU tracing\noutput, so update tracing.txt. Also update a few comments to clarify\nthe synchronization design.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "29494be71afe2a16ad04e344306a620d7cc22d06", "tree": "d16a2acd1566be326483d5217d39b78d2ef798c5", "parents": [ "7b27d5475f86186914e54e4a6bb994e9a985337b" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Wed Oct 20 14:13:06 2010 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Nov 29 22:01:58 2010 -0800" }, "message": "rcu,cleanup: simplify the code when cpu is dying\n\nWhen we handle the CPU_DYING notifier, the whole system is stopped except\nfor the current CPU. We therefore need no synchronization with the other\nCPUs. This allows us to move any orphaned RCU callbacks directly to the\nlist of any online CPU without needing to run them through the global\norphan lists. These global orphan lists can therefore be dispensed with.\nThis commit makes thes changes, though currently victimizes CPU 0 @@@.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "7b27d5475f86186914e54e4a6bb994e9a985337b", "tree": "36349c88d75497db89956d6721d82c4f889cbc2b", "parents": [ "deb7a41815a8a32d4f9ea2af7a48ed1175222cec" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Thu Oct 21 11:29:05 2010 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Nov 29 22:01:58 2010 -0800" }, "message": "rcu,cleanup: move synchronize_sched_expedited() out of sched.c\n\nThe first version of synchronize_sched_expedited() used the migration\ncode in the scheduler, and was therefore implemented in kernel/sched.c.\nHowever, the more recent version of this code no longer uses the\nmigration code, so this commit moves it to the main RCU source files.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "81a294c44e973dc7182e4733421b7cb2daba3c29", "tree": "e76cbbd289fcdc04eff921c01d4dc4b767d6288c", "parents": [ "dd7c4d89730a1be2c1d361a8ae1f0fe9465ccf9c" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Aug 30 09:52:50 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Sep 02 16:15:30 2010 -0700" }, "message": "rcu: fix _oddness handling of verbose stall warnings\n\nCONFIG_RCU_CPU_STALL_VERBOSE depends on CONFIG_TREE_PREEMPT_RCU, but\nrcu_bootup_announce_oddness() complains if CONFIG_RCU_CPU_STALL_VERBOSE\nis not set even in the case of CONFIG_TREE_RCU. This commit therefore\nfixes rcu_bootup_announce_oddness() to avoid insisting on impossibilities.\n\nReported-by: Guy Martin \u003cgmsoft@tuxicoman.be\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "80dcf60e6b97c7363971e7a0a788d8484d35f8a6", "tree": "2f493fdf6a7a52675733ac74daa9e1cd9342a030", "parents": [ "7b0b759b65247cbc66384a912be9acf8d4800636" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Aug 19 16:57:45 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Aug 20 09:00:17 2010 -0700" }, "message": "rcu: apply TINY_PREEMPT_RCU read-side speedup to TREE_PREEMPT_RCU\n\nReplace one of the ACCESS_ONCE() calls in each of __rcu_read_lock()\nand __rcu_read_unlock() with barrier() as suggested by Steve Rostedt in\norder to avoid the potential compiler-optimization-induced bug noted by\nMathieu Desnoyers.\n\nLocated-by: Mathieu Desnoyers \u003cmathieu.desnoyers@polymtl.ca\u003e\nSuggested-by: Steven Rostedt \u003crostedt@goodmis.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "7b0b759b65247cbc66384a912be9acf8d4800636", "tree": "085a18f506193725ab16fa61cae41f93dcc7c3e9", "parents": [ "73d4da4d360136826b36f78f5cf72b29da82c8a6" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Aug 17 14:18:46 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Aug 20 09:00:16 2010 -0700" }, "message": "rcu: combine duplicate code, courtesy of CONFIG_PREEMPT_RCU\n\nThe CONFIG_PREEMPT_RCU kernel configuration parameter was recently\nre-introduced, but as an indication of the type of RCU (preemptible\nvs. non-preemptible) instead of as selecting a given implementation.\nThis commit uses CONFIG_PREEMPT_RCU to combine duplicate code\nfrom include/linux/rcutiny.h and include/linux/rcutree.h into\ninclude/linux/rcupdate.h. This commit also combines a few other pieces\nof duplicate code that have accumulated.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "53d84e004d5e8c018be395c4330dc72fd60bd13e", "tree": "6d5b265442ac42c7cc73147b0de0d029644e312b", "parents": [ "8cdd32a918350430483751feaae1c19cef816f69" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Aug 10 14:28:53 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Aug 20 09:00:12 2010 -0700" }, "message": "rcu: permit suppressing current grace period\u0027s CPU stall warnings\n\nWhen using a kernel debugger, a long sojourn in the debugger can get\nyou lots of RCU CPU stall warnings once you resume. This might not be\nhelpful, especially if you are using the system console. This patch\ntherefore allows RCU CPU stall warnings to be suppressed, but only for\nthe duration of the current set of grace periods.\n\nThis differs from Jason\u0027s original patch in that it adds support for\ntiny RCU and preemptible RCU, and uses a slightly different method for\nsuppressing the RCU CPU stall warning messages.\n\nSigned-off-by: Jason Wessel \u003cjason.wessel@windriver.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Jason Wessel \u003cjason.wessel@windriver.com\u003e\n" }, { "commit": "77d8485a8b5416c615b6acd95f01bfcacd7d81ff", "tree": "725742c078b9712d875e9c633d249b6da14e5f00", "parents": [ "742734eea0cf5314cde5945963ed964be167bd84" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Jul 08 17:38:59 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Aug 19 17:18:02 2010 -0700" }, "message": "rcu: improve kerneldoc for rcu_read_lock(), call_rcu(), and synchronize_rcu()\n\nMake it explicit that new RCU read-side critical sections that start\nafter call_rcu() and synchronize_rcu() start might still be running\nafter the end of the relevant grace period.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "394f99a9007d4274f7076bb8553ab0ff9707688b", "tree": "7d379f91321cec58b87fd5f5089947872689d591", "parents": [ "e546f485e1d7520ca0200502cdcc11b503f4805c" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Mon Jun 28 16:25:04 2010 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Aug 19 17:18:01 2010 -0700" }, "message": "rcu: simplify the usage of percpu data\n\n\u0026percpu_data is compatible with allocated percpu data.\n\nAnd we use it and remove the \"-\u003erda[NR_CPUS]\" array, saving significant\nstorage on systems with large numbers of CPUs. This does add an additional\nlevel of indirection and thus an additional cache line referenced, but\nbecause -\u003erda is not used on the read side, this is OK.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nReviewed-by: Tejun Heo \u003ctj@kernel.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "72d5a9f7a9542f88397558c65bcfc3b115a65e34", "tree": "e46d3ad1948de45b9641d1a8b88f88bfb3809007", "parents": [ "4376030a54860dedab9d848dfa7cc700a6025c0b" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 17:12:17 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue May 11 16:10:47 2010 -0700" }, "message": "rcu: remove all rcu head initializations, except on_stack initializations\n\nRemove all rcu head inits. We don\u0027t care about the RCU head state before passing\nit to call_rcu() anyway. Only leave the \"on_stack\" variants so debugobjects can\nkeep track of objects on stack.\n\nSigned-off-by: Mathieu Desnoyers \u003cmathieu.desnoyers@efficios.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "d822ed1094032ab524344a9a474c93128d9c2159", "tree": "36f4a24c7501c799a071384e4450a823b387d18b", "parents": [ "77e38ed347162423c6b72e06c865a121081c2bb6" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat May 08 19:58:22 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:35 2010 -0700" }, "message": "rcu: fix build bug in RCU_FAST_NO_HZ builds\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "77e38ed347162423c6b72e06c865a121081c2bb6", "tree": "328b746af71b47c4af97e6ea06dc24b12528a955", "parents": [ "d14aada8e20bdf81ffd43f433b123972cf575b32" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Apr 25 21:04:29 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:35 2010 -0700" }, "message": "rcu: RCU_FAST_NO_HZ must check RCU dyntick state\n\nThe current version of RCU_FAST_NO_HZ reproduces the old CLASSIC_RCU\ndyntick-idle bug, as it fails to detect CPUs that have interrupted\nor NMIed out of dyntick-idle mode. Fix this by making rcu_needs_cpu()\ncheck the state in the per-CPU rcu_dynticks variables, thus correctly\ndetecting the dyntick-idle state from an RCU perspective.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "26845c2860cebebe6ce2d9d01ae3cb3db84b7e29", "tree": "83849948ea7bffb5e87eecc17a7275e357740632", "parents": [ "c68de2097a8799549a3c3bf27cbfeea24a604284" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Apr 13 14:19:23 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:34 2010 -0700" }, "message": "rcu: print boot-time console messages if RCU configs out of ordinary\n\nPrint boot-time messages if tracing is enabled, if fanout is set\nto non-default values, if exact fanout is specified, if accelerated\ndyntick-idle grace periods have been enabled, if RCU-lockdep is enabled,\nif rcutorture has been boot-time enabled, if the CPU stall detector has\nbeen disabled, or if four-level hierarchy has been enabled.\n\nThis is all for TREE_RCU and TREE_PREEMPT_RCU. TINY_RCU will be handled\nseparately, if at all.\n\nSuggested-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "25502a6c13745f4650cc59322bd198194f55e796", "tree": "d76cc659d3ea797c5da4630e219ac363d17c44a6", "parents": [ "99652b54de1ee094236f7171485214071af4ef31" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Apr 01 17:37:01 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:33 2010 -0700" }, "message": "rcu: refactor RCU\u0027s context-switch handling\n\nThe addition of preemptible RCU to treercu resulted in a bit of\nconfusion and inefficiency surrounding the handling of context switches\nfor RCU-sched and for RCU-preempt. For RCU-sched, a context switch\nis a quiescent state, pure and simple, just like it always has been.\nFor RCU-preempt, a context switch is in no way a quiescent state, but\nspecial handling is required when a task blocks in an RCU read-side\ncritical section.\n\nHowever, the callout from the scheduler and the outer loop in ksoftirqd\nstill calls something named rcu_sched_qs(), whose name is no longer\naccurate. Furthermore, when rcu_check_callbacks() notes an RCU-sched\nquiescent state, it ends up unnecessarily (though harmlessly, aside\nfrom the performance hit) enqueuing the current task if it happens to\nbe running in an RCU-preempt read-side critical section. This not only\nincreases the maximum latency of scheduler_tick(), it also needlessly\nincreases the overhead of the next outermost rcu_read_unlock() invocation.\n\nThis patch addresses this situation by separating the notion of RCU\u0027s\ncontext-switch handling from that of RCU-sched\u0027s quiescent states.\nThe context-switch handling is covered by rcu_note_context_switch() in\ngeneral and by rcu_preempt_note_context_switch() for preemptible RCU.\nThis permits rcu_sched_qs() to handle quiescent states and only quiescent\nstates. It also reduces the maximum latency of scheduler_tick(), though\nprobably by much less than a microsecond. Finally, it means that tasks\nwithin preemptible-RCU read-side critical sections avoid incurring the\noverhead of queuing unless there really is a context switch.\n\nSuggested-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nAcked-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: Ingo Molnar \u003cmingo@elte.hu\u003e\nCc: Peter Zijlstra \u003cpeterz@infradead.org\u003e\n" }, { "commit": "5db356736acb9ba717df1aa9444e4e44cbb30a71", "tree": "719d734fc3ef0f5fca3a7bf12a00dccfde278864", "parents": [ "0c34029abdfdea64420cb4264c4e91a776b22157" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Tue Mar 30 18:40:36 2010 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:31 2010 -0700" }, "message": "rcu: ignore offline CPUs in last non-dyntick-idle CPU check\n\nOffline CPUs are not in nohz_cpu_mask, but can be ignored when checking\nfor the last non-dyntick-idle CPU. This patch therefore only checks\nonline CPUs for not being dyntick idle, allowing fast entry into\nfull-system dyntick-idle state even when there are some offline CPUs.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "622ea685f1fafdf84d612440535c84341f0860b8", "tree": "68363e9a5a408ac78488c8c19e918fb6b17b76a7", "parents": [ "71da81324c83ef65bb196c7f874ac1c6996d8287" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat Feb 27 14:53:07 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sun Feb 28 09:17:42 2010 +0100" }, "message": "rcu: Fix holdoff for accelerated GPs for last non-dynticked CPU\n\nMake the holdoff only happen when the full number of attempts\nhave been made.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1267311188-16603-1-git-send-email-paulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "71da81324c83ef65bb196c7f874ac1c6996d8287", "tree": "b470c5769ad88d2e2af8fe25cc84e7df85fb3d7c", "parents": [ "0b1c87278a8c7e394022ec184a0b44a3886b6fde" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Feb 26 16:38:58 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat Feb 27 09:53:53 2010 +0100" }, "message": "rcu: Fix accelerated GPs for last non-dynticked CPU\n\nThis patch disables irqs across the call to rcu_needs_cpu(). It\nalso enforces a hold-off period so that the idle loop doesn\u0027t\nsoftirq itself to death when there are lots of RCU callbacks in\nflight on the last non-dynticked CPU.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1267231138-27856-3-git-send-email-paulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "a47cd880b50e14b0b6f5e9d426ae9a2676c9c474", "tree": "c134dcec52450ec92ea853f4aeeef8e3967a36cb", "parents": [ "f5f654096487c6d526c47bb66308f9de81f091cf" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Feb 26 16:38:56 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat Feb 27 09:53:52 2010 +0100" }, "message": "rcu: Fix accelerated grace periods for last non-dynticked CPU\n\nIt is invalid to invoke __rcu_process_callbacks() with irqs\ndisabled, so do it indirectly via raise_softirq(). This\nrequires a state-machine implementation to cycle through the\ngrace-period machinery the required number of times.\n\nLocated-by: Ingo Molnar \u003cmingo@elte.hu\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1267231138-27856-1-git-send-email-paulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "1ed509a225008c9e8c0644fbd22168e09a7383a0", "tree": "db65a58258bb968e7bc922dab14f4f0714a7667e", "parents": [ "6155fec92e85f07d99e9746234496215443ffb0d" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Feb 22 17:05:05 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu Feb 25 10:35:02 2010 +0100" }, "message": "rcu: Add RCU_CPU_STALL_VERBOSE to dump detailed per-task information\n\nWhen RCU detects a grace-period stall, it currently just prints\nout the PID of any tasks doing the stalling. This patch adds\nRCU_CPU_STALL_VERBOSE, which enables the more-verbose reporting\nfrom sched_show_task().\n\nSuggested-by: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1266887105-1528-21-git-send-email-paulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "3acd9eb31c5f7eb97cb2009fa41472710fb4a10f", "tree": "a339775ed9683bdf0f4d7b47a75648a6f86a8446", "parents": [ "1304afb225288a2e250d6a7495462c28e5509cbb" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Feb 22 17:05:03 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu Feb 25 10:34:59 2010 +0100" }, "message": "rcu: Fix deadlock in TREE_PREEMPT_RCU CPU stall detection\n\nUnder TREE_PREEMPT_RCU, print_other_cpu_stall() invokes\nrcu_print_task_stall() with the root rcu_node structure\u0027s -\u003elock\nheld, and rcu_print_task_stall() acquires that same lock for\nself-deadlock. Fix this by removing the lock acquisition from\nrcu_print_task_stall(), and making all callers acquire the lock\ninstead.\n\nTested-by: John Kacur \u003cjkacur@redhat.com\u003e\nTested-by: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nLocated-by: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1266887105-1528-19-git-send-email-paulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "1304afb225288a2e250d6a7495462c28e5509cbb", "tree": "1ba76e304f718f7ce89532e1c4d276b4af439c07", "parents": [ "20133cfce7d0bbdcc0c398301030c091f5675c88" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Feb 22 17:05:02 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu Feb 25 10:34:58 2010 +0100" }, "message": "rcu: Convert to raw_spinlocks\n\nThe spinlocks in rcutree need to be real spinlocks in\npreempt-rt. Convert them to raw_spinlocks.\n\nSigned-off-by: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1266887105-1528-18-git-send-email-paulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "8bd93a2c5d4cab2ae17d06350daa7dbf546a4634", "tree": "3facbdbfbcc1b169fad20f456b0a2521adadfb25", "parents": [ "998f2ac3fea93bfa8b55c279fff68f7c5b9ab93d" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Feb 22 17:04:59 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu Feb 25 10:34:55 2010 +0100" }, "message": "rcu: Accelerate grace period if last non-dynticked CPU\n\nCurrently, rcu_needs_cpu() simply checks whether the current CPU\nhas an outstanding RCU callback, which means that the last CPU\nto go into dyntick-idle mode might wait a few ticks for the\nrelevant grace periods to complete. However, if all the other\nCPUs are in dyntick-idle mode, and if this CPU is in a quiescent\nstate (which it is for RCU-bh and RCU-sched any time that we are\nconsidering going into dyntick-idle mode), then the grace period\nis instantly complete.\n\nThis patch therefore repeatedly invokes the RCU grace-period\nmachinery in order to force any needed grace periods to complete\nquickly. It does so a limited number of times in order to\nprevent starvation by an RCU callback function that might pass\nitself to call_rcu().\n\nHowever, if any CPU other than the current one is not in\ndyntick-idle mode, fall back to simply checking (with fix to bug\nnoted by Lai Jiangshan). Also, take advantage of last\ngrace-period forcing, the opportunity to do so noted by Steve\nRostedt. And apply simplified #ifdef condition suggested by\nFrederic Weisbecker.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1266887105-1528-15-git-send-email-paulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "cba8244a0f1c277b6b1e48ed6504fa434119e24d", "tree": "becfa7d6f653774f72f411002626e791243e5057", "parents": [ "f9094d8e5587cf21091a9516628147c0b55e4264" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Jan 04 16:04:01 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Wed Jan 13 09:06:06 2010 +0100" }, "message": "rcu: Add debug check for too many rcu_read_unlock()\n\nTREE_PREEMPT_RCU maintains an rcu_read_lock_nesting counter in\nthe task structure, which happens to be a signed int. So this\npatch adds a check for this counter being negative at the end of\n__rcu_read_unlock(). This check is under CONFIG_PROVE_LOCKING,\nso can be thought of as being part of lockdep.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c12626498423064-git-send-email-\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "bf66f18e79e34c421bbd8f6511e2c556b779df2f", "tree": "2348e3bc8392addf7959fbe94003f7e0da2a1c8b", "parents": [ "46a1e34eda805501a8b32f26394faa435149f6d1" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Jan 04 15:09:10 2010 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Wed Jan 13 09:06:05 2010 +0100" }, "message": "rcu: Add force_quiescent_state() testing to rcutorture\n\nAdd force_quiescent_state() testing to rcutorture, with a\nseparate thread that repeatedly invokes force_quiescent_state()\nin bursts. This can greatly increase the probability of\nencountering certain types of race conditions.\n\nSuggested-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1262646551116-git-send-email-\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "d9a3da0699b24a589b27a61e1a5b5bd30d9db669", "tree": "f7440e396a6c818f3cef514ccc31ab55d88025ef", "parents": [ "cf244dc01bf68e1ad338b82447f8686d24ea4435" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Dec 02 12:10:15 2009 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu Dec 03 11:35:25 2009 +0100" }, "message": "rcu: Add expedited grace-period support for preemptible RCU\n\nImplement an synchronize_rcu_expedited() for preemptible RCU\nthat actually is expedited. This uses\nsynchronize_sched_expedited() to force all threads currently\nrunning in a preemptible-RCU read-side critical section onto the\nappropriate -\u003eblocked_tasks[] list, then takes a snapshot of all\nof these lists and waits for them to drain.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1259784616158-git-send-email-\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "d3f6bad3911736e44ba11f3f3f6ac4e8c837fdfc", "tree": "55b0c2e11ab3ec02e9065cf3d46ffd20386dfed2", "parents": [ "6ebb237bece23275d1da149b61a342f0d4d06a08" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Dec 02 12:10:13 2009 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu Dec 03 11:34:26 2009 +0100" }, "message": "rcu: Rename \"quiet\" functions\n\nThe number of \"quiet\" functions has grown recently, and the\nnames are no longer very descriptive. The point of all of these\nfunctions is to do some portion of the task of reporting a\nquiescent state, so rename them accordingly:\n\no\tcpu_quiet() becomes rcu_report_qs_rdp(), which reports a\n\tquiescent state to the per-CPU rcu_data structure. If this\n\tturns out to be a new quiescent state for this grace period,\n\tthen rcu_report_qs_rnp() will be invoked to propagate the\n\tquiescent state up the rcu_node hierarchy.\n\no\tcpu_quiet_msk() becomes rcu_report_qs_rnp(), which reports\n\ta quiescent state for a given CPU (or possibly a set of CPUs)\n\tup the rcu_node hierarchy.\n\no\tcpu_quiet_msk_finish() becomes rcu_report_qs_rsp(), which\n\treports a full set of quiescent states to the global rcu_state\n\tstructure.\n\no\ttask_quiet() becomes rcu_report_unblock_qs_rnp(), which reports\n\ta quiescent state due to a task exiting an RCU read-side critical\n\tsection that had previously blocked in that same critical section.\n\tAs indicated by the new name, this type of quiescent state is\n\treported up the rcu_node hierarchy (using rcu_report_qs_rnp()\n\tto do so).\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nAcked-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\nAcked-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c12597846163698-git-send-email-\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "6ebb237bece23275d1da149b61a342f0d4d06a08", "tree": "5f3d99f6ce2e1e98736e5708eecb5bf217f78fde", "parents": [ "9f680ab41485edfdc96331b70afa7513aa0a7720" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Nov 22 08:53:50 2009 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sun Nov 22 18:58:16 2009 +0100" }, "message": "rcu: Re-arrange code to reduce #ifdef pain\n\nRemove #ifdefs from kernel/rcupdate.c and\ninclude/linux/rcupdate.h by moving code to\ninclude/linux/rcutiny.h, include/linux/rcutree.h, and\nkernel/rcutree.c.\n\nAlso remove some definitions that are no longer used.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c1258908830885-git-send-email-\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "b668c9cf3e58739dac54a1d6f42f2b4bdd980b3e", "tree": "c0165d39532a2314f8187d765f3c8ddf88b72831", "parents": [ "2f51f9884f6a36b0fe9636d5a1937e5cbd25723b" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Nov 22 08:53:48 2009 -0800" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sun Nov 22 18:58:15 2009 +0100" }, "message": "rcu: Fix grace-period-stall bug on large systems with CPU hotplug\n\nWhen the last CPU of a given leaf rcu_node structure goes\noffline, all of the tasks queued on that leaf rcu_node structure\n(due to having blocked in their current RCU read-side critical\nsections) are requeued onto the root rcu_node structure. This\nrequeuing is carried out by rcu_preempt_offline_tasks().\nHowever, it is possible that these queued tasks are the only\nthing preventing the leaf rcu_node structure from reporting a\nquiescent state up the rcu_node hierarchy. Unfortunately, the\nold code would fail to do this reporting, resulting in a\ngrace-period stall given the following sequence of events:\n\n1.\tKernel built for more than 32 CPUs on 32-bit systems or for more\n\tthan 64 CPUs on 64-bit systems, so that there is more than one\n\trcu_node structure. (Or CONFIG_RCU_FANOUT is artificially set\n\tto a number smaller than CONFIG_NR_CPUS.)\n\n2.\tThe kernel is built with CONFIG_TREE_PREEMPT_RCU.\n\n3.\tA task running on a CPU associated with a given leaf rcu_node\n\tstructure blocks while in an RCU read-side critical section\n\t-and- that CPU has not yet passed through a quiescent state\n\tfor the current RCU grace period. This will cause the task\n\tto be queued on the leaf rcu_node\u0027s blocked_tasks[] array, in\n\tparticular, on the element of this array corresponding to the\n\tcurrent grace period.\n\n4.\tEach of the remaining CPUs corresponding to this same leaf rcu_node\n\tstructure pass through a quiescent state. However, the task is\n\tstill in its RCU read-side critical section, so these quiescent\n\tstates cannot be reported further up the rcu_node hierarchy.\n\tNevertheless, all bits in the leaf rcu_node structure\u0027s -\u003eqsmask\n\tfield are now zero.\n\n5.\tEach of the remaining CPUs go offline. (The events in step\n\t#4 and #5 can happen in any order as long as each CPU passes\n\tthrough a quiescent state before going offline.)\n\n6.\tWhen the last CPU goes offline, __rcu_offline_cpu() will invoke\n\trcu_preempt_offline_tasks(), which will move the task to the\n\troot rcu_node structure, but without reporting a quiescent state\n\tup the rcu_node hierarchy (and this failure to report a quiescent\n\tstate is the bug).\n\n\tBut because this leaf rcu_node structure\u0027s -\u003eqsmask field is\n\talready zero and its -\u003eblock_tasks[] entries are all empty,\n\tforce_quiescent_state() will skip this rcu_node structure.\n\n\tTherefore, grace periods are now hung.\n\nThis patch abstracts some code out of rcu_read_unlock_special(),\ncalling the result task_quiet() by analogy with cpu_quiet(), and\ninvokes task_quiet() from both rcu_read_lock_special() and\n__rcu_offline_cpu(). Invoking task_quiet() from\n__rcu_offline_cpu() reports the quiescent state up the rcu_node\nhierarchy, fixing the bug. This ends up requiring a separate\nlock_class_key per level of the rcu_node hierarchy, which this\npatch also provides.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: laijs@cn.fujitsu.com\nCc: dipankar@in.ibm.com\nCc: mathieu.desnoyers@polymtl.ca\nCc: josh@joshtriplett.org\nCc: dvhltc@us.ibm.com\nCc: niv@us.ibm.com\nCc: peterz@infradead.org\nCc: rostedt@goodmis.org\nCc: Valdis.Kletnieks@vt.edu\nCc: dhowells@redhat.com\nLKML-Reference: \u003c12589088301770-git-send-email-\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" } ], "next": "0e0fc1c23e04c15e814763f2b366e92d87d8b95d" }