)]}' { "log": [ { "commit": "4968c300e1fa5389fdf1f1ebd8b8e4aec9aa4a9e", "tree": "3424a8c3d2543096acad7319ec57fa8f62813620", "parents": [ "101db7b41d8d6c070278bca1f7bce814ecbf781d" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Dec 07 16:32:40 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:22 2011 -0800" }, "message": "rcu: Augment rcu_batch_end tracing for idle and callback state\n\nThe current rcu_batch_end event trace records only the name of the RCU\nflavor and the total number of callbacks that remain queued on the\ncurrent CPU. This is insufficient for testing and tuning the new\ndyntick-idle RCU_FAST_NO_HZ code, so this commit adds idle state along\nwith whether or not any of the callbacks that were ready to invoke\nat the beginning of rcu_do_batch() are still queued.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "2d1dc9a600edf33321bcdc1c808b7957d8a3f3e1", "tree": "7b3701752afca2f0b3a80bbee901affa7c72a67c", "parents": [ "f23f7fa1c8effca19b52b98fc71016109d21db59" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Nov 30 17:29:18 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:11 2011 -0800" }, "message": "rcu: Remove redundant rcu_cpu_stall_suppress declaration\n\nNo point in having two identical rcu_cpu_stall_suppress declarations,\nso remove the more obscure of the two.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "dff1672d9199fffddb58fa7970ccf59005fc35f3", "tree": "a9740f65dfc203d183cb06ce0faeed0de5cef58b", "parents": [ "facc4e159672b4ed10aa18147bfa187b013c9505" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 29 15:57:13 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:09 2011 -0800" }, "message": "rcu: Keep invoking callbacks if CPU otherwise idle\n\nThe rcu_do_batch() function that invokes callbacks for TREE_RCU and\nTREE_PREEMPT_RCU normally throttles callback invocation to avoid degrading\nscheduling latency. However, as long as the CPU would otherwise be idle,\nthere is no downside to continuing to invoke any callbacks that have passed\nthrough their grace periods. In fact, processing such callbacks in a\ntimely manner has the benefit of increasing the probability that the\nCPU can enter the power-saving dyntick-idle mode.\n\nTherefore, this commit allows callback invocation to continue beyond the\npreset limit as long as the scheduler does not have some other task to\nrun and as long as context is that of the idle task or the relevant\nRCU kthread.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "facc4e159672b4ed10aa18147bfa187b013c9505", "tree": "0b058683e1dde49a73b858fc29477e6f76860add", "parents": [ "b6fc6020140db437069d5bec447858fcfd64d31c" ], "author": { "name": "Frederic Weisbecker", "email": "fweisbec@gmail.com", "time": "Mon Nov 28 16:26:56 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:08 2011 -0800" }, "message": "rcu: Irq nesting is always 0 on rcu_enter_idle_common\n\nBecause tasks don\u0027t nest, the -\u003edyntick_nesting must always be zero upon\nentry to rcu_idle_enter_common(). Therefore, pass \"0\" rather than the\ncounter itself.\n\nSigned-off-by: Frederic Weisbecker \u003cfweisbec@gmail.com\u003e\nCc: Josh Triplett \u003cjosh@joshtriplett.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "b6fc6020140db437069d5bec447858fcfd64d31c", "tree": "43b4f6aa420bd636a30116443f2417cd2176a679", "parents": [ "7cb92499000e3c86dae653077b1465458a039ef6" ], "author": { "name": "Frederic Weisbecker", "email": "fweisbec@gmail.com", "time": "Mon Nov 28 16:18:56 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:07 2011 -0800" }, "message": "rcu: Don\u0027t check irq nesting from rcu idle entry/exit\n\nBecause tasks do not nest, rcu_idle_enter() and rcu_idle_exit() do\nnot need to check for nesting. This commit therefore moves nesting\nchecks from rcu_idle_enter_common() to rcu_irq_exit() and from\nrcu_idle_exit_common() to rcu_irq_enter().\n\nSigned-off-by: Frederic Weisbecker \u003cfweisbec@gmail.com\u003e\nCc: Josh Triplett \u003cjosh@joshtriplett.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "7cb92499000e3c86dae653077b1465458a039ef6", "tree": "ebc982a5cc562b4fe0cb8f20541f45a5506a0b5f", "parents": [ "3842a0832a1d6eb0b31421f8810a813135967512" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Nov 28 12:28:34 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:07 2011 -0800" }, "message": "rcu: Permit dyntick-idle with callbacks pending\n\nThe current implementation of RCU_FAST_NO_HZ prevents CPUs from entering\ndyntick-idle state if they have RCU callbacks pending. Unfortunately,\nthis has the side-effect of often preventing them from entering this\nstate, especially if at least one other CPU is not in dyntick-idle state.\nHowever, the resulting per-tick wakeup is wasteful in many cases: if the\nCPU has already fully responded to the current RCU grace period, there\nwill be nothing for it to do until this grace period ends, which will\nfrequently take several jiffies.\n\nThis commit therefore permits a CPU that has done everything that the\ncurrent grace period has asked of it (rcu_pending() \u003d\u003d 0) even if it\nstill as RCU callbacks pending. However, such a CPU posts a timer to\nwake it up several jiffies later (6 jiffies, based on experience with\ngrace-period lengths). This wakeup is required to handle situations\nthat can result in all CPUs being in dyntick-idle mode, thus failing\nto ever complete the current grace period. If a CPU wakes up before\nthe timer goes off, then it cancels that timer, thus avoiding spurious\nwakeups.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "f0e7c19db8798b4b991a2c71911e71f5dfdb348f", "tree": "fce748b22aeb29f8e28a28028211e777bc4123dd", "parents": [ "c92b131bdcf89bf79870f1631d07547241a98f6c" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Nov 23 15:02:05 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:05 2011 -0800" }, "message": "rcu: Identify dyntick-idle CPUs on first force_quiescent_state() pass\n\nFixes and workarounds for a number of issues (for example, that in\ndf4012edc) make it safe to once again detect dyntick-idle CPUs on the\nfirst pass of force_quiescent_state(), so this commit makes that change.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "c92b131bdcf89bf79870f1631d07547241a98f6c", "tree": "618f38e1f4e6f71e056b240613a89915730b03f9", "parents": [ "3ad0decf98d97b9039d8ed47cee287366b929cdf" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Nov 23 13:38:58 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:04 2011 -0800" }, "message": "rcu: Remove dynticks false positives and RCU failures\n\nAssertions in rcu_init_percpu_data() unknowingly relied on outgoing\nCPUs being turned off before reaching the idle loop. Unfortunately,\nwhen running under kvm/qemu on x86, CPUs really can get to idle before\nbegin shut off. These CPUs are then born in dyntick-idle mode from an\nRCU perspective, which results in splats in rcu_init_percpu_data() and\nin RCU wrongly ignoring those CPUs despite them being active. This in\nturn can cause RCU to end grace periods prematurely, potentially freeing\nup memory that the newly onlined CPUs were still using. This is most\ndecidedly not what we need to see in an RCU implementation.\n\nThis commit therefore replaces the assertions in rcu_init_percpu_data()\nwith code that forces RCU\u0027s dyntick-idle view of newly onlined CPUs to\nmatch reality.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "f535a607c13c7b674e0788ca5765779aa74a01c3", "tree": "433a07d16ff11d9f67e7991831cebf3cadfd9939", "parents": [ "84ad00cb61f1cb21f0b63bc6f7dc254399eb3830" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 22 20:43:02 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:32:02 2011 -0800" }, "message": "rcu: Eliminate RCU_FAST_NO_HZ grace-period hang\n\nWith the new implementation of RCU_FAST_NO_HZ, it was possible to hang\nRCU grace periods as follows:\n\no\tCPU 0 attempts to go idle, cycles several times through the\n\trcu_prepare_for_idle() loop, then goes dyntick-idle when\n\tRCU needs nothing more from it, while still having at least\n\ton RCU callback pending.\n\no\tCPU 1 goes idle with no callbacks.\n\nBoth CPUs can then stay in dyntick-idle mode indefinitely, preventing\nthe RCU grace period from ever completing, possibly hanging the system.\n\nThis commit therefore prevents CPUs that have RCU callbacks from entering\ndyntick-idle mode. This approach also eliminates the need for the\nend-of-grace-period IPIs used previously.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "99745b6a83414006f5c1e83efaebb423b41b67ef", "tree": "454ab3cf035a8d9a42f378066b166e8b2e6ba927", "parents": [ "c4f3060843506ba6d473ab9a0afe5bd5dc93a00d" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Nov 10 15:48:45 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:48 2011 -0800" }, "message": "rcu: Make RCU use the new is_idle_task() API\n\nChange from direct comparison of -\u003epid with zero to is_idle_task().\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "11dbaa8cb79a6e4a234a134898436f717a663f01", "tree": "79d18cd1af72016fcc54768388e46253208abe2e", "parents": [ "aea1b35e29e658d42d7ba2237f3aa7f93e18509d" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Nov 02 07:38:25 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:44 2011 -0800" }, "message": "rcu: Fix idle-task checks\n\nRCU has traditionally relied on idle_cpu() to determine whether a given\nCPU is running in the context of an idle task, but commit 908a3283\n(Fix idle_cpu()) has invalidated this approach. After commit 908a3283,\nidle_cpu() will return true if the current CPU is currently running the\nidle task, and will be doing so for the foreseeable future. RCU instead\nneeds to know whether or not the current CPU is currently running the\nidle task, regardless of what the near future might bring.\n\nThis commit therefore switches from idle_cpu() to \"current-\u003epid !\u003d 0\".\n\nReported-by: Wu Fengguang \u003cfengguang.wu@intel.com\u003e\nSuggested-by: Carsten Emde \u003cC.Emde@osadl.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nAcked-by: Steven Rostedt \u003crostedt@goodmis.org\u003e\nTested-by: Wu Fengguang \u003cfengguang.wu@intel.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "aea1b35e29e658d42d7ba2237f3aa7f93e18509d", "tree": "0221b1c12b5c3e3ed7f2bb2ffc957b09891bcb51", "parents": [ "0989cb46783188ea7346ba6490be0046b9b7a725" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Nov 02 06:54:54 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:43 2011 -0800" }, "message": "rcu: Allow dyntick-idle mode for CPUs with callbacks\n\nCurrently, RCU does not permit a CPU to enter dyntick-idle mode if that\nCPU has any RCU callbacks queued. This means that workloads for which\neach CPU wakes up and does some RCU updates every few ticks will never\nenter dyntick-idle mode. This can result in significant unnecessary power\nconsumption, so this patch permits a given to enter dyntick-idle mode if\nit has callbacks, but only if that same CPU has completed all current\nwork for the RCU core. We determine use rcu_pending() to determine\nwhether a given CPU has completed all current work for the RCU core.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "0989cb46783188ea7346ba6490be0046b9b7a725", "tree": "ab3ca7e6099efd9ca814e9202a679388f36ecc90", "parents": [ "4145fa7fbee3ec1e61c52825b146192885d9759f" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Nov 01 08:57:21 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:42 2011 -0800" }, "message": "rcu: Add more information to the wrong-idle-task complaint\n\nThe current code just complains if the current task is not the idle task.\nThis commit therefore adds printing of the identity of the idle task.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "4145fa7fbee3ec1e61c52825b146192885d9759f", "tree": "58e0767a59bf110acb99da3fb5c1f0f0b5170be8", "parents": [ "9ceae0e248fb553c702d51d5275167d462f4efd2" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Oct 31 15:01:54 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:42 2011 -0800" }, "message": "rcu: Deconfuse dynticks entry-exit tracing\n\nThe trace_rcu_dyntick() trace event did not print both the old and\nthe new value of the nesting level, and furthermore printed only\nthe low-order 32 bits of it. This could result in some confusion\nwhen interpreting trace-event dumps, so this commit prints both\nthe old and the new value, prints the full 64 bits, and also selects\nthe process-entry/exit increment to print nicely in hexadecimal.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "e6b80a3b0994ea6c3d876d72464f2debbfcfeb05", "tree": "207337c207fecf2e05e52ffb3003831d240529fd", "parents": [ "a0f8eefb127f5be07628954f310a7fc8c82b2fc3" ], "author": { "name": "Frederic Weisbecker", "email": "fweisbec@gmail.com", "time": "Fri Oct 07 16:25:18 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:30 2011 -0800" }, "message": "rcu: Detect illegal rcu dereference in extended quiescent state\n\nReport that none of the rcu read lock maps are held while in an RCU\nextended quiescent state (the section between rcu_idle_enter()\nand rcu_idle_exit()). This helps detect any use of rcu_dereference()\nand friends from within the section in idle where RCU is not allowed.\n\nThis way we can guarantee an extended quiescent window where the CPU\ncan be put in dyntick idle mode or can simply aoid to be part of any\nglobal grace period completion while in the idle loop.\n\nUses of RCU from such mode are totally ignored by RCU, hence the\nimportance of these checks.\n\nSigned-off-by: Frederic Weisbecker \u003cfweisbec@gmail.com\u003e\nCc: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: Ingo Molnar \u003cmingo@elte.hu\u003e\nCc: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nCc: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nCc: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "b40d293eb36ba40cd428b6d178db911174689702", "tree": "78109a13c0bf86608f3caaea547fd9e948aee743", "parents": [ "34240697d619c439c55f21989680024dcb604aab" ], "author": { "name": "Thomas Gleixner", "email": "tglx@linutronix.de", "time": "Sat Oct 22 07:12:34 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:28 2011 -0800" }, "message": "rcu: Omit self-awaken when setting up expedited grace period\n\nWhen setting up an expedited grace period, if there were no readers, the\ntask will awaken itself. This commit removes this useless self-awakening.\n\nSigned-off-by: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "34240697d619c439c55f21989680024dcb604aab", "tree": "5fa80ddb82706121520677e8e700e5081f5edaa1", "parents": [ "2c01531f08f8ba663a20d8472a3032f6df133b6e" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Oct 03 11:38:52 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:27 2011 -0800" }, "message": "rcu: Disable preemption in rcu_is_cpu_idle()\n\nBecause rcu_is_cpu_idle() is to be used to check for extended quiescent\nstates in RCU-preempt read-side critical sections, it cannot assume that\npreemption is disabled. And preemption must be disabled when accessing\nthe dyntick-idle state, because otherwise the following sequence of events\ncould occur:\n\n1.\tTask A on CPU 1 enters rcu_is_cpu_idle() and picks up the pointer\n\tto CPU 1\u0027s per-CPU variables.\n\n2.\tTask B preempts Task A and starts running on CPU 1.\n\n3.\tTask A migrates to CPU 2.\n\n4.\tTask B blocks, leaving CPU 1 idle.\n\n5.\tTask A continues execution on CPU 2, accessing CPU 1\u0027s dyntick-idle\n\tinformation using the pointer fetched in step 1 above, and finds\n\tthat CPU 1 is idle.\n\n6.\tTask A therefore incorrectly concludes that it is executing in\n\tan extended quiescent state, possibly issuing a spurious splat.\n\nTherefore, this commit disables preemption within the rcu_is_cpu_idle()\nfunction.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "9b2e4f1880b789be1f24f9684f7a54b90310b5c0", "tree": "1fa922e0616e298837a7079cb49118188a58186c", "parents": [ "b804cb9e91c6c304959c69d4f9daeef4ffdba71c" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Fri Sep 30 12:10:22 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:24 2011 -0800" }, "message": "rcu: Track idleness independent of idle tasks\n\nEarlier versions of RCU used the scheduling-clock tick to detect idleness\nby checking for the idle task, but handled idleness differently for\nCONFIG_NO_HZ\u003dy. But there are now a number of uses of RCU read-side\ncritical sections in the idle task, for example, for tracing. A more\nfine-grained detection of idleness is therefore required.\n\nThis commit presses the old dyntick-idle code into full-time service,\nso that rcu_idle_enter(), previously known as rcu_enter_nohz(), is\nalways invoked at the beginning of an idle loop iteration. Similarly,\nrcu_idle_exit(), previously known as rcu_exit_nohz(), is always invoked\nat the end of an idle-loop iteration. This allows the idle task to\nuse RCU everywhere except between consecutive rcu_idle_enter() and\nrcu_idle_exit() calls, in turn allowing architecture maintainers to\nspecify exactly where in the idle loop that RCU may be used.\n\nBecause some of the userspace upcall uses can result in what looks\nto RCU like half of an interrupt, it is not possible to expect that\nthe irq_enter() and irq_exit() hooks will give exact counts. This\npatch therefore expands the -\u003edynticks_nesting counter to 64 bits\nand uses two separate bitfields to count process/idle transitions\nand interrupt entry/exit transitions. It is presumed that userspace\nupcalls do not happen in the idle loop or from usermode execution\n(though usermode might do a system call that results in an upcall).\nThe counter is hard-reset on each process/idle transition, which\navoids the interrupt entry/exit error from accumulating. Overflow\nis avoided by the 64-bitness of the -\u003edyntick_nesting counter.\n\nThis commit also adds warnings if a non-idle task asks RCU to enter\nidle state (and these checks will need some adjustment before applying\nFrederic\u0027s OS-jitter patches (http://lkml.org/lkml/2011/10/7/246).\nIn addition, validation of -\u003edynticks and -\u003edynticks_nesting is added.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "af446b702c58b700cc5fa99f6edc78b99e55b995", "tree": "0eacf146b7f1b221e844c52db6061e31928ff99e", "parents": [ "dc47ce90c3a822cd7c9e9339fe4d5f61dcb26b50" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat Sep 10 21:54:08 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Dec 11 10:31:20 2011 -0800" }, "message": "rcu: -\u003esignaled better named -\u003efqs_state\n\nThe -\u003esignaled field was named before complications in the form of\ndyntick-idle mode and offlined CPUs. These complications have required\nthat force_quiescent_state() be implemented as a state machine, instead\nof simply unconditionally sending reschedule IPIs. Therefore, this\ncommit renames -\u003esignaled to -\u003efqs_state to catch up with the new\nforce_quiescent_state() reality.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "9984de1a5a8a96275fcab818f7419af5a3c86e71", "tree": "1935d411752707a1621c5caf64f75dfe105beb3a", "parents": [ "7c77509c542927ee2a3c8812fad84957e51bf67d" ], "author": { "name": "Paul Gortmaker", "email": "paul.gortmaker@windriver.com", "time": "Mon May 23 14:51:41 2011 -0400" }, "committer": { "name": "Paul Gortmaker", "email": "paul.gortmaker@windriver.com", "time": "Mon Oct 31 09:20:12 2011 -0400" }, "message": "kernel: Map most files to use export.h instead of module.h\n\nThe changed files were only including linux/module.h for the\nEXPORT_SYMBOL infrastructure, and nothing else. Revector them\nonto the isolated export header for faster compile times.\n\nNothing to see here but a whole lot of instances of:\n\n -#include \u003clinux/module.h\u003e\n +#include \u003clinux/export.h\u003e\n\nThis commit is only changing the kernel dir; next targets\nwill probably be mm, fs, the arch dirs, etc.\n\nSigned-off-by: Paul Gortmaker \u003cpaul.gortmaker@windriver.com\u003e\n" }, { "commit": "afe24b122eb6edb5f1cb942570ac8d766105c7fc", "tree": "c1f9e2fcbcf2d374f36ee3bfc45babf576cb6246", "parents": [ "e90c53d3e238dd0b7b02964370e8fece1778df96" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Aug 24 16:52:09 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:49 2011 -0700" }, "message": "rcu: Move propagation of -\u003ecompleted from rcu_start_gp() to rcu_report_qs_rsp()\n\nIt is possible for the CPU that noted the end of the prior grace period\nto not need a new one, and therefore to decide to propagate -\u003ecompleted\nthroughout the rcu_node tree without starting another grace period.\nHowever, in so doing, it releases the root rcu_node structure\u0027s lock,\nwhich can allow some other CPU to start another grace period. The first\nCPU will be propagating -\u003ecompleted in parallel with the second CPU\ninitializing the rcu_node tree for the new grace period. In theory\nthis is harmless, but in practice we need to keep things simple.\n\nThis commit therefore moves the propagation of -\u003ecompleted to\nrcu_report_qs_rsp(), and refrains from marking the old grace period\nas having been completed until it has finished doing this. This\nprevents anyone from starting a new grace period concurrently with\nmarking the old grace period as having been completed.\n\nOf course, the optimization where a CPU needing a new grace period\ndoesn\u0027t bother marking the old one completed is still in effect:\nIn that case, the marking happens implicitly as part of initializing\nthe new grace period.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e90c53d3e238dd0b7b02964370e8fece1778df96", "tree": "23e5b8623c2c37bd8a4656c66266209872f80701", "parents": [ "5b61b0baa9e80289c53413e573befc5790a04ac7" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat Aug 20 18:29:32 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:48 2011 -0700" }, "message": "rcu: Remove rcu_needs_cpu_flush() to avoid false quiescent states\n\nThe purpose of rcu_needs_cpu_flush() was to iterate on pushing the\ncurrent grace period in order to help the current CPU enter dyntick-idle\nmode. However, this can result in failures if the CPU starts entering\ndyntick-idle mode, but then backs out. In this case, the call to\nrcu_pending() from rcu_needs_cpu_flush() might end up announcing a\nnon-existing quiescent state.\n\nThis commit therefore removes rcu_needs_cpu_flush() in favor of letting\nthe dyntick-idle machinery at the end of the softirq handler push the\nloop along via its call to rcu_pending().\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "5b61b0baa9e80289c53413e573befc5790a04ac7", "tree": "5f3c4b002089b6d6e92fafaba42ab4a42ef1f1f9", "parents": [ "ab8f11e5f6655861ad4758a7da76b2fc0e0dcc98" ], "author": { "name": "Mike Galbraith", "email": "efault@gmx.de", "time": "Fri Aug 19 11:39:11 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:47 2011 -0700" }, "message": "rcu: Wire up RCU_BOOST_PRIO for rcutree\n\nRCU boost threads start life at RCU_BOOST_PRIO, while others remain\nat RCU_KTHREAD_PRIO. While here, change thread names to match other\nkthreads, and adjust rcu_yield() to not override the priority set by\nthe user. This last change sets the stage for runtime changes to\npriority in the -rt tree.\n\nSigned-off-by: Mike Galbraith \u003cefault@gmx.de\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "06ae115a1d551cd952d80df06eaf8b5153351875", "tree": "858163c4983a7cc92bffab2267597f3bb802071b", "parents": [ "9bc8b5586f94be6391458074ecbba8827ba8ba9d" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sun Aug 14 15:56:54 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:42 2011 -0700" }, "message": "rcu: Avoid having just-onlined CPU resched itself when RCU is idle\n\nCPUs set rdp-\u003eqs_pending when coming online to resolve races with\ngrace-period start. However, this means that if RCU is idle, the\njust-onlined CPU might needlessly send itself resched IPIs. Adjust\nthe online-CPU initialization to avoid this, and also to correctly\ncause the CPU to respond to the current grace period if needed.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Josh Boyer \u003cjwboyer@redhat.com\u003e\nTested-by: Christian Hoffmann \u003cemail@christianhoffmann.info\u003e\n" }, { "commit": "9bc8b5586f94be6391458074ecbba8827ba8ba9d", "tree": "323804bb7180337c266f88519cea6b1f42114ead", "parents": [ "037067a1b6f9a70f862f3ed9d59fe28b7cd55ac4" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat Aug 13 13:31:47 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:41 2011 -0700" }, "message": "rcu: Suppress NMI backtraces when stall ends before dump\n\nIt is possible for an RCU CPU stall to end just as it is detected, in\nwhich case the current code will uselessly dump all CPU\u0027s stacks.\nThis commit therefore checks for this condition and refrains from\nsending needless NMIs.\n\nAnd yes, the stall might also end just after we checked all CPUs and\ntasks, but in that case we would at least have given some clue as\nto which CPU/task was at fault.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "037067a1b6f9a70f862f3ed9d59fe28b7cd55ac4", "tree": "0f54deff3994de1bdb54707b7943307244ed692f", "parents": [ "82e78d80fc392ac7e98326bc8beeb8a679913ffd" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sun Aug 07 20:26:31 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:40 2011 -0700" }, "message": "rcu: Prohibit grace periods during early boot\n\nGreater use of RCU during early boot (before the scheduler is operating)\nis causing RCU to attempt to start grace periods during that time, which\nin turn is resulting in both RCU and the callback functions attempting\nto use the scheduler before it is ready.\n\nThis commit prevents these problems by prohibiting RCU grace periods\nuntil after the scheduler has spawned the first non-idle task.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "5c51dd7349d4bb26f845f17f85daa168f5fa03f2", "tree": "a846bf11cb973e8d0bb28d9c07653663759fb6c8", "parents": [ "4627e240dfee4a0a46a58010b1b721b4ded1918f" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Aug 04 06:59:03 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:37 2011 -0700" }, "message": "rcu: Prevent early boot set_need_resched() from __rcu_pending()\n\nThere isn\u0027t a whole lot of point in poking the scheduler before there\nare other tasks to switch to. This commit therefore adds a check\nfor rcu_scheduler_fully_active in __rcu_pending() to suppress any\npre-scheduler calls to set_need_resched(). The downside of this approach\nis additional runtime overhead in a reasonably hot code path.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "4627e240dfee4a0a46a58010b1b721b4ded1918f", "tree": "807cc0618bc843800aa85fd6d20b4e9ee7adb74b", "parents": [ "6206ab9bab620fc0fbbed30ce20d145b0b3d1840" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Aug 03 03:34:24 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:36 2011 -0700" }, "message": "rcu: Dump local stack if cannot dump all CPUs\u0027 stacks\n\nThe trigger_all_cpu_backtrace() function is a no-op in architectures that\ndo not define arch_trigger_all_cpu_backtrace. On such architectures, RCU\nCPU stall warning messages contain no stack trace information, which makes\ndebugging quite difficult. This commit therefore substitutes dump_stack()\nfor architectures that do not define arch_trigger_all_cpu_backtrace,\nso that at least the local CPU\u0027s stack is dumped as part of the RCU CPU\nstall warning message.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "7eb4f4553ceaa6c64da83c8a71d5a991c0188655", "tree": "308ce033d4c449effc3d6b99eb0adf8857357124", "parents": [ "717f98f0f87c16c1e5768ac8f67c27f3d3535a25" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat Jul 30 07:32:48 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:32 2011 -0700" }, "message": "rcu: Make rcu_implicit_dynticks_qs() locals be correct size\n\nWhen the -\u003edynticks field in the rcu_dynticks structure changed to an\natomic_t, its size on 64-bit systems changed from 64 bits to 32 bits.\nThe local variables in rcu_implicit_dynticks_qs() need to change as\nwell, hence this commit.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "717f98f0f87c16c1e5768ac8f67c27f3d3535a25", "tree": "4d2234eb29a350a3a5b1d7e4d29445c57b66f9f2", "parents": [ "fc0763f53e3ff6a6bfa66934662a3446b9ca6f16" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Aug 05 22:39:02 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:31 2011 -0700" }, "message": "rcu: Eliminate in_irq() checks in rcu_enter_nohz()\n\nThe in_irq() check in rcu_enter_nohz() is redundant because if we really\nare in an interrupt, the attempt to re-enter dyntick-idle mode will invoke\nrcu_needs_cpu() in any case, which will force the check for RCU callbacks.\nSo this commit removes the check along with the set_need_resched().\n\nSuggested-by: Frederic Weisbecker \u003cfweisbec@gmail.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e4cc1f22b2f4e9b0207a8cdb63e56dcf99e82d35", "tree": "ec3fad6587f0a83e764a571394bdfb85a84edf30", "parents": [ "d4c08f2ac311a360230eef7e5395b0ec8d8f0670" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Jun 27 00:17:43 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:22 2011 -0700" }, "message": "rcu: Simplify quiescent-state accounting\n\nThere is often a delay between the time that a CPU passes through a\nquiescent state and the time that this quiescent state is reported to the\nRCU core. It is quite possible that the grace period ended before the\nquiescent state could be reported, for example, some other CPU might have\ndeduced that this CPU passed through dyntick-idle mode. It is critically\nimportant that quiescent state be counted only against the grace period\nthat was in effect at the time that the quiescent state was detected.\n\nPreviously, this was handled by recording the number of the last grace\nperiod to complete when passing through a quiescent state. The RCU\ncore then checks this number against the current value, and rejects\nthe quiescent state if there is a mismatch. However, one additional\npossibility must be accounted for, namely that the quiescent state was\nrecorded after the prior grace period completed but before the current\ngrace period started. In this case, the RCU core must reject the\nquiescent state, but the recorded number will match. This is handled\nwhen the CPU becomes aware of a new grace period -- at that point,\nit invalidates any prior quiescent state.\n\nThis works, but is a bit indirect. The new approach records the current\ngrace period, and the RCU core checks to see (1) that this is still the\ncurrent grace period and (2) that this grace period has not yet ended.\nThis approach simplifies reasoning about correctness, and this commit\nchanges over to this new approach.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "d4c08f2ac311a360230eef7e5395b0ec8d8f0670", "tree": "06e425b8153e076fbe43b037cf4497ac50afddb7", "parents": [ "965a002b4f1a458c5dcb334ec29f48a0046faa25" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sat Jun 25 06:36:56 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:21 2011 -0700" }, "message": "rcu: Add grace-period, quiescent-state, and call_rcu trace events\n\nAdd trace events to record grace-period start and end, quiescent states,\nCPUs noticing grace-period start and end, grace-period initialization,\ncall_rcu() invocation, tasks blocking in RCU read-side critical sections,\ntasks exiting those same critical sections, force_quiescent_state()\ndetection of dyntick-idle and offline CPUs, CPUs entering and leaving\ndyntick-idle mode (except from NMIs), CPUs coming online and going\noffline, and CPUs being kicked for staying in dyntick-idle mode for too\nlong (as in many weeks, even on 32-bit systems).\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n\nrcu: Add the rcu flavor to callback trace events\n\nThe earlier trace events for registering RCU callbacks and for invoking\nthem did not include the RCU flavor (rcu_bh, rcu_preempt, or rcu_sched).\nThis commit adds the RCU flavor to those trace events.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e0f23060adfa3f27beaa7918eff70258b88471b6", "tree": "f0e8ed4ad97a1fc4d3c2c5af639a63063ae52e32", "parents": [ "72fe701b70e6ced35d734b676c13efbc8fc769a9" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Jun 21 01:29:39 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:16 2011 -0700" }, "message": "rcu: Update comments to reflect softirqs vs. kthreads\n\nWe now have kthreads only for flavors of RCU that support boosting,\nso update the now-misleading comments accordingly.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "72fe701b70e6ced35d734b676c13efbc8fc769a9", "tree": "87bece8406b18e27409a33355c384c9f759915ae", "parents": [ "e99033c5c160f1f247c665923a66acec693a967c" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Jun 21 01:14:54 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:15 2011 -0700" }, "message": "rcu: Add RCU type to callback-invocation tracing\n\nAdd a string to the rcu_batch_start() and rcu_batch_end() trace\nmessages that indicates the RCU type (\"rcu_sched\", \"rcu_bh\", or\n\"rcu_preempt\"). The trace messages for the actual invocations\nthemselves are not marked, as it should be clear from the\nrcu_batch_start() and rcu_batch_end() events before and after.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "e99033c5c160f1f247c665923a66acec693a967c", "tree": "c5b0863ad4b673134070872a159055237186f0fe", "parents": [ "300df91ca9358f7f09298eec9503c12b32054ef7" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Jun 21 00:13:44 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:14 2011 -0700" }, "message": "rcu: Put names into TINY_RCU structures under RCU_TRACE\n\nIn order to allow event tracing to distinguish between flavors of\nRCU, we need those names in the relevant RCU data structures. TINY_RCU\nhas avoided them for memory-footprint reasons, so add them only if\nCONFIG_RCU_TRACE\u003dy.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "300df91ca9358f7f09298eec9503c12b32054ef7", "tree": "8917e0c2ee94a5385f4968094c0e1d9a7fdc0055", "parents": [ "29c00b4a1d9e277786120032aa8364631820d863" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sat Jun 18 22:26:31 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:13 2011 -0700" }, "message": "rcu: Event-trace markers for computing RCU CPU utilization\n\nThis commit adds the trace_rcu_utilization() marker that is to be\nused to allow postprocessing scripts compute RCU\u0027s CPU utilization,\ngive or take event-trace overhead. Note that we do not include RCU\u0027s\ndyntick-idle interface because event tracing requires RCU protection,\nwhich is not available in dyntick-idle mode.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "29c00b4a1d9e277786120032aa8364631820d863", "tree": "23bf0c8cdb5268ef92ef9ed007639705e922b1b0", "parents": [ "9d68197c05201d8edc70d58bd1d5dad05d8455e8" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Fri Jun 17 15:53:19 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:38:12 2011 -0700" }, "message": "rcu: Add event-tracing for RCU callback invocation\n\nThere was recently some controversy about the overhead of invoking RCU\ncallbacks. Add TRACE_EVENT()s to obtain fine-grained timings for the\nstart and stop of a batch of callbacks and also for each callback invoked.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "2c42818e962e2858334bf45bfc56662b3752df34", "tree": "192364123c9aeeab282c53168e51eddece9d8be4", "parents": [ "f039d1f1884b2fe9c13d28f59d8330f0b0518fc4" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 22:14:36 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Sep 28 21:36:42 2011 -0700" }, "message": "rcu: Abstract common code for RCU grace-period-wait primitives\n\nPull the code that waits for an RCU grace period into a single function,\nwhich is then called by synchronize_rcu() and friends in the case of\nTREE_RCU and TREE_PREEMPT_RCU, and from rcu_barrier() and friends in\nthe case of TINY_RCU and TINY_PREEMPT_RCU.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "b0d304172f49061b4ff78f9e2b02719ac69c8a7e", "tree": "882123e565ced895910490514f6e676c708b5001", "parents": [ "620917de59eeb934b9f8cf35cc2d95c1ac8ed0fc" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sun Jul 10 15:57:35 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jul 13 08:17:56 2011 -0700" }, "message": "rcu: Prevent RCU callbacks from executing before scheduler initialized\n\nUnder some rare but real combinations of configuration parameters, RCU\ncallbacks are posted during early boot that use kernel facilities that\nare not yet initialized. Therefore, when these callbacks are invoked,\nhard hangs and crashes ensue. This commit therefore prevents RCU\ncallbacks from being invoked until after the scheduler is fully up and\nrunning, as in after multiple tasks have been spawned.\n\nIt might well turn out that a better approach is to identify the specific\nRCU callbacks that are causing this problem, but that discussion will\nwait until such time as someone really needs an RCU callback to be invoked\n(as opposed to merely registered) during early boot.\n\nReported-by: julie Sullivan \u003ckernelmail.jms@gmail.com\u003e\nReported-by: RKK \u003ckulkarni.ravi4@gmail.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Konrad Rzeszutek Wilk \u003ckonrad.wilk@oracle.com\u003e\nTested-by: julie Sullivan \u003ckernelmail.jms@gmail.com\u003e\nTested-by: RKK \u003ckulkarni.ravi4@gmail.com\u003e\n" }, { "commit": "f8b7fc6b514f34a51875dd48dff70d4d17a54f38", "tree": "dc6583851e3536516761b0fd9b07f5a3a1743ebb", "parents": [ "a46e0899eec7a3069bcadd45dfba7bf67c6ed016" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Jun 16 08:26:32 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Jun 16 16:12:05 2011 -0700" }, "message": "rcu: Move RCU_BOOST #ifdefs to header file\n\nThe commit \"use softirq instead of kthreads except when RCU_BOOST\u003dy\"\njust applied #ifdef in place. This commit is a cleanup that moves\nthe newly #ifdef\u0027ed code to the header file kernel/rcutree_plugin.h.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "a46e0899eec7a3069bcadd45dfba7bf67c6ed016", "tree": "78158b4056fe1365d5086f66769abdc3ef2643c3", "parents": [ "09223371deac67d08ca0b70bd18787920284c967" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jun 15 15:47:09 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jun 15 23:07:21 2011 -0700" }, "message": "rcu: use softirq instead of kthreads except when RCU_BOOST\u003dy\n\nThis patch #ifdefs RCU kthreads out of the kernel unless RCU_BOOST\u003dy,\nthus eliminating context-switch overhead if RCU priority boosting has\nnot been configured.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "09223371deac67d08ca0b70bd18787920284c967", "tree": "0ad0354a93ac209c8cfd2b79d03d0462aa3c5c58", "parents": [ "9a432736904d386cda28b987b38ba14dae960ecc" ], "author": { "name": "Shaohua Li", "email": "shaohua.li@intel.com", "time": "Tue Jun 14 13:26:25 2011 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jun 14 15:25:39 2011 -0700" }, "message": "rcu: Use softirq to address performance regression\n\nCommit a26ac2455ffcf3(rcu: move TREE_RCU from softirq to kthread)\nintroduced performance regression. In an AIM7 test, this commit degraded\nperformance by about 40%.\n\nThe commit runs rcu callbacks in a kthread instead of softirq. We observed\nhigh rate of context switch which is caused by this. Out test system has\n64 CPUs and HZ is 1000, so we saw more than 64k context switch per second\nwhich is caused by RCU\u0027s per-CPU kthread. A trace showed that most of\nthe time the RCU per-CPU kthread doesn\u0027t actually handle any callbacks,\nbut instead just does a very small amount of work handling grace periods.\nThis means that RCU\u0027s per-CPU kthreads are making the scheduler do quite\na bit of work in order to allow a very small amount of RCU-related\nprocessing to be done.\n\nAlex Shi\u0027s analysis determined that this slowdown is due to lock\ncontention within the scheduler. Unfortunately, as Peter Zijlstra points\nout, the scheduler\u0027s real-time semantics require global action, which\nmeans that this contention is inherent in real-time scheduling. (Yes,\nperhaps someone will come up with a workaround -- otherwise, -rt is not\ngoing to do well on large SMP systems -- but this patch will work around\nthis issue in the meantime. And \"the meantime\" might well be forever.)\n\nThis patch therefore re-introduces softirq processing to RCU, but only\nfor core RCU work. RCU callbacks are still executed in kthread context,\nso that only a small amount of RCU work runs in softirq context in the\ncommon case. This should minimize ksoftirqd execution, allowing us to\nskip boosting of ksoftirqd for CONFIG_RCU_BOOST\u003dy kernels.\n\nSigned-off-by: Shaohua Li \u003cshaohua.li@intel.com\u003e\nTested-by: \"Alex,Shi\" \u003calex.shi@intel.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "9a432736904d386cda28b987b38ba14dae960ecc", "tree": "20294203f9c6132d79737e9ab54c746546afe7d4", "parents": [ "59c5f46fbe01a00eedf54a23789634438bb80603" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 30 20:38:55 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Jun 14 15:25:15 2011 -0700" }, "message": "rcu: Simplify curing of load woes\n\nMake the functions creating the kthreads wake them up. Leverage the\nfact that the per-node and boost kthreads can run anywhere, thus\ndispensing with the need to wake them up once the incoming CPU has\ngone fully online.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Daniel J Blueman \u003cdaniel.blueman@gmail.com\u003e\n" }, { "commit": "d72bce0e67e8afc6eb959f656013cbb577426f1e", "tree": "9c93d4df9aa895d6f2f555e0cf50e7ae5ebaded4", "parents": [ "55922c9d1b84b89cb946c777fddccb3247e7df2c" ], "author": { "name": "Peter Zijlstra", "email": "peterz@infradead.org", "time": "Mon May 30 13:34:51 2011 +0200" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Tue May 31 10:01:48 2011 +0200" }, "message": "rcu: Cure load woes\n\nCommit cc3ce5176d83 (rcu: Start RCU kthreads in TASK_INTERRUPTIBLE\nstate) fudges a sleeping task\u0027 state, resulting in the scheduler seeing\na TASK_UNINTERRUPTIBLE task going to sleep, but a TASK_INTERRUPTIBLE\ntask waking up. The result is unbalanced load calculation.\n\nThe problem that patch tried to address is that the RCU threads could\nstay in UNINTERRUPTIBLE state for quite a while and triggering the hung\ntask detector due to on-demand wake-ups.\n\nCure the problem differently by always giving the tasks at least one\nwake-up once the CPU is fully up and running, this will kick them out of\nthe initial UNINTERRUPTIBLE state and into the regular INTERRUPTIBLE\nwait state.\n\n[ The alternative would be teaching kthread_create() to start threads as\n INTERRUPTIBLE but that needs a tad more thought. ]\n\nReported-by: Damien Wyart \u003cdamien.wyart@free.fr\u003e\nSigned-off-by: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nAcked-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nLink: http://lkml.kernel.org/r/1306755291.1200.2872.camel@twins\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "cc3ce5176d83cd8ae1134f86e208ea758d6cb78e", "tree": "a28ada4c1e5fd8c3e8210ca1a1df950af8a704ff", "parents": [ "08bca60a6912ad225254250c0a9c3a05b4152cfa" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed May 25 13:42:06 2011 -0700" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat May 28 17:41:56 2011 +0200" }, "message": "rcu: Start RCU kthreads in TASK_INTERRUPTIBLE state\n\nUpon creation, kthreads are in TASK_UNINTERRUPTIBLE state, which can\nresult in softlockup warnings. Because some of RCU\u0027s kthreads can\nlegitimately be idle indefinitely, start them in TASK_INTERRUPTIBLE\nstate in order to avoid those warnings.\n\nSuggested-by: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Yinghai Lu \u003cyinghai@kernel.org\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "08bca60a6912ad225254250c0a9c3a05b4152cfa", "tree": "760b20e6eaaa02412fcecb6bc5c3598b6bc0fdce", "parents": [ "8826f3b0397562eee6f8785d548be9dfdb169100" ], "author": { "name": "Peter Zijlstra", "email": "a.p.zijlstra@chello.nl", "time": "Fri May 20 16:06:29 2011 -0700" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat May 28 17:41:52 2011 +0200" }, "message": "rcu: Remove waitqueue usage for cpu, node, and boost kthreads\n\nIt is not necessary to use waitqueues for the RCU kthreads because\nwe always know exactly which thread is to be awakened. In addition,\nwake_up() only issues an actual wakeup when there is a thread waiting on\nthe queue, which was why there was an extra explicit wake_up_process()\nto get the RCU kthreads started.\n\nEliminating the waitqueues (and wake_up()) in favor of wake_up_process()\neliminates the need for the initial wake_up_process() and also shrinks\nthe data structure size a bit. The wakeup logic is placed in a new\nrcu_wait() macro.\n\nSigned-off-by: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "8826f3b0397562eee6f8785d548be9dfdb169100", "tree": "3b911e0ffe56b1792126f3f36f1fa4de39fd5979", "parents": [ "55c2945aa9d4d907ec5ca4f6a4e30ae908d8d30d" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed May 11 05:41:41 2011 -0700" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat May 28 17:41:49 2011 +0200" }, "message": "rcu: Avoid acquiring rcu_node locks in timer functions\n\nThis commit switches manipulations of the rcu_node -\u003ewakemask field\nto atomic operations, which allows rcu_cpu_kthread_timer() to avoid\nacquiring the rcu_node lock. This should avoid the following lockdep\nsplat reported by Valdis Kletnieks:\n\n[ 12.872150] usb 1-4: new high speed USB device number 3 using ehci_hcd\n[ 12.986667] usb 1-4: New USB device found, idVendor\u003d413c, idProduct\u003d2513\n[ 12.986679] usb 1-4: New USB device strings: Mfr\u003d0, Product\u003d0, SerialNumber\u003d0\n[ 12.987691] hub 1-4:1.0: USB hub found\n[ 12.987877] hub 1-4:1.0: 3 ports detected\n[ 12.996372] input: PS/2 Generic Mouse as /devices/platform/i8042/serio1/input/input10\n[ 13.071471] udevadm used greatest stack depth: 3984 bytes left\n[ 13.172129]\n[ 13.172130] \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\n[ 13.172425] [ INFO: possible circular locking dependency detected ]\n[ 13.172650] 2.6.39-rc6-mmotm0506 #1\n[ 13.172773] -------------------------------------------------------\n[ 13.172997] blkid/267 is trying to acquire lock:\n[ 13.173009] (\u0026p-\u003epi_lock){-.-.-.}, at: [\u003cffffffff81032d8f\u003e] try_to_wake_up+0x29/0x1aa\n[ 13.173009]\n[ 13.173009] but task is already holding lock:\n[ 13.173009] (rcu_node_level_0){..-...}, at: [\u003cffffffff810901cc\u003e] rcu_cpu_kthread_timer+0x27/0x58\n[ 13.173009]\n[ 13.173009] which lock already depends on the new lock.\n[ 13.173009]\n[ 13.173009]\n[ 13.173009] the existing dependency chain (in reverse order) is:\n[ 13.173009]\n[ 13.173009] -\u003e #2 (rcu_node_level_0){..-...}:\n[ 13.173009] [\u003cffffffff810679b9\u003e] check_prevs_add+0x8b/0x104\n[ 13.173009] [\u003cffffffff81067da1\u003e] validate_chain+0x36f/0x3ab\n[ 13.173009] [\u003cffffffff8106846b\u003e] __lock_acquire+0x369/0x3e2\n[ 13.173009] [\u003cffffffff81068a0f\u003e] lock_acquire+0xfc/0x14c\n[ 13.173009] [\u003cffffffff815697f1\u003e] _raw_spin_lock+0x36/0x45\n[ 13.173009] [\u003cffffffff81090794\u003e] rcu_read_unlock_special+0x8c/0x1d5\n[ 13.173009] [\u003cffffffff8109092c\u003e] __rcu_read_unlock+0x4f/0xd7\n[ 13.173009] [\u003cffffffff81027bd3\u003e] rcu_read_unlock+0x21/0x23\n[ 13.173009] [\u003cffffffff8102cc34\u003e] cpuacct_charge+0x6c/0x75\n[ 13.173009] [\u003cffffffff81030cc6\u003e] update_curr+0x101/0x12e\n[ 13.173009] [\u003cffffffff810311d0\u003e] check_preempt_wakeup+0xf7/0x23b\n[ 13.173009] [\u003cffffffff8102acb3\u003e] check_preempt_curr+0x2b/0x68\n[ 13.173009] [\u003cffffffff81031d40\u003e] ttwu_do_wakeup+0x76/0x128\n[ 13.173009] [\u003cffffffff81031e49\u003e] ttwu_do_activate.constprop.63+0x57/0x5c\n[ 13.173009] [\u003cffffffff81031e96\u003e] scheduler_ipi+0x48/0x5d\n[ 13.173009] [\u003cffffffff810177d5\u003e] smp_reschedule_interrupt+0x16/0x18\n[ 13.173009] [\u003cffffffff815710f3\u003e] reschedule_interrupt+0x13/0x20\n[ 13.173009] [\u003cffffffff810b66d1\u003e] rcu_read_unlock+0x21/0x23\n[ 13.173009] [\u003cffffffff810b739c\u003e] find_get_page+0xa9/0xb9\n[ 13.173009] [\u003cffffffff810b8b48\u003e] filemap_fault+0x6a/0x34d\n[ 13.173009] [\u003cffffffff810d1a25\u003e] __do_fault+0x54/0x3e6\n[ 13.173009] [\u003cffffffff810d447a\u003e] handle_pte_fault+0x12c/0x1ed\n[ 13.173009] [\u003cffffffff810d48f7\u003e] handle_mm_fault+0x1cd/0x1e0\n[ 13.173009] [\u003cffffffff8156cfee\u003e] do_page_fault+0x42d/0x5de\n[ 13.173009] [\u003cffffffff8156a75f\u003e] page_fault+0x1f/0x30\n[ 13.173009]\n[ 13.173009] -\u003e #1 (\u0026rq-\u003elock){-.-.-.}:\n[ 13.173009] [\u003cffffffff810679b9\u003e] check_prevs_add+0x8b/0x104\n[ 13.173009] [\u003cffffffff81067da1\u003e] validate_chain+0x36f/0x3ab\n[ 13.173009] [\u003cffffffff8106846b\u003e] __lock_acquire+0x369/0x3e2\n[ 13.173009] [\u003cffffffff81068a0f\u003e] lock_acquire+0xfc/0x14c\n[ 13.173009] [\u003cffffffff815697f1\u003e] _raw_spin_lock+0x36/0x45\n[ 13.173009] [\u003cffffffff81027e19\u003e] __task_rq_lock+0x8b/0xd3\n[ 13.173009] [\u003cffffffff81032f7f\u003e] wake_up_new_task+0x41/0x108\n[ 13.173009] [\u003cffffffff810376c3\u003e] do_fork+0x265/0x33f\n[ 13.173009] [\u003cffffffff81007d02\u003e] kernel_thread+0x6b/0x6d\n[ 13.173009] [\u003cffffffff8153a9dd\u003e] rest_init+0x21/0xd2\n[ 13.173009] [\u003cffffffff81b1db4f\u003e] start_kernel+0x3bb/0x3c6\n[ 13.173009] [\u003cffffffff81b1d29f\u003e] x86_64_start_reservations+0xaf/0xb3\n[ 13.173009] [\u003cffffffff81b1d393\u003e] x86_64_start_kernel+0xf0/0xf7\n[ 13.173009]\n[ 13.173009] -\u003e #0 (\u0026p-\u003epi_lock){-.-.-.}:\n[ 13.173009] [\u003cffffffff81067788\u003e] check_prev_add+0x68/0x20e\n[ 13.173009] [\u003cffffffff810679b9\u003e] check_prevs_add+0x8b/0x104\n[ 13.173009] [\u003cffffffff81067da1\u003e] validate_chain+0x36f/0x3ab\n[ 13.173009] [\u003cffffffff8106846b\u003e] __lock_acquire+0x369/0x3e2\n[ 13.173009] [\u003cffffffff81068a0f\u003e] lock_acquire+0xfc/0x14c\n[ 13.173009] [\u003cffffffff815698ea\u003e] _raw_spin_lock_irqsave+0x44/0x57\n[ 13.173009] [\u003cffffffff81032d8f\u003e] try_to_wake_up+0x29/0x1aa\n[ 13.173009] [\u003cffffffff81032f3c\u003e] wake_up_process+0x10/0x12\n[ 13.173009] [\u003cffffffff810901e9\u003e] rcu_cpu_kthread_timer+0x44/0x58\n[ 13.173009] [\u003cffffffff81045286\u003e] call_timer_fn+0xac/0x1e9\n[ 13.173009] [\u003cffffffff8104556d\u003e] run_timer_softirq+0x1aa/0x1f2\n[ 13.173009] [\u003cffffffff8103e487\u003e] __do_softirq+0x109/0x26a\n[ 13.173009] [\u003cffffffff8157144c\u003e] call_softirq+0x1c/0x30\n[ 13.173009] [\u003cffffffff81003207\u003e] do_softirq+0x44/0xf1\n[ 13.173009] [\u003cffffffff8103e8b9\u003e] irq_exit+0x58/0xc8\n[ 13.173009] [\u003cffffffff81017f5a\u003e] smp_apic_timer_interrupt+0x79/0x87\n[ 13.173009] [\u003cffffffff81570fd3\u003e] apic_timer_interrupt+0x13/0x20\n[ 13.173009] [\u003cffffffff810bd51a\u003e] get_page_from_freelist+0x2aa/0x310\n[ 13.173009] [\u003cffffffff810bdf03\u003e] __alloc_pages_nodemask+0x178/0x243\n[ 13.173009] [\u003cffffffff8101fe2f\u003e] pte_alloc_one+0x1e/0x3a\n[ 13.173009] [\u003cffffffff810d27fe\u003e] __pte_alloc+0x22/0x14b\n[ 13.173009] [\u003cffffffff810d48a8\u003e] handle_mm_fault+0x17e/0x1e0\n[ 13.173009] [\u003cffffffff8156cfee\u003e] do_page_fault+0x42d/0x5de\n[ 13.173009] [\u003cffffffff8156a75f\u003e] page_fault+0x1f/0x30\n[ 13.173009]\n[ 13.173009] other info that might help us debug this:\n[ 13.173009]\n[ 13.173009] Chain exists of:\n[ 13.173009] \u0026p-\u003epi_lock --\u003e \u0026rq-\u003elock --\u003e rcu_node_level_0\n[ 13.173009]\n[ 13.173009] Possible unsafe locking scenario:\n[ 13.173009]\n[ 13.173009] CPU0 CPU1\n[ 13.173009] ---- ----\n[ 13.173009] lock(rcu_node_level_0);\n[ 13.173009] lock(\u0026rq-\u003elock);\n[ 13.173009] lock(rcu_node_level_0);\n[ 13.173009] lock(\u0026p-\u003epi_lock);\n[ 13.173009]\n[ 13.173009] *** DEADLOCK ***\n[ 13.173009]\n[ 13.173009] 3 locks held by blkid/267:\n[ 13.173009] #0: (\u0026mm-\u003emmap_sem){++++++}, at: [\u003cffffffff8156cdb4\u003e] do_page_fault+0x1f3/0x5de\n[ 13.173009] #1: (\u0026yield_timer){+.-...}, at: [\u003cffffffff810451da\u003e] call_timer_fn+0x0/0x1e9\n[ 13.173009] #2: (rcu_node_level_0){..-...}, at: [\u003cffffffff810901cc\u003e] rcu_cpu_kthread_timer+0x27/0x58\n[ 13.173009]\n[ 13.173009] stack backtrace:\n[ 13.173009] Pid: 267, comm: blkid Not tainted 2.6.39-rc6-mmotm0506 #1\n[ 13.173009] Call Trace:\n[ 13.173009] \u003cIRQ\u003e [\u003cffffffff8154a529\u003e] print_circular_bug+0xc8/0xd9\n[ 13.173009] [\u003cffffffff81067788\u003e] check_prev_add+0x68/0x20e\n[ 13.173009] [\u003cffffffff8100c861\u003e] ? save_stack_trace+0x28/0x46\n[ 13.173009] [\u003cffffffff810679b9\u003e] check_prevs_add+0x8b/0x104\n[ 13.173009] [\u003cffffffff81067da1\u003e] validate_chain+0x36f/0x3ab\n[ 13.173009] [\u003cffffffff8106846b\u003e] __lock_acquire+0x369/0x3e2\n[ 13.173009] [\u003cffffffff81032d8f\u003e] ? try_to_wake_up+0x29/0x1aa\n[ 13.173009] [\u003cffffffff81068a0f\u003e] lock_acquire+0xfc/0x14c\n[ 13.173009] [\u003cffffffff81032d8f\u003e] ? try_to_wake_up+0x29/0x1aa\n[ 13.173009] [\u003cffffffff810901a5\u003e] ? rcu_check_quiescent_state+0x82/0x82\n[ 13.173009] [\u003cffffffff815698ea\u003e] _raw_spin_lock_irqsave+0x44/0x57\n[ 13.173009] [\u003cffffffff81032d8f\u003e] ? try_to_wake_up+0x29/0x1aa\n[ 13.173009] [\u003cffffffff81032d8f\u003e] try_to_wake_up+0x29/0x1aa\n[ 13.173009] [\u003cffffffff810901a5\u003e] ? rcu_check_quiescent_state+0x82/0x82\n[ 13.173009] [\u003cffffffff81032f3c\u003e] wake_up_process+0x10/0x12\n[ 13.173009] [\u003cffffffff810901e9\u003e] rcu_cpu_kthread_timer+0x44/0x58\n[ 13.173009] [\u003cffffffff810901a5\u003e] ? rcu_check_quiescent_state+0x82/0x82\n[ 13.173009] [\u003cffffffff81045286\u003e] call_timer_fn+0xac/0x1e9\n[ 13.173009] [\u003cffffffff810451da\u003e] ? del_timer+0x75/0x75\n[ 13.173009] [\u003cffffffff810901a5\u003e] ? rcu_check_quiescent_state+0x82/0x82\n[ 13.173009] [\u003cffffffff8104556d\u003e] run_timer_softirq+0x1aa/0x1f2\n[ 13.173009] [\u003cffffffff8103e487\u003e] __do_softirq+0x109/0x26a\n[ 13.173009] [\u003cffffffff8106365f\u003e] ? tick_dev_program_event+0x37/0xf6\n[ 13.173009] [\u003cffffffff810a0e4a\u003e] ? time_hardirqs_off+0x1b/0x2f\n[ 13.173009] [\u003cffffffff8157144c\u003e] call_softirq+0x1c/0x30\n[ 13.173009] [\u003cffffffff81003207\u003e] do_softirq+0x44/0xf1\n[ 13.173009] [\u003cffffffff8103e8b9\u003e] irq_exit+0x58/0xc8\n[ 13.173009] [\u003cffffffff81017f5a\u003e] smp_apic_timer_interrupt+0x79/0x87\n[ 13.173009] [\u003cffffffff81570fd3\u003e] apic_timer_interrupt+0x13/0x20\n[ 13.173009] \u003cEOI\u003e [\u003cffffffff810bd384\u003e] ? get_page_from_freelist+0x114/0x310\n[ 13.173009] [\u003cffffffff810bd51a\u003e] ? get_page_from_freelist+0x2aa/0x310\n[ 13.173009] [\u003cffffffff812220e7\u003e] ? clear_page_c+0x7/0x10\n[ 13.173009] [\u003cffffffff810bd1ef\u003e] ? prep_new_page+0x14c/0x1cd\n[ 13.173009] [\u003cffffffff810bd51a\u003e] get_page_from_freelist+0x2aa/0x310\n[ 13.173009] [\u003cffffffff810bdf03\u003e] __alloc_pages_nodemask+0x178/0x243\n[ 13.173009] [\u003cffffffff810d46b9\u003e] ? __pmd_alloc+0x87/0x99\n[ 13.173009] [\u003cffffffff8101fe2f\u003e] pte_alloc_one+0x1e/0x3a\n[ 13.173009] [\u003cffffffff810d46b9\u003e] ? __pmd_alloc+0x87/0x99\n[ 13.173009] [\u003cffffffff810d27fe\u003e] __pte_alloc+0x22/0x14b\n[ 13.173009] [\u003cffffffff810d48a8\u003e] handle_mm_fault+0x17e/0x1e0\n[ 13.173009] [\u003cffffffff8156cfee\u003e] do_page_fault+0x42d/0x5de\n[ 13.173009] [\u003cffffffff810d915f\u003e] ? sys_brk+0x32/0x10c\n[ 13.173009] [\u003cffffffff810a0e4a\u003e] ? time_hardirqs_off+0x1b/0x2f\n[ 13.173009] [\u003cffffffff81065c4f\u003e] ? trace_hardirqs_off_caller+0x3f/0x9c\n[ 13.173009] [\u003cffffffff812235dd\u003e] ? trace_hardirqs_off_thunk+0x3a/0x3c\n[ 13.173009] [\u003cffffffff8156a75f\u003e] page_fault+0x1f/0x30\n[ 14.010075] usb 5-1: new full speed USB device number 2 using uhci_hcd\n\nReported-by: Valdis Kletnieks \u003cValdis.Kletnieks@vt.edu\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "29f742f88a32c9ab8cf6d9ba69e1ea918be5aa58", "tree": "a38aa38c8025e050ec82a7e64d02dca07f90ffc7", "parents": [ "f62508f68d04adefc4cf9b0177ba02c8818b3eec", "23b5c8fa01b723c70a20d6e4ef4ff54c7656d6e1" ], "author": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Fri May 27 12:38:52 2011 +0200" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Sat May 28 17:41:05 2011 +0200" }, "message": "Merge branch \u0027rcu/urgent\u0027 of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu into core/urgent\n" }, { "commit": "23b5c8fa01b723c70a20d6e4ef4ff54c7656d6e1", "tree": "d03faad5e19848b35a019793b9a1cbc0bb68a708", "parents": [ "4305ce7894dd38b0633bfc8978437320119223bd" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Sep 07 10:38:22 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 09:42:23 2011 -0700" }, "message": "rcu: Decrease memory-barrier usage based on semi-formal proof\n\n(Note: this was reverted, and is now being re-applied in pieces, with\nthis being the fifth and final piece. See below for the reason that\nit is now felt to be safe to re-apply this.)\n\nCommit d09b62d fixed grace-period synchronization, but left some smp_mb()\ninvocations in rcu_process_callbacks() that are no longer needed, but\nsheer paranoia prevented them from being removed. This commit removes\nthem and provides a proof of correctness in their absence. It also adds\na memory barrier to rcu_report_qs_rsp() immediately before the update to\nrsp-\u003ecompleted in order to handle the theoretical possibility that the\ncompiler or CPU might move massive quantities of code into a lock-based\ncritical section. This also proves that the sheer paranoia was not\nentirely unjustified, at least from a theoretical point of view.\n\nIn addition, the old dyntick-idle synchronization depended on the fact\nthat grace periods were many milliseconds in duration, so that it could\nbe assumed that no dyntick-idle CPU could reorder a memory reference\nacross an entire grace period. Unfortunately for this design, the\naddition of expedited grace periods breaks this assumption, which has\nthe unfortunate side-effect of requiring atomic operations in the\nfunctions that track dyntick-idle state for RCU. (There is some hope\nthat the algorithms used in user-level RCU might be applied here, but\nsome work is required to handle the NMIs that user-space applications\ncan happily ignore. For the short term, better safe than sorry.)\n\nThis proof assumes that neither compiler nor CPU will allow a lock\nacquisition and release to be reordered, as doing so can result in\ndeadlock. The proof is as follows:\n\n1.\tA given CPU declares a quiescent state under the protection of\n\tits leaf rcu_node\u0027s lock.\n\n2.\tIf there is more than one level of rcu_node hierarchy, the\n\tlast CPU to declare a quiescent state will also acquire the\n\t-\u003elock of the next rcu_node up in the hierarchy, but only\n\tafter releasing the lower level\u0027s lock. The acquisition of this\n\tlock clearly cannot occur prior to the acquisition of the leaf\n\tnode\u0027s lock.\n\n3.\tStep 2 repeats until we reach the root rcu_node structure.\n\tPlease note again that only one lock is held at a time through\n\tthis process. The acquisition of the root rcu_node\u0027s -\u003elock\n\tmust occur after the release of that of the leaf rcu_node.\n\n4.\tAt this point, we set the -\u003ecompleted field in the rcu_state\n\tstructure in rcu_report_qs_rsp(). However, if the rcu_node\n\thierarchy contains only one rcu_node, then in theory the code\n\tpreceding the quiescent state could leak into the critical\n\tsection. We therefore precede the update of -\u003ecompleted with a\n\tmemory barrier. All CPUs will therefore agree that any updates\n\tpreceding any report of a quiescent state will have happened\n\tbefore the update of -\u003ecompleted.\n\n5.\tRegardless of whether a new grace period is needed, rcu_start_gp()\n\twill propagate the new value of -\u003ecompleted to all of the leaf\n\trcu_node structures, under the protection of each rcu_node\u0027s -\u003elock.\n\tIf a new grace period is needed immediately, this propagation\n\twill occur in the same critical section that -\u003ecompleted was\n\tset in, but courtesy of the memory barrier in #4 above, is still\n\tseen to follow any pre-quiescent-state activity.\n\n6.\tWhen a given CPU invokes __rcu_process_gp_end(), it becomes\n\taware of the end of the old grace period and therefore makes\n\tany RCU callbacks that were waiting on that grace period eligible\n\tfor invocation.\n\n\tIf this CPU is the same one that detected the end of the grace\n\tperiod, and if there is but a single rcu_node in the hierarchy,\n\twe will still be in the single critical section. In this case,\n\tthe memory barrier in step #4 guarantees that all callbacks will\n\tbe seen to execute after each CPU\u0027s quiescent state.\n\n\tOn the other hand, if this is a different CPU, it will acquire\n\tthe leaf rcu_node\u0027s -\u003elock, and will again be serialized after\n\teach CPU\u0027s quiescent state for the old grace period.\n\nOn the strength of this proof, this commit therefore removes the memory\nbarriers from rcu_process_callbacks() and adds one to rcu_report_qs_rsp().\nThe effect is to reduce the number of memory barriers by one and to\nreduce the frequency of execution from about once per scheduling tick\nper CPU to once per grace period.\n\nThis was reverted do to hangs found during testing by Yinghai Lu and\nIngo Molnar. Frederic Weisbecker supplied Yinghai with tracing that\nlocated the underlying problem, and Frederic also provided the fix.\n\nThe underlying problem was that the HARDIRQ_ENTER() macro from\nlib/locking-selftest.c invoked irq_enter(), which in turn invokes\nrcu_irq_enter(), but HARDIRQ_EXIT() invoked __irq_exit(), which\ndoes not invoke rcu_irq_exit(). This situation resulted in calls\nto rcu_irq_enter() that were not balanced by the required calls to\nrcu_irq_exit(). Therefore, after these locking selftests completed,\nRCU\u0027s dyntick-idle nesting count was a large number (for example,\n72), which caused RCU to to conclude that the affected CPU was not in\ndyntick-idle mode when in fact it was.\n\nRCU would therefore incorrectly wait for this dyntick-idle CPU, resulting\nin hangs.\n\nIn contrast, with Frederic\u0027s patch, which replaces the irq_enter()\nin HARDIRQ_ENTER() with an __irq_enter(), these tests don\u0027t ever call\neither rcu_irq_enter() or rcu_irq_exit(), which works because the CPU\nrunning the test is already marked as not being in dyntick-idle mode.\nThis means that the rcu_irq_enter() and rcu_irq_exit() calls and RCU\nthen has no problem working out which CPUs are in dyntick-idle mode and\nwhich are not.\n\nThe reason that the imbalance was not noticed before the barrier patch\nwas applied is that the old implementation of rcu_enter_nohz() ignored\nthe nesting depth. This could still result in delays, but much shorter\nones. Whenever there was a delay, RCU would IPI the CPU with the\nunbalanced nesting level, which would eventually result in rcu_enter_nohz()\nbeing called, which in turn would force RCU to see that the CPU was in\ndyntick-idle mode.\n\nThe reason that very few people noticed the problem is that the mismatched\nirq_enter() vs. __irq_exit() occured only when the kernel was built with\nCONFIG_DEBUG_LOCKING_API_SELFTESTS.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "4305ce7894dd38b0633bfc8978437320119223bd", "tree": "eafe4497236f8dd4ddfdf90fe06a5377912a9c35", "parents": [ "b5904090c754327ed6c2ecaefed4f7d473df393f" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 16 14:27:31 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 09:42:22 2011 -0700" }, "message": "rcu: Make rcu_enter_nohz() pay attention to nesting\n\nThe old version of rcu_enter_nohz() forced RCU into nohz mode even if\nthe nesting count was non-zero. This change causes rcu_enter_nohz()\nto hold off for non-zero nesting counts.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "b5904090c754327ed6c2ecaefed4f7d473df393f", "tree": "46cf53f98413171d92de3b5e11c9327309f240b7", "parents": [ "1135633bddcf7a819a1490c18d04965c490bcc1e" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 16 02:52:04 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 09:42:21 2011 -0700" }, "message": "rcu: Don\u0027t do reschedule unless in irq\n\nCondition the set_need_resched() in rcu_irq_exit() on in_irq(). This\nshould be a no-op, because rcu_irq_exit() should only be called from irq.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "1135633bddcf7a819a1490c18d04965c490bcc1e", "tree": "99facc2b3416a7c00cf042d2f782a87859e30d26", "parents": [ "0bbcc529fcea9c7de5e2e7243f9913b8f7302a8c" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 16 02:44:06 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 09:42:21 2011 -0700" }, "message": "rcu: Remove old memory barriers from rcu_process_callbacks()\n\nSecond step of partitioning of commit e59fb3120b.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "0bbcc529fcea9c7de5e2e7243f9913b8f7302a8c", "tree": "22403b7c9bf3b87818c3a8a606a1bc33dd12bd1a", "parents": [ "ba9f207c9f82115aba4ce04b22e0081af0ae300f" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 16 02:24:04 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 26 09:42:20 2011 -0700" }, "message": "rcu: Add memory barriers\n\nAdd the memory barriers added by e59fb3120b.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "268bb0ce3e87872cb9290c322b0d35bce230d88f", "tree": "c8331ade4a3e24fc589c4eb62731bc2312d35333", "parents": [ "257313b2a87795e07a0bdf58d0fffbdba8b31051" ], "author": { "name": "Linus Torvalds", "email": "torvalds@linux-foundation.org", "time": "Fri May 20 12:50:29 2011 -0700" }, "committer": { "name": "Linus Torvalds", "email": "torvalds@linux-foundation.org", "time": "Fri May 20 12:50:29 2011 -0700" }, "message": "sanitize \u003clinux/prefetch.h\u003e usage\n\nCommit e66eed651fd1 (\"list: remove prefetching from regular list\niterators\") removed the include of prefetch.h from list.h, which\nuncovered several cases that had apparently relied on that rather\nobscure header file dependency.\n\nSo this fixes things up a bit, using\n\n grep -L linux/prefetch.h $(git grep -l \u0027[^a-z_]prefetchw*(\u0027 -- \u0027*.[ch]\u0027)\n grep -L \u0027prefetchw*(\u0027 $(git grep -l \u0027linux/prefetch.h\u0027 -- \u0027*.[ch]\u0027)\n\nto guide us in finding files that either need \u003clinux/prefetch.h\u003e\ninclusion, or have it despite not needing it.\n\nThere are more of them around (mostly network drivers), but this gets\nmany core ones.\n\nReported-by: Stephen Rothwell \u003csfr@canb.auug.org.au\u003e\nSigned-off-by: Linus Torvalds \u003ctorvalds@linux-foundation.org\u003e\n" }, { "commit": "80d02085d99039b3b7f3a73c8896226b0cb1ba07", "tree": "c310902423ecb00effadcb59c60cbf118d4037cb", "parents": [ "11c476f31a0fabc6e604da5b09a6590b57c3fb20" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 12 01:08:07 2011 -0700" }, "committer": { "name": "Ingo Molnar", "email": "mingo@elte.hu", "time": "Thu May 19 23:25:29 2011 +0200" }, "message": "Revert \"rcu: Decrease memory-barrier usage based on semi-formal proof\"\n\nThis reverts commit e59fb3120becfb36b22ddb8bd27d065d3cdca499.\n\nThis reversion was due to (extreme) boot-time slowdowns on SPARC seen by\nYinghai Lu and on x86 by Ingo\n.\nThis is a non-trivial reversion due to intervening commits.\n\nConflicts:\n\n\tDocumentation/RCU/trace.txt\n\tkernel/rcutree.c\n\nSigned-off-by: Ingo Molnar \u003cmingo@elte.hu\u003e\n" }, { "commit": "1217ed1ba5c67393293dfb0f03c353b118dadeb4", "tree": "a765356c8418e134de85fd05d9fe6eda41de859c", "parents": [ "29ce831000081dd757d3116bf774aafffc4b6b20" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed May 04 21:43:49 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Sat May 07 22:50:45 2011 -0700" }, "message": "rcu: permit rcu_read_unlock() to be called while holding runqueue locks\n\nAvoid calling into the scheduler while holding core RCU locks. This\nallows rcu_read_unlock() to be called while holding the runqueue locks,\nbut only as long as there was no chance of the RCU read-side critical\nsection having been preempted. (Otherwise, if RCU priority boosting\nis enabled, rcu_read_unlock() might call into the scheduler in order to\nunboost itself, which might allows self-deadlock on the runqueue locks\nwithin the scheduler.)\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "29ce831000081dd757d3116bf774aafffc4b6b20", "tree": "0b992d993b02d3479a150e9fc508d110dc45631d", "parents": [ "bad6e1393cb505fe17747344a23666464daa3fa7" ], "author": { "name": "Gleb Natapov", "email": "gleb@redhat.com", "time": "Wed May 04 16:31:03 2011 +0300" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:59 2011 -0700" }, "message": "rcu: provide rcu_virt_note_context_switch() function.\n\nProvide rcu_virt_note_context_switch() for vitalization use to note\nquiescent state during guest entry.\n\nSigned-off-by: Gleb Natapov \u003cgleb@redhat.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "bad6e1393cb505fe17747344a23666464daa3fa7", "tree": "521dcada78d0f19400d14547381f0f3b2e38fea2", "parents": [ "b554d7de8d112fca4188da3bf0d7f8b56f42fb95" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 02 23:40:04 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:59 2011 -0700" }, "message": "rcu: get rid of signed overflow in check_cpu_stall()\n\nSigned integer overflow is undefined by the C standard, so move\ncalculations to unsigned.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "2655d57ef35aa327a2e58a1c5dc7b65c65003f4e", "tree": "4a71e82b23ee84705cad15cd16490f8c5aace74f", "parents": [ "baa1ae0c9f1c618bc60706efa75fef3508bcee58" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Apr 07 22:47:23 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:59 2011 -0700" }, "message": "rcu: prevent call_rcu() from diving into rcu core if irqs disabled\n\nThis commit marks a first step towards making call_rcu() have\nreal-time behavior. If irqs are disabled, don\u0027t dive into the\nRCU core. Later on, this new early exit will wake up the\nper-CPU kthread, which first must be modified to handle the\ncases involving callback storms.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "baa1ae0c9f1c618bc60706efa75fef3508bcee58", "tree": "e9a4558a3fcef41e42477d6461ccf655d01d5432", "parents": [ "9ab1544eb4196ca8d05c433b2eb56f74496b1ee3" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sat Mar 26 22:01:35 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:59 2011 -0700" }, "message": "rcu: further lower priority in rcu_yield()\n\nAlthough rcu_yield() dropped from real-time to normal priority, there\nis always the possibility that the competing tasks have been niced.\nSo nice to 19 in rcu_yield() to help ensure that other tasks have a\nbetter chance of running.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "9ab1544eb4196ca8d05c433b2eb56f74496b1ee3", "tree": "be3a7897cf52920df4da41ded060e23150bdb849", "parents": [ "6cc68793e380bb51f447d8d02af873b7bc01f222" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Fri Mar 18 11:15:47 2011 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:59 2011 -0700" }, "message": "rcu: introduce kfree_rcu()\n\nMany rcu callbacks functions just call kfree() on the base structure.\nThese functions are trivial, but their size adds up, and furthermore\nwhen they are used in a kernel module, that module must invoke the\nhigh-latency rcu_barrier() function at module-unload time.\n\nThe kfree_rcu() function introduced by this commit addresses this issue.\nRather than encoding a function address in the embedded rcu_head\nstructure, kfree_rcu() instead encodes the offset of the rcu_head\nstructure within the base structure. Because the functions are not\nallowed in the low-order 4096 bytes of kernel virtual memory, offsets\nup to 4095 bytes can be accommodated. If the offset is larger than\n4095 bytes, a compile-time error will be generated in __kfree_rcu().\nIf this error is triggered, you can either fall back to use of call_rcu()\nor rearrange the structure to position the rcu_head structure into the\nfirst 4096 bytes.\n\nNote that the allowable offset might decrease in the future, for example,\nto allow something like kmem_cache_free_rcu().\n\nThe new kfree_rcu() function can replace code as follows:\n\n\tcall_rcu(\u0026p-\u003ercu, simple_kfree_callback);\n\nwhere \"simple_kfree_callback()\" might be defined as follows:\n\n\tvoid simple_kfree_callback(struct rcu_head *p)\n\t{\n\t\tstruct foo *q \u003d container_of(p, struct foo, rcu);\n\n\t\tkfree(q);\n\t}\n\nwith the following:\n\n\tkfree_rcu(\u0026p-\u003ercu, rcu);\n\nNote that the \"rcu\" is the name of a field in the structure being\nfreed. The reason for using this rather than passing in a pointer\nto the base structure is that the above approach allows better type\nchecking.\n\nThis commit is based on earlier work by Lai Jiangshan and Manfred Spraul:\n\nLai\u0027s V1 patch: http://lkml.org/lkml/2008/9/18/1\nManfred\u0027s patch: http://lkml.org/lkml/2009/1/2/115\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Manfred Spraul \u003cmanfred@colorfullife.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: David Howells \u003cdhowells@redhat.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "6cc68793e380bb51f447d8d02af873b7bc01f222", "tree": "f2b66ace23acc0a31a08f136ad25659d6b51c4b1", "parents": [ "13491a0ee1ef862b6c842132b6eb9c5e721af5ad" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Mar 02 13:15:15 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:59 2011 -0700" }, "message": "rcu: fix spelling\n\nThe \"preemptible\" spelling is preferable. May as well fix it.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "f0a07aeaf8935b7e9ef8032ce6546169f143951c", "tree": "31c6ee98ebb7e3040ad036415702eb23ff929e01", "parents": [ "108aae22339f445c134aeb48eca25df1014ab08d" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Feb 23 11:10:52 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:57 2011 -0700" }, "message": "rcu: Switch to this_cpu() primitives\n\nThis removes a couple of lines from invoke_rcu_cpu_kthread(), improving\nreadability.\n\nReported-by: Christoph Lameter \u003ccl@linux.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "5ece5bab3ed8594ce2c85c6c6e6b82109db36ca7", "tree": "c9ef8faa5dbb9e7c82893657e5aa2c1040987f96", "parents": [ "15ba0ba860871cf74b48b1bb47c26c91a66126f3" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Fri Apr 22 18:08:51 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:57 2011 -0700" }, "message": "rcu: Add forward-progress diagnostic for per-CPU kthreads\n\nIncrement a per-CPU counter on each pass through rcu_cpu_kthread()\u0027s\nservice loop, and add it to the rcudata trace output.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "15ba0ba860871cf74b48b1bb47c26c91a66126f3", "tree": "2043eeca7d6df62fc0ae918b61abada073f81415", "parents": [ "a9f4793d8900dc5dc09b3951bdcd4731290e06fe" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Apr 06 16:01:16 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:56 2011 -0700" }, "message": "rcu: add grace-period age and more kthread state to tracing\n\nThis commit adds the age in jiffies of the current grace period along\nwith the duration in jiffies of the longest grace period since boot\nto the rcu/rcugp debugfs file. It also adds an additional \"O\" state\nto kthread tracing to differentiate between the kthread waiting due to\nhaving nothing to do on the one hand and waiting due to being on the\nwrong CPU on the other hand.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "4a29865689dbb87a02e3b0fff4a4ae5041273173", "tree": "72c9f1641a9fb05e3ae4d8cc068f62f9c566bae9", "parents": [ "d71df90eadfc35aa549ff9a850842673febca71f" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Sun Apr 03 21:33:51 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:56 2011 -0700" }, "message": "rcu: make rcutorture version numbers available through debugfs\n\nIt is not possible to accurately correlate rcutorture output with that\nof debugfs. This patch therefore adds a debugfs file that prints out\nthe rcutorture version number, permitting easy correlation.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "d71df90eadfc35aa549ff9a850842673febca71f", "tree": "dd275a3f4848784bc6625574341c11befc1b3823", "parents": [ "0ac3d136b2e3cdf1161178223bc5da14a06241d0" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Tue Mar 29 17:48:28 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:56 2011 -0700" }, "message": "rcu: add tracing for RCU\u0027s kthread run states.\n\nAdd tracing to help debugging situations when RCU\u0027s kthreads are not\nrunning but are supposed to be.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "e3995a25fa361ce987a7d0ade00b17e3151519d7", "tree": "ccf8a6ae71bc5cb289080b04f00170a758e33c59", "parents": [ "0f962a5e7277c34987b77dc82fc9aefcedc95e27" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Apr 18 15:31:26 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:55 2011 -0700" }, "message": "rcu: put per-CPU kthread at non-RT priority during CPU hotplug operations\n\nIf you are doing CPU hotplug operations, it is best not to have\nCPU-bound realtime tasks running CPU-bound on the outgoing CPU.\nSo this commit makes per-CPU kthreads run at non-realtime priority\nduring that time.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "0f962a5e7277c34987b77dc82fc9aefcedc95e27", "tree": "b265d93dcc8f2669f8fb9b353344b9f6cc354728", "parents": [ "27f4d28057adf98750cf863c40baefb12f5b6d21" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Thu Apr 14 12:13:53 2011 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:55 2011 -0700" }, "message": "rcu: Force per-rcu_node kthreads off of the outgoing CPU\n\nThe scheduler has had some heartburn in the past when too many real-time\nkthreads were affinitied to the outgoing CPU. So, this commit lightens\nthe load by forcing the per-rcu_node and the boost kthreads off of the\noutgoing CPU. Note that RCU\u0027s per-CPU kthread remains on the outgoing\nCPU until the bitter end, as it must in order to preserve correctness.\n\nAlso avoid disabling hardirqs across calls to set_cpus_allowed_ptr(),\ngiven that this function can block.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "27f4d28057adf98750cf863c40baefb12f5b6d21", "tree": "a66584d2cb68cebda327313f78b8f4eb602ed4ef", "parents": [ "a26ac2455ffcf3be5c6ef92bc6df7182700f2114" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Feb 07 12:47:15 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:55 2011 -0700" }, "message": "rcu: priority boosting for TREE_PREEMPT_RCU\n\nAdd priority boosting for TREE_PREEMPT_RCU, similar to that for\nTINY_PREEMPT_RCU. This is enabled by the default-off RCU_BOOST\nkernel parameter. The priority to which to boost preempted\nRCU readers is controlled by the RCU_BOOST_PRIO kernel parameter\n(defaulting to real-time priority 1) and the time to wait before\nboosting the readers who are blocking a given grace period is\ncontrolled by the RCU_BOOST_DELAY kernel parameter (defaulting to\n500 milliseconds).\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "a26ac2455ffcf3be5c6ef92bc6df7182700f2114", "tree": "601697c7c3fd152a1b3f29f43d3a028fc8fefd5f", "parents": [ "12f5f524cafef3ab689929b118f2dfb8bf2be321" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Wed Jan 12 14:10:23 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: move TREE_RCU from softirq to kthread\n\nIf RCU priority boosting is to be meaningful, callback invocation must\nbe boosted in addition to preempted RCU readers. Otherwise, in presence\nof CPU real-time threads, the grace period ends, but the callbacks don\u0027t\nget invoked. If the callbacks don\u0027t get invoked, the associated memory\ndoesn\u0027t get freed, so the system is still subject to OOM.\n\nBut it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit\nmoves the callback invocations to a kthread, which can be boosted easily.\n\nAlso add comments and properly synchronized all accesses to\nrcu_cpu_kthread_task, as suggested by Lai Jiangshan.\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "12f5f524cafef3ab689929b118f2dfb8bf2be321", "tree": "639473556b6edf9b79e0a18d5ba58f80eea76519", "parents": [ "e59fb3120becfb36b22ddb8bd27d065d3cdca499" ], "author": { "name": "Paul E. McKenney", "email": "paul.mckenney@linaro.org", "time": "Mon Nov 29 21:56:39 2010 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: merge TREE_PREEPT_RCU blocked_tasks[] lists\n\nCombine the current TREE_PREEMPT_RCU -\u003eblocked_tasks[] lists in the\nrcu_node structure into a single -\u003eblkd_tasks list with -\u003egp_tasks\nand -\u003eexp_tasks tail pointers. This is in preparation for RCU priority\nboosting, which will add a third dimension to the combinatorial explosion\nin the -\u003eblocked_tasks[] case, but simply a third pointer in the new\n-\u003eblkd_tasks case.\n\nAlso update documentation to reflect blocked_tasks[] merge\n\nSigned-off-by: Paul E. McKenney \u003cpaul.mckenney@linaro.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "e59fb3120becfb36b22ddb8bd27d065d3cdca499", "tree": "37eaadfe112b64caae943fc7469274bc96553d92", "parents": [ "a00e0d714fbded07a7a2254391ce9ed5a5cb9d82" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Sep 07 10:38:22 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: Decrease memory-barrier usage based on semi-formal proof\n\nCommit d09b62d fixed grace-period synchronization, but left some smp_mb()\ninvocations in rcu_process_callbacks() that are no longer needed, but\nsheer paranoia prevented them from being removed. This commit removes\nthem and provides a proof of correctness in their absence. It also adds\na memory barrier to rcu_report_qs_rsp() immediately before the update to\nrsp-\u003ecompleted in order to handle the theoretical possibility that the\ncompiler or CPU might move massive quantities of code into a lock-based\ncritical section. This also proves that the sheer paranoia was not\nentirely unjustified, at least from a theoretical point of view.\n\nIn addition, the old dyntick-idle synchronization depended on the fact\nthat grace periods were many milliseconds in duration, so that it could\nbe assumed that no dyntick-idle CPU could reorder a memory reference\nacross an entire grace period. Unfortunately for this design, the\naddition of expedited grace periods breaks this assumption, which has\nthe unfortunate side-effect of requiring atomic operations in the\nfunctions that track dyntick-idle state for RCU. (There is some hope\nthat the algorithms used in user-level RCU might be applied here, but\nsome work is required to handle the NMIs that user-space applications\ncan happily ignore. For the short term, better safe than sorry.)\n\nThis proof assumes that neither compiler nor CPU will allow a lock\nacquisition and release to be reordered, as doing so can result in\ndeadlock. The proof is as follows:\n\n1.\tA given CPU declares a quiescent state under the protection of\n\tits leaf rcu_node\u0027s lock.\n\n2.\tIf there is more than one level of rcu_node hierarchy, the\n\tlast CPU to declare a quiescent state will also acquire the\n\t-\u003elock of the next rcu_node up in the hierarchy, but only\n\tafter releasing the lower level\u0027s lock. The acquisition of this\n\tlock clearly cannot occur prior to the acquisition of the leaf\n\tnode\u0027s lock.\n\n3.\tStep 2 repeats until we reach the root rcu_node structure.\n\tPlease note again that only one lock is held at a time through\n\tthis process. The acquisition of the root rcu_node\u0027s -\u003elock\n\tmust occur after the release of that of the leaf rcu_node.\n\n4.\tAt this point, we set the -\u003ecompleted field in the rcu_state\n\tstructure in rcu_report_qs_rsp(). However, if the rcu_node\n\thierarchy contains only one rcu_node, then in theory the code\n\tpreceding the quiescent state could leak into the critical\n\tsection. We therefore precede the update of -\u003ecompleted with a\n\tmemory barrier. All CPUs will therefore agree that any updates\n\tpreceding any report of a quiescent state will have happened\n\tbefore the update of -\u003ecompleted.\n\n5.\tRegardless of whether a new grace period is needed, rcu_start_gp()\n\twill propagate the new value of -\u003ecompleted to all of the leaf\n\trcu_node structures, under the protection of each rcu_node\u0027s -\u003elock.\n\tIf a new grace period is needed immediately, this propagation\n\twill occur in the same critical section that -\u003ecompleted was\n\tset in, but courtesy of the memory barrier in #4 above, is still\n\tseen to follow any pre-quiescent-state activity.\n\n6.\tWhen a given CPU invokes __rcu_process_gp_end(), it becomes\n\taware of the end of the old grace period and therefore makes\n\tany RCU callbacks that were waiting on that grace period eligible\n\tfor invocation.\n\n\tIf this CPU is the same one that detected the end of the grace\n\tperiod, and if there is but a single rcu_node in the hierarchy,\n\twe will still be in the single critical section. In this case,\n\tthe memory barrier in step #4 guarantees that all callbacks will\n\tbe seen to execute after each CPU\u0027s quiescent state.\n\n\tOn the other hand, if this is a different CPU, it will acquire\n\tthe leaf rcu_node\u0027s -\u003elock, and will again be serialized after\n\teach CPU\u0027s quiescent state for the old grace period.\n\nOn the strength of this proof, this commit therefore removes the memory\nbarriers from rcu_process_callbacks() and adds one to rcu_report_qs_rsp().\nThe effect is to reduce the number of memory barriers by one and to\nreduce the frequency of execution from about once per scheduling tick\nper CPU to once per grace period.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "a00e0d714fbded07a7a2254391ce9ed5a5cb9d82", "tree": "7c213dbf1e2561655d07749b6d689027460fb8cc", "parents": [ "0ee5623f9a6e52df90a78bd21179f8ab370e102e" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Feb 08 17:14:39 2011 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu May 05 23:16:54 2011 -0700" }, "message": "rcu: Remove conditional compilation for RCU CPU stall warnings\n\nThe RCU CPU stall warnings can now be controlled using the\nrcu_cpu_stall_suppress boot-time parameter or via the same parameter\nfrom sysfs. There is therefore no longer any reason to have\nkernel config parameters for this feature. This commit therefore\nremoves the RCU_CPU_STALL_DETECTOR and RCU_CPU_STALL_DETECTOR_RUNNABLE\nkernel config parameters. The RCU_CPU_STALL_TIMEOUT parameter remains\nto allow the timeout to be tuned and the RCU_CPU_STALL_VERBOSE parameter\nremains to allow task-stall information to be suppressed if desired.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "72eb6a791459c87a0340318840bb3bd9252b627b", "tree": "3bfb8ad99f9c7e511f37f72d57b56a2cea06d753", "parents": [ "23d69b09b78c4876e134f104a3814c30747c53f1", "55ee4ef30241a62b700f79517e6d5ef2ddbefa67" ], "author": { "name": "Linus Torvalds", "email": "torvalds@linux-foundation.org", "time": "Fri Jan 07 17:02:58 2011 -0800" }, "committer": { "name": "Linus Torvalds", "email": "torvalds@linux-foundation.org", "time": "Fri Jan 07 17:02:58 2011 -0800" }, "message": "Merge branch \u0027for-2.6.38\u0027 of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu\n\n* \u0027for-2.6.38\u0027 of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu: (30 commits)\n gameport: use this_cpu_read instead of lookup\n x86: udelay: Use this_cpu_read to avoid address calculation\n x86: Use this_cpu_inc_return for nmi counter\n x86: Replace uses of current_cpu_data with this_cpu ops\n x86: Use this_cpu_ops to optimize code\n vmstat: User per cpu atomics to avoid interrupt disable / enable\n irq_work: Use per cpu atomics instead of regular atomics\n cpuops: Use cmpxchg for xchg to avoid lock semantics\n x86: this_cpu_cmpxchg and this_cpu_xchg operations\n percpu: Generic this_cpu_cmpxchg() and this_cpu_xchg support\n percpu,x86: relocate this_cpu_add_return() and friends\n connector: Use this_cpu operations\n xen: Use this_cpu_inc_return\n taskstats: Use this_cpu_ops\n random: Use this_cpu_inc_return\n fs: Use this_cpu_inc_return in buffer.c\n highmem: Use this_cpu_xx_return() operations\n vmstat: Use this_cpu_inc_return for vm statistics\n x86: Support for this_cpu_add, sub, dec, inc_return\n percpu: Generic support for this_cpu_add, sub, dec, inc_return\n ...\n\nFixed up conflicts: in arch/x86/kernel/{apic/nmi.c, apic/x2apic_uv_x.c, process.c}\nas per Tejun.\n" }, { "commit": "b52573d2796274f7f31cfeff7185c320adcd4f12", "tree": "61580632fa8fbf80225ba7339274ada0c013a01b", "parents": [ "0209f6490b030f35349a2bb71294f3fd75b0f36d" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Dec 14 17:36:02 2010 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 17 12:34:49 2010 -0800" }, "message": "rcu: reduce __call_rcu()-induced contention on rcu_node structures\n\nWhen the current __call_rcu() function was written, the expedited\nAPIs did not exist. The __call_rcu() implementation therefore went\nto great lengths to detect the end of old grace periods and to start\nnew ones, all in the name of reducing grace-period latency. Now the\nexpedited APIs do exist, and the usage of __call_rcu() has increased\nconsiderably. This commit therefore causes __call_rcu() to avoid\nworrying about grace periods unless there are a large number of\nRCU callbacks stacked up on the current CPU.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "0209f6490b030f35349a2bb71294f3fd75b0f36d", "tree": "159e233c6fcaf63fbb1a18f5db54af38a8336ee3", "parents": [ "121dfc4b3eba9e2f3c42d35205a3510cc65b9931" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Dec 14 16:07:52 2010 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 17 12:34:20 2010 -0800" }, "message": "rcu: limit rcu_node leaf-level fanout\n\nSome recent benchmarks have indicated possible lock contention on the\nleaf-level rcu_node locks. This commit therefore limits the number of\nCPUs per leaf-level rcu_node structure to 16, in other words, there\ncan be at most 16 rcu_data structures fanning into a given rcu_node\nstructure. Prior to this, the limit was 32 on 32-bit systems and 64 on\n64-bit systems.\n\nNote that the fanout of non-leaf rcu_node structures is unchanged. The\norganization of accesses to the rcu_node tree is such that references\nto non-leaf rcu_node structures are much less frequent than to the\nleaf structures.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "121dfc4b3eba9e2f3c42d35205a3510cc65b9931", "tree": "bd132a839cf3618f1d7602efbb1e144a8e04cec4", "parents": [ "5ff8e6f0535fe730e921ca347bc38dcb9e01791a" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 10 15:02:47 2010 -0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 17 12:34:20 2010 -0800" }, "message": "rcu: fine-tune grace-period begin/end checks\n\nUse the CPU\u0027s bit in rnp-\u003eqsmask to determine whether or not the CPU\nshould try to report a quiescent state. Handle overflow in the check\nfor rdp-\u003egpnum having fallen behind.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "5ff8e6f0535fe730e921ca347bc38dcb9e01791a", "tree": "84940db74c49e9312cd6b0295fa03a04603673a3", "parents": [ "20377f32dcb77941d450728da18cce5b1a7faec5" ], "author": { "name": "Frederic Weisbecker", "email": "fweisbec@gmail.com", "time": "Fri Dec 10 22:11:11 2010 +0100" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 17 12:34:19 2010 -0800" }, "message": "rcu: Keep gpnum and completed fields synchronized\n\nWhen a CPU that was in an extended quiescent state wakes\nup and catches up with grace periods that remote CPUs\ncompleted on its behalf, we update the completed field\nbut not the gpnum that keeps a stale value of a backward\ngrace period ID.\n\nLater, note_new_gpnum() will interpret the shift between\nthe local CPU and the node grace period ID as some new grace\nperiod to handle and will then start to hunt quiescent state.\n\nBut if every grace periods have already been completed, this\ninterpretation becomes broken. And we\u0027ll be stuck in clusters\nof spurious softirqs because rcu_report_qs_rdp() will make\nthis broken state run into infinite loop.\n\nThe solution, as suggested by Lai Jiangshan, is to ensure that\nthe gpnum and completed fields are well synchronized when we catch\nup with completed grace periods on their behalf by other cpus.\nThis way we won\u0027t start noting spurious new grace periods.\n\nSuggested-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Frederic Weisbecker \u003cfweisbec@gmail.com\u003e\nCc: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: Ingo Molnar \u003cmingo@elte.hu\u003e\nCc: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nCc: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nCc: Steven Rostedt \u003crostedt@goodmis.org\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "20377f32dcb77941d450728da18cce5b1a7faec5", "tree": "ef92da7c1fabcc68685021655842b3b18a6ee773", "parents": [ "e27fc9641e8ddc8146f8e01f06e5eba2469698de" ], "author": { "name": "Frederic Weisbecker", "email": "fweisbec@gmail.com", "time": "Fri Dec 10 22:11:10 2010 +0100" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Dec 17 12:34:18 2010 -0800" }, "message": "rcu: Stop chasing QS if another CPU did it for us\n\nWhen a CPU is idle and others CPUs handled its extended\nquiescent state to complete grace periods on its behalf,\nit will catch up with completed grace periods numbers\nwhen it wakes up.\n\nBut at this point there might be no more grace period to\ncomplete, but still the woken CPU always keeps its stale\nqs_pending value and will then continue to chase quiescent\nstates even if its not needed anymore.\n\nThis results in clusters of spurious softirqs until a new\nreal grace period is started. Because if we continue to\nchase quiescent states but we have completed every grace\nperiods, rcu_report_qs_rdp() is puzzled and makes that\nstate run into infinite loops.\n\nAs suggested by Lai Jiangshan, just reset qs_pending if\nsomeone completed every grace periods on our behalf.\n\nSuggested-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Frederic Weisbecker \u003cfweisbec@gmail.com\u003e\nCc: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: Ingo Molnar \u003cmingo@elte.hu\u003e\nCc: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nCc: Peter Zijlstra \u003ca.p.zijlstra@chello.nl\u003e\nCc: Steven Rostedt \u003crostedt@goodmis.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "909ea96468096b07fbb41aaf69be060d92bd9271", "tree": "a7e015edd96b5f674874fe78cdd889769e130a2a", "parents": [ "780f36d8b3fa9572f731d4fb85067b2e45e6f993" ], "author": { "name": "Christoph Lameter", "email": "cl@linux.com", "time": "Wed Dec 08 16:22:55 2010 +0100" }, "committer": { "name": "Tejun Heo", "email": "tj@kernel.org", "time": "Fri Dec 17 15:07:19 2010 +0100" }, "message": "core: Replace __get_cpu_var with __this_cpu_read if not used for an address.\n\n__get_cpu_var() can be replaced with this_cpu_read and will then use a\nsingle read instruction with implied address calculation to access the\ncorrect per cpu instance.\n\nHowever, the address of a per cpu variable passed to __this_cpu_read()\ncannot be determined (since it\u0027s an implied address conversion through\nsegment prefixes). Therefore apply this only to uses of __get_cpu_var\nwhere the address of the variable is not used.\n\nCc: Pekka Enberg \u003cpenberg@cs.helsinki.fi\u003e\nCc: Hugh Dickins \u003chughd@google.com\u003e\nCc: Thomas Gleixner \u003ctglx@linutronix.de\u003e\nAcked-by: H. Peter Anvin \u003chpa@zytor.com\u003e\nSigned-off-by: Christoph Lameter \u003ccl@linux.com\u003e\nSigned-off-by: Tejun Heo \u003ctj@kernel.org\u003e\n" }, { "commit": "2d999e03b7c8305b4385dd20992e4ed3e827177b", "tree": "3402eeb03a6cc7ac8013974983f09e46f5ecd562", "parents": [ "29494be71afe2a16ad04e344306a620d7cc22d06" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Oct 20 12:06:18 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Nov 29 22:01:59 2010 -0800" }, "message": "rcu: update documentation/comments for Lai\u0027s adoption patch\n\nLai\u0027s RCU-callback immediate-adoption patch changes the RCU tracing\noutput, so update tracing.txt. Also update a few comments to clarify\nthe synchronization design.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "29494be71afe2a16ad04e344306a620d7cc22d06", "tree": "d16a2acd1566be326483d5217d39b78d2ef798c5", "parents": [ "7b27d5475f86186914e54e4a6bb994e9a985337b" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Wed Oct 20 14:13:06 2010 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Nov 29 22:01:58 2010 -0800" }, "message": "rcu,cleanup: simplify the code when cpu is dying\n\nWhen we handle the CPU_DYING notifier, the whole system is stopped except\nfor the current CPU. We therefore need no synchronization with the other\nCPUs. This allows us to move any orphaned RCU callbacks directly to the\nlist of any online CPU without needing to run them through the global\norphan lists. These global orphan lists can therefore be dispensed with.\nThis commit makes thes changes, though currently victimizes CPU 0 @@@.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "4ee0a603926cad973e4d384f48c5e279a0fd4118", "tree": "da507192551987d4841fccc541a5b94adfef4f01", "parents": [ "b0a0f667a349247bd7f05f806b662a25653822bc" ], "author": { "name": "Dongdong Deng", "email": "dongdong.deng@windriver.com", "time": "Tue Sep 28 16:32:43 2010 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Oct 07 10:41:06 2010 -0700" }, "message": "rcu: using ACCESS_ONCE() to observe the jiffies_stall/rnp-\u003eqsmask value\n\nUsing ACCESS_ONCE() to observe the jiffies_stall/rnp-\u003eqsmask value\ndue to the caller didn\u0027t hold the root_rcu/rnp node\u0027s lock. Although\nuse without ACCESS_ONCE() is safe due to the value loaded being used\nbut once, the ACCESS_ONCE() is a good documentation aid -- the variables\nare being loaded without the services of a lock.\n\nSigned-off-by: Dongdong Deng \u003cdongdong.deng@windriver.com\u003e\nCC: Dipankar Sarma \u003cdipankar@in.ibm.com\u003e\nCC: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "269dcc1c2ec25864308ee03a3fa26ea819d9f5d0", "tree": "1c81ee6a9151ab1580628f6e9f85c469ceccb5c0", "parents": [ "0ddea0ead2ced99eaaaddff2beb755381e5c89f8" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Sep 07 14:23:09 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Sep 23 09:16:53 2010 -0700" }, "message": "rcu: Add tracing data to support queueing models\n\nThe current tracing data is not sufficient to deduce the average time\nthat a callback spends waiting for a grace period to end. Add three\nper-CPU counters recording the number of callbacks invoked (ci), the\nnumber of callbacks orphaned (co), and the number of callbacks adopted\n(ca). Given the existing callback queue length (ql), the average wait\ntime in absence of CPU hotplug operations is ql/ci. The units of wait\ntime will be in terms of the duration over which ci was measured.\n\nIn the presence of CPU hotplug operations, there is room for argument,\nbut ql/(ci-co+ca) won\u0027t steer you too far wrong.\n\nAlso fixes a typo called out by Lucas De Marchi \u003clucas.de.marchi@gmail.com\u003e.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "53d84e004d5e8c018be395c4330dc72fd60bd13e", "tree": "6d5b265442ac42c7cc73147b0de0d029644e312b", "parents": [ "8cdd32a918350430483751feaae1c19cef816f69" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Aug 10 14:28:53 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Aug 20 09:00:12 2010 -0700" }, "message": "rcu: permit suppressing current grace period\u0027s CPU stall warnings\n\nWhen using a kernel debugger, a long sojourn in the debugger can get\nyou lots of RCU CPU stall warnings once you resume. This might not be\nhelpful, especially if you are using the system console. This patch\ntherefore allows RCU CPU stall warnings to be suppressed, but only for\nthe duration of the current set of grace periods.\n\nThis differs from Jason\u0027s original patch in that it adds support for\ntiny RCU and preemptible RCU, and uses a slightly different method for\nsuppressing the RCU CPU stall warning messages.\n\nSigned-off-by: Jason Wessel \u003cjason.wessel@windriver.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nTested-by: Jason Wessel \u003cjason.wessel@windriver.com\u003e\n" }, { "commit": "8cdd32a918350430483751feaae1c19cef816f69", "tree": "b219bf5daf42a5fa0b48caf8b94174be2aa9a63f", "parents": [ "9079fd7c2e06a92cf27d05224a1f478581916c5b" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Aug 09 14:23:03 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Aug 20 09:00:11 2010 -0700" }, "message": "rcu: refer RCU CPU stall-warning victims to stallwarn.txt\n\nThere is some documentation on RCU CPU stall warnings contained in\nDocumentation/RCU/stallwarn.txt, but it will not be apparent to someone\nwho runs into such a warning while under time pressure. This commit\ntherefore adds comments preceding the printk()s pointing out the\nlocation of this documentation.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "910b1b7e19a292ff685001caf1bf1a9775b771a1", "tree": "7ec42bf018ab62c3c2180fbe168cd6130556cb80", "parents": [ "687d7a960aea46e016182c7ce346d62c4dbd0366" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jul 21 08:05:56 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Aug 19 17:18:04 2010 -0700" }, "message": "rcu: Allow RCU CPU stall warnings to be off at boot, but manually enablable\n\nCurrently, if RCU CPU stall warnings are enabled, they are enabled\nimmediately upon boot. They can be manually disabled via /sys (and\nalso re-enabled via /sys), and are automatically disabled upon panic.\nHowever, some users need RCU CPU stalls to be disabled at boot time,\nbut to be enabled without rebuilding/rebooting. For example, someone\nrunning a real-time application in production might not want the\nadditional latency of RCU CPU stall detection in normal operation, but\nmight need to enable it at any point for fault isolation purposes.\n\nThis commit therefore provides a new CONFIG_RCU_CPU_STALL_DETECTOR_RUNNABLE\nkernel configuration parameter that maintains the current behavior\n(enable at boot) by default, but allows a kernel to be configured\nwith RCU CPU stall detection built into the kernel, but disabled at\nboot time.\n\nRequested-by: Clark Williams \u003cwilliams@redhat.com\u003e\nRequested-by: John Kacur \u003cjkacur@redhat.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "f2e0dd7090eddef427ab9d9f81de122244cded51", "tree": "71f6dbc174eb9405c1d4313fafa2b5db369d5939", "parents": [ "ded5e5ed2f3348ba2f9a319c6497e46c22850e97" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jul 14 14:38:30 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Aug 19 17:18:03 2010 -0700" }, "message": "rcu: allow RCU CPU stall warning messages to be controlled in /sys\n\nSet the permissions of the rcu_cpu_stall_suppress to 644 to enable RCU\nCPU stall warnings to be enabled and disabled at runtime via sysfs.\n\nSuggested-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "742734eea0cf5314cde5945963ed964be167bd84", "tree": "5716bb62b38f7c65099ff8bc3142457e60637eb3", "parents": [ "b163760e37047781b37c412cde54d146ac4b651f" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Jun 30 11:43:52 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Aug 19 17:18:02 2010 -0700" }, "message": "rcu: add boot parameter to suppress RCU CPU stall warning messages\n\nAlthough the RCU CPU stall warning messages are a very good way to alert\npeople to a problem, once alerted, it is sometimes helpful to shut them\noff in order to avoid obscuring other messages that might be being used\nto track down the problem. Although you can rebuild the kernel with\nCONFIG_RCU_CPU_STALL_DETECTOR\u003dn, this is sometimes inconvenient. This\ncommit therefore adds a boot parameter named \"rcu_cpu_stall_suppress\"\nthat shuts these messages off without requiring a rebuild (though a\nreboot might be needed for those not brave enough to patch their kernel\nwhile it is running).\n\nThis message-suppression was already in place for the panic case, so this\ncommit need only rename the variable and export it via module_param().\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "394f99a9007d4274f7076bb8553ab0ff9707688b", "tree": "7d379f91321cec58b87fd5f5089947872689d591", "parents": [ "e546f485e1d7520ca0200502cdcc11b503f4805c" ], "author": { "name": "Lai Jiangshan", "email": "laijs@cn.fujitsu.com", "time": "Mon Jun 28 16:25:04 2010 +0800" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Aug 19 17:18:01 2010 -0700" }, "message": "rcu: simplify the usage of percpu data\n\n\u0026percpu_data is compatible with allocated percpu data.\n\nAnd we use it and remove the \"-\u003erda[NR_CPUS]\" array, saving significant\nstorage on systems with large numbers of CPUs. This does add an additional\nlevel of indirection and thus an additional cache line referenced, but\nbecause -\u003erda is not used on the read side, this is OK.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nReviewed-by: Tejun Heo \u003ctj@kernel.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\n" }, { "commit": "551d55a944b143ef26fbd482d1c463199d6f65cf", "tree": "6911d3f8e8719ba5ca43c83acdf87cbc8276d7d1", "parents": [ "875352c94224c88f5aa28cb77206f993bd31b7a2" ], "author": { "name": "Mathieu Desnoyers", "email": "mathieu.desnoyers@efficios.com", "time": "Sat Apr 17 08:48:42 2010 -0400" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon Jun 14 16:37:26 2010 -0700" }, "message": "tree/tiny rcu: Add debug RCU head objects\n\nHelps finding racy users of call_rcu(), which results in hangs because list\nentries are overwritten and/or skipped.\n\nChangelog since v4:\n- Bissectability is now OK\n- Now generate a WARN_ON_ONCE() for non-initialized rcu_head passed to\n call_rcu(). Statically initialized objects are detected with\n object_is_static().\n- Rename rcu_head_init_on_stack to init_rcu_head_on_stack.\n- Remove init_rcu_head() completely.\n\nChangelog since v3:\n- Include comments from Lai Jiangshan\n\nThis new patch version is based on the debugobjects with the newly introduced\n\"active state\" tracker.\n\nNon-initialized entries are all considered as \"statically initialized\". An\nactivation fixup (triggered by call_rcu()) takes care of performing the debug\nobject initialization without issuing any warning. Since we cannot increase the\nsize of struct rcu_head, I don\u0027t see much room to put an identifier for\nstatically initialized rcu_head structures. So for now, we have to live without\n\"activation without explicit init\" detection. But the main purpose of this debug\noption is to detect double-activations (double call_rcu() use of a rcu_head\nbefore the callback is executed), which is correctly addressed here.\n\nThis also detects potential internal RCU callback corruption, which would cause\nthe callbacks to be executed twice.\n\nSigned-off-by: Mathieu Desnoyers \u003cmathieu.desnoyers@efficios.com\u003e\nCC: David S. Miller \u003cdavem@davemloft.net\u003e\nCC: \"Paul E. McKenney\" \u003cpaulmck@linux.vnet.ibm.com\u003e\nCC: akpm@linux-foundation.org\nCC: mingo@elte.hu\nCC: laijs@cn.fujitsu.com\nCC: dipankar@in.ibm.com\nCC: josh@joshtriplett.org\nCC: dvhltc@us.ibm.com\nCC: niv@us.ibm.com\nCC: tglx@linutronix.de\nCC: peterz@infradead.org\nCC: rostedt@goodmis.org\nCC: Valdis.Kletnieks@vt.edu\nCC: dhowells@redhat.com\nCC: eric.dumazet@gmail.com\nCC: Alexey Dobriyan \u003cadobriyan@gmail.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nReviewed-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\n" }, { "commit": "72d5a9f7a9542f88397558c65bcfc3b115a65e34", "tree": "e46d3ad1948de45b9641d1a8b88f88bfb3809007", "parents": [ "4376030a54860dedab9d848dfa7cc700a6025c0b" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 17:12:17 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue May 11 16:10:47 2010 -0700" }, "message": "rcu: remove all rcu head initializations, except on_stack initializations\n\nRemove all rcu head inits. We don\u0027t care about the RCU head state before passing\nit to call_rcu() anyway. Only leave the \"on_stack\" variants so debugobjects can\nkeep track of objects on stack.\n\nSigned-off-by: Mathieu Desnoyers \u003cmathieu.desnoyers@efficios.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "d21670acab9fcb4bc74a40b68a6941059234c55c", "tree": "6a4c054bc4dbadf0524b4e221889a8da558dbdaf", "parents": [ "4a90a0681cf6cd21cd444184302aa045156486b3" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Apr 14 17:39:26 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:35 2010 -0700" }, "message": "rcu: reduce the number of spurious RCU_SOFTIRQ invocations\n\nLai Jiangshan noted that up to 10% of the RCU_SOFTIRQ are spurious, and\ntraced this down to the fact that the current grace-period machinery\nwill uselessly raise RCU_SOFTIRQ when a given CPU needs to go through\na quiescent state, but has not yet done so. In this situation, there\nmight well be nothing that RCU_SOFTIRQ can do, and the overhead can be\nworth worrying about in the ksoftirqd case. This patch therefore avoids\nraising RCU_SOFTIRQ in this situation.\n\nChanges since v1 (http://lkml.org/lkml/2010/3/30/122 from Lai Jiangshan):\n\no\tOmit the rcu_qs_pending() prechecks, as they aren\u0027t that\n\tmuch less expensive than the quiescent-state checks.\n\no\tMerge with the set_need_resched() patch that reduces IPIs.\n\no\tAdd the new n_rp_report_qs field to the rcu_pending tracing output.\n\no\tUpdate the tracing documentation accordingly.\n\nSigned-off-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "4a90a0681cf6cd21cd444184302aa045156486b3", "tree": "5e54587e0844d51f6c015f4c489cbf021c9db509", "parents": [ "4300aa642cc9ecb35f2e0683dd294fb790ef028c" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Wed Apr 14 16:48:11 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:35 2010 -0700" }, "message": "rcu: permit discontiguous cpu_possible_mask CPU numbering\n\nTREE_RCU assumes that CPU numbering is contiguous, but some users need\nlarge holes in the numbering to better map to hardware layout. This patch\nmakes TREE_RCU (and TREE_PREEMPT_RCU) tolerate large holes in the CPU\nnumbering. However, NR_CPUS must still be greater than the largest\nCPU number.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "4300aa642cc9ecb35f2e0683dd294fb790ef028c", "tree": "b70d676b02c6311a638fdaa6f3df8773b39fade1", "parents": [ "26845c2860cebebe6ce2d9d01ae3cb3db84b7e29" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Apr 13 16:18:22 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:34 2010 -0700" }, "message": "rcu: improve RCU CPU stall-warning messages\n\nThe existing RCU CPU stall-warning messages can be confusing, especially\nin the case where one CPU detects a single other stalled CPU. In addition,\nthe console messages did not say which flavor of RCU detected the stall,\nwhich can make it difficult to work out exactly what is causing the stall.\nThis commit improves these messages.\n\nRequested-by: Dhaval Giani \u003cdhaval.giani@gmail.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "26845c2860cebebe6ce2d9d01ae3cb3db84b7e29", "tree": "83849948ea7bffb5e87eecc17a7275e357740632", "parents": [ "c68de2097a8799549a3c3bf27cbfeea24a604284" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Tue Apr 13 14:19:23 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:34 2010 -0700" }, "message": "rcu: print boot-time console messages if RCU configs out of ordinary\n\nPrint boot-time messages if tracing is enabled, if fanout is set\nto non-default values, if exact fanout is specified, if accelerated\ndyntick-idle grace periods have been enabled, if RCU-lockdep is enabled,\nif rcutorture has been boot-time enabled, if the CPU stall detector has\nbeen disabled, or if four-level hierarchy has been enabled.\n\nThis is all for TREE_RCU and TREE_PREEMPT_RCU. TINY_RCU will be handled\nseparately, if at all.\n\nSuggested-by: Josh Triplett \u003cjosh@joshtriplett.org\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "c68de2097a8799549a3c3bf27cbfeea24a604284", "tree": "acf2e5cb6de06185002a7d22daa91d893068551b", "parents": [ "55ec936ff4e57cc626db336a7bf33b267390e9b4" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Apr 15 10:12:40 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:34 2010 -0700" }, "message": "rcu: disable CPU stall warnings upon panic\n\nThe current RCU CPU stall warnings remain enabled even after a panic\noccurs, which some people have found to be a bit counterproductive.\nThis patch therefore uses a notifier to disable stall warnings once a\npanic occurs.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "bbad937983147c017c25406860287cb94da9af7c", "tree": "57311598c360b92b189c7ec337146e4bc779c73d", "parents": [ "25502a6c13745f4650cc59322bd198194f55e796" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Fri Apr 02 16:17:17 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:34 2010 -0700" }, "message": "rcu: slim down rcutiny by removing rcu_scheduler_active and friends\n\nTINY_RCU does not need rcu_scheduler_active unless CONFIG_DEBUG_LOCK_ALLOC.\nSo conditionally compile rcu_scheduler_active in order to slim down\nrcutiny a bit more. Also gets rid of an EXPORT_SYMBOL_GPL, which is\nresponsible for most of the slimming.\n\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\n" }, { "commit": "25502a6c13745f4650cc59322bd198194f55e796", "tree": "d76cc659d3ea797c5da4630e219ac363d17c44a6", "parents": [ "99652b54de1ee094236f7171485214071af4ef31" ], "author": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Thu Apr 01 17:37:01 2010 -0700" }, "committer": { "name": "Paul E. McKenney", "email": "paulmck@linux.vnet.ibm.com", "time": "Mon May 10 11:08:33 2010 -0700" }, "message": "rcu: refactor RCU\u0027s context-switch handling\n\nThe addition of preemptible RCU to treercu resulted in a bit of\nconfusion and inefficiency surrounding the handling of context switches\nfor RCU-sched and for RCU-preempt. For RCU-sched, a context switch\nis a quiescent state, pure and simple, just like it always has been.\nFor RCU-preempt, a context switch is in no way a quiescent state, but\nspecial handling is required when a task blocks in an RCU read-side\ncritical section.\n\nHowever, the callout from the scheduler and the outer loop in ksoftirqd\nstill calls something named rcu_sched_qs(), whose name is no longer\naccurate. Furthermore, when rcu_check_callbacks() notes an RCU-sched\nquiescent state, it ends up unnecessarily (though harmlessly, aside\nfrom the performance hit) enqueuing the current task if it happens to\nbe running in an RCU-preempt read-side critical section. This not only\nincreases the maximum latency of scheduler_tick(), it also needlessly\nincreases the overhead of the next outermost rcu_read_unlock() invocation.\n\nThis patch addresses this situation by separating the notion of RCU\u0027s\ncontext-switch handling from that of RCU-sched\u0027s quiescent states.\nThe context-switch handling is covered by rcu_note_context_switch() in\ngeneral and by rcu_preempt_note_context_switch() for preemptible RCU.\nThis permits rcu_sched_qs() to handle quiescent states and only quiescent\nstates. It also reduces the maximum latency of scheduler_tick(), though\nprobably by much less than a microsecond. Finally, it means that tasks\nwithin preemptible-RCU read-side critical sections avoid incurring the\noverhead of queuing unless there really is a context switch.\n\nSuggested-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nAcked-by: Lai Jiangshan \u003claijs@cn.fujitsu.com\u003e\nSigned-off-by: Paul E. McKenney \u003cpaulmck@linux.vnet.ibm.com\u003e\nCc: Ingo Molnar \u003cmingo@elte.hu\u003e\nCc: Peter Zijlstra \u003cpeterz@infradead.org\u003e\n" } ], "next": "0c34029abdfdea64420cb4264c4e91a776b22157" }