)]}' { "log": [ { "commit": "fee096deb4f33897937b974cb2c5168bab7935be", "tree": "c86e5ed5b3435ff0f0266f343b19f8cc7be63340", "parents": [ "d0e27b7808dc667f3015be0b6888f6d680e222c8" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:12:05 2009 +0000" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:12:05 2009 +0000" }, "message": "CacheFiles: Catch an overly long wait for an old active object\n\nCatch an overly long wait for an old, dying active object when we want to\nreplace it with a new one. The probability is that all the slow-work threads\nare hogged, and the delete can\u0027t get a look in.\n\nWhat we do instead is:\n\n (1) if there\u0027s nothing in the slow work queue, we sleep until either the dying\n object has finished dying or there is something in the slow work queue\n behind which we can queue our object.\n\n (2) if there is something in the slow work queue, we return ETIMEDOUT to\n fscache_lookup_object(), which then puts us back on the slow work queue,\n presumably behind the deletion that we\u0027re blocked by. We are then\n deferred for a while until we work our way back through the queue -\n without blocking a slow-work thread unnecessarily.\n\nA backtrace similar to the following may appear in the log without this patch:\n\n\tINFO: task kslowd004:5711 blocked for more than 120 seconds.\n\t\"echo 0 \u003e /proc/sys/kernel/hung_task_timeout_secs\" disables this message.\n\tkslowd004 D 0000000000000000 0 5711 2 0x00000080\n\t ffff88000340bb80 0000000000000046 ffff88002550d000 0000000000000000\n\t ffff88002550d000 0000000000000007 ffff88000340bfd8 ffff88002550d2a8\n\t 000000000000ddf0 00000000000118c0 00000000000118c0 ffff88002550d2a8\n\tCall Trace:\n\t [\u003cffffffff81058e21\u003e] ? trace_hardirqs_on+0xd/0xf\n\t [\u003cffffffffa011c4d8\u003e] ? cachefiles_wait_bit+0x0/0xd [cachefiles]\n\t [\u003cffffffffa011c4e1\u003e] cachefiles_wait_bit+0x9/0xd [cachefiles]\n\t [\u003cffffffff81353153\u003e] __wait_on_bit+0x43/0x76\n\t [\u003cffffffff8111ae39\u003e] ? ext3_xattr_get+0x1ec/0x270\n\t [\u003cffffffff813531ef\u003e] out_of_line_wait_on_bit+0x69/0x74\n\t [\u003cffffffffa011c4d8\u003e] ? cachefiles_wait_bit+0x0/0xd [cachefiles]\n\t [\u003cffffffff8104c125\u003e] ? wake_bit_function+0x0/0x2e\n\t [\u003cffffffffa011bc79\u003e] cachefiles_mark_object_active+0x203/0x23b [cachefiles]\n\t [\u003cffffffffa011c209\u003e] cachefiles_walk_to_object+0x558/0x827 [cachefiles]\n\t [\u003cffffffffa011a429\u003e] cachefiles_lookup_object+0xac/0x12a [cachefiles]\n\t [\u003cffffffffa00aa1e9\u003e] fscache_lookup_object+0x1c7/0x214 [fscache]\n\t [\u003cffffffffa00aafc5\u003e] fscache_object_state_machine+0xa5/0x52d [fscache]\n\t [\u003cffffffffa00ab4ac\u003e] fscache_object_slow_work_execute+0x5f/0xa0 [fscache]\n\t [\u003cffffffff81082093\u003e] slow_work_execute+0x18f/0x2d1\n\t [\u003cffffffff8108239a\u003e] slow_work_thread+0x1c5/0x308\n\t [\u003cffffffff8104c0f1\u003e] ? autoremove_wake_function+0x0/0x34\n\t [\u003cffffffff810821d5\u003e] ? slow_work_thread+0x0/0x308\n\t [\u003cffffffff8104be91\u003e] kthread+0x7a/0x82\n\t [\u003cffffffff8100beda\u003e] child_rip+0xa/0x20\n\t [\u003cffffffff8100b87c\u003e] ? restore_args+0x0/0x30\n\t [\u003cffffffff8104be17\u003e] ? kthread+0x0/0x82\n\t [\u003cffffffff8100bed0\u003e] ? child_rip+0x0/0x20\n\t1 lock held by kslowd004/5711:\n\t #0: (\u0026sb-\u003es_type-\u003ei_mutex_key#7/1){+.+.+.}, at: [\u003cffffffffa011be64\u003e] cachefiles_walk_to_object+0x1b3/0x827 [cachefiles]\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\n" }, { "commit": "a17754fb8c28af19cd70dcbec6d5b0773b94e0c1", "tree": "d7c25b217c684153eadbac78ab9b1bbff08b75f6", "parents": [ "868411be3f445a83fafbd734f3e426400138add5" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:52 2009 +0000" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:52 2009 +0000" }, "message": "CacheFiles: Don\u0027t write a full page if there\u0027s only a partial page to cache\n\ncachefiles_write_page() writes a full page to the backing file for the last\npage of the netfs file, even if the netfs file\u0027s last page is only a partial\npage.\n\nThis causes the EOF on the backing file to be extended beyond the EOF of the\nnetfs, and thus the backing file will be truncated by cachefiles_attr_changed()\ncalled from cachefiles_lookup_object().\n\nSo we need to limit the write we make to the backing file on that last page\nsuch that it doesn\u0027t push the EOF too far.\n\nAlso, if a backing file that has a partial page at the end is expanded, we\ndiscard the partial page and refetch it on the basis that we then have a hole\nin the file with invalid data, and should the power go out... A better way to\ndeal with this could be to record a note that the partial page contains invalid\ndata until the correct data is written into it.\n\nThis isn\u0027t a problem for netfs\u0027s that discard the whole backing file if the\nfile size changes (such as NFS).\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\n" }, { "commit": "60d543ca724be155c2b6166e36a00c80b21bd810", "tree": "90ec6edd77ddb7666dbf7069aa2e001f155cea49", "parents": [ "d461d26dde901b0523c46b0317e7fccf574a3933" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:45 2009 +0000" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:45 2009 +0000" }, "message": "FS-Cache: Start processing an object\u0027s operations on that object\u0027s death\n\nStart processing an object\u0027s operations when that object moves into the DYING\nstate as the object cannot be destroyed until all its outstanding operations\nhave completed.\n\nFurthermore, make sure that read and allocation operations handle being woken\nup on a dead object. Such events are recorded in the Allocs.abt and\nRetrvls.abt statistics as viewable through /proc/fs/fscache/stats.\n\nThe code for waiting for object activation for the read and allocation\noperations is also extracted into its own function as it is much the same in\nall cases, differing only in the stats incremented.\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\n" }, { "commit": "201a15428bd54f83eccec8b7c64a04b8f9431204", "tree": "326fcce64ce96657253fd141a3f4a767ac95418a", "parents": [ "e3d4d28b1c8cc7c26536a50b43d86ccd39878550" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:35 2009 +0000" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:35 2009 +0000" }, "message": "FS-Cache: Handle pages pending storage that get evicted under OOM conditions\n\nHandle netfs pages that the vmscan algorithm wants to evict from the pagecache\nunder OOM conditions, but that are waiting for write to the cache. Under these\nconditions, vmscan calls the releasepage() function of the netfs, asking if a\npage can be discarded.\n\nThe problem is typified by the following trace of a stuck process:\n\n\tkslowd005 D 0000000000000000 0 4253 2 0x00000080\n\t ffff88001b14f370 0000000000000046 ffff880020d0d000 0000000000000007\n\t 0000000000000006 0000000000000001 ffff88001b14ffd8 ffff880020d0d2a8\n\t 000000000000ddf0 00000000000118c0 00000000000118c0 ffff880020d0d2a8\n\tCall Trace:\n\t [\u003cffffffffa00782d8\u003e] __fscache_wait_on_page_write+0x8b/0xa7 [fscache]\n\t [\u003cffffffff8104c0f1\u003e] ? autoremove_wake_function+0x0/0x34\n\t [\u003cffffffffa0078240\u003e] ? __fscache_check_page_write+0x63/0x70 [fscache]\n\t [\u003cffffffffa00b671d\u003e] nfs_fscache_release_page+0x4e/0xc4 [nfs]\n\t [\u003cffffffffa00927f0\u003e] nfs_release_page+0x3c/0x41 [nfs]\n\t [\u003cffffffff810885d3\u003e] try_to_release_page+0x32/0x3b\n\t [\u003cffffffff81093203\u003e] shrink_page_list+0x316/0x4ac\n\t [\u003cffffffff8109372b\u003e] shrink_inactive_list+0x392/0x67c\n\t [\u003cffffffff813532fa\u003e] ? __mutex_unlock_slowpath+0x100/0x10b\n\t [\u003cffffffff81058df0\u003e] ? trace_hardirqs_on_caller+0x10c/0x130\n\t [\u003cffffffff8135330e\u003e] ? mutex_unlock+0x9/0xb\n\t [\u003cffffffff81093aa2\u003e] shrink_list+0x8d/0x8f\n\t [\u003cffffffff81093d1c\u003e] shrink_zone+0x278/0x33c\n\t [\u003cffffffff81052d6c\u003e] ? ktime_get_ts+0xad/0xba\n\t [\u003cffffffff81094b13\u003e] try_to_free_pages+0x22e/0x392\n\t [\u003cffffffff81091e24\u003e] ? isolate_pages_global+0x0/0x212\n\t [\u003cffffffff8108e743\u003e] __alloc_pages_nodemask+0x3dc/0x5cf\n\t [\u003cffffffff81089529\u003e] grab_cache_page_write_begin+0x65/0xaa\n\t [\u003cffffffff8110f8c0\u003e] ext3_write_begin+0x78/0x1eb\n\t [\u003cffffffff81089ec5\u003e] generic_file_buffered_write+0x109/0x28c\n\t [\u003cffffffff8103cb69\u003e] ? current_fs_time+0x22/0x29\n\t [\u003cffffffff8108a509\u003e] __generic_file_aio_write+0x350/0x385\n\t [\u003cffffffff8108a588\u003e] ? generic_file_aio_write+0x4a/0xae\n\t [\u003cffffffff8108a59e\u003e] generic_file_aio_write+0x60/0xae\n\t [\u003cffffffff810b2e82\u003e] do_sync_write+0xe3/0x120\n\t [\u003cffffffff8104c0f1\u003e] ? autoremove_wake_function+0x0/0x34\n\t [\u003cffffffff810b18e1\u003e] ? __dentry_open+0x1a5/0x2b8\n\t [\u003cffffffff810b1a76\u003e] ? dentry_open+0x82/0x89\n\t [\u003cffffffffa00e693c\u003e] cachefiles_write_page+0x298/0x335 [cachefiles]\n\t [\u003cffffffffa0077147\u003e] fscache_write_op+0x178/0x2c2 [fscache]\n\t [\u003cffffffffa0075656\u003e] fscache_op_execute+0x7a/0xd1 [fscache]\n\t [\u003cffffffff81082093\u003e] slow_work_execute+0x18f/0x2d1\n\t [\u003cffffffff8108239a\u003e] slow_work_thread+0x1c5/0x308\n\t [\u003cffffffff8104c0f1\u003e] ? autoremove_wake_function+0x0/0x34\n\t [\u003cffffffff810821d5\u003e] ? slow_work_thread+0x0/0x308\n\t [\u003cffffffff8104be91\u003e] kthread+0x7a/0x82\n\t [\u003cffffffff8100beda\u003e] child_rip+0xa/0x20\n\t [\u003cffffffff8100b87c\u003e] ? restore_args+0x0/0x30\n\t [\u003cffffffff8102ef83\u003e] ? tg_shares_up+0x171/0x227\n\t [\u003cffffffff8104be17\u003e] ? kthread+0x0/0x82\n\t [\u003cffffffff8100bed0\u003e] ? child_rip+0x0/0x20\n\nIn the above backtrace, the following is happening:\n\n (1) A page storage operation is being executed by a slow-work thread\n (fscache_write_op()).\n\n (2) FS-Cache farms the operation out to the cache to perform\n (cachefiles_write_page()).\n\n (3) CacheFiles is then calling Ext3 to perform the actual write, using Ext3\u0027s\n standard write (do_sync_write()) under KERNEL_DS directly from the netfs\n page.\n\n (4) However, for Ext3 to perform the write, it must allocate some memory, in\n particular, it must allocate at least one page cache page into which it\n can copy the data from the netfs page.\n\n (5) Under OOM conditions, the memory allocator can\u0027t immediately come up with\n a page, so it uses vmscan to find something to discard\n (try_to_free_pages()).\n\n (6) vmscan finds a clean netfs page it might be able to discard (possibly the\n one it\u0027s trying to write out).\n\n (7) The netfs is called to throw the page away (nfs_release_page()) - but it\u0027s\n called with __GFP_WAIT, so the netfs decides to wait for the store to\n complete (__fscache_wait_on_page_write()).\n\n (8) This blocks a slow-work processing thread - possibly against itself.\n\nThe system ends up stuck because it can\u0027t write out any netfs pages to the\ncache without allocating more memory.\n\nTo avoid this, we make FS-Cache cancel some writes that aren\u0027t in the middle of\nactually being performed. This means that some data won\u0027t make it into the\ncache this time. To support this, a new FS-Cache function is added\nfscache_maybe_release_page() that replaces what the netfs releasepage()\nfunctions used to do with respect to the cache.\n\nThe decisions fscache_maybe_release_page() makes are counted and displayed\nthrough /proc/fs/fscache/stats on a line labelled \"VmScan\". There are four\ncounters provided: \"nos\u003dN\" - pages that weren\u0027t pending storage; \"gon\u003dN\" -\npages that were pending storage when we first looked, but weren\u0027t by the time\nwe got the object lock; \"bsy\u003dN\" - pages that we ignored as they were actively\nbeing written when we looked; and \"can\u003dN\" - pages that we cancelled the storage\nof.\n\nWhat I\u0027d really like to do is alter the behaviour of the cancellation\nheuristics, depending on how necessary it is to expel pages. If there are\nplenty of other pages that aren\u0027t waiting to be written to the cache that\ncould be ejected first, then it would be nice to hold up on immediate\ncancellation of cache writes - but I don\u0027t see a way of doing that.\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\n" }, { "commit": "1bccf513ac49d44604ba1cddcc29f5886e70f1b6", "tree": "096ba75a3d02018c5f6e1857aaf1d41471733850", "parents": [ "6897e3df8fc37bd4a58bbcdef8306da7fc175584" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:25 2009 +0000" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:25 2009 +0000" }, "message": "FS-Cache: Fix lock misorder in fscache_write_op()\n\nFS-Cache has two structs internally for keeping track of the internal state of\na cached file: the fscache_cookie struct, which represents the netfs\u0027s state,\nand fscache_object struct, which represents the cache\u0027s state. Each has a\npointer that points to the other (when both are in existence), and each has a\nspinlock for pointer maintenance.\n\nSince netfs operations approach these structures from the cookie side, they get\nthe cookie lock first, then the object lock. Cache operations, on the other\nhand, approach from the object side, and get the object lock first. It is not\nthen permitted for a cache operation to get the cookie lock whilst it is\nholding the object lock lest deadlock occur; instead, it must do one of two\nthings:\n\n (1) increment the cookie usage counter, drop the object lock and then get both\n locks in order, or\n\n (2) simply hold the object lock as certain parts of the cookie may not be\n altered whilst the object lock is held.\n\nIt is also not permitted to follow either pointer without holding the lock at\nthe end you start with. To break the pointers between the cookie and the\nobject, both locks must be held.\n\nfscache_write_op(), however, violates the locking rules: It attempts to get the\ncookie lock without (a) checking that the cookie pointer is a valid pointer,\nand (b) holding the object lock to protect the cookie pointer whilst it follows\nit. This is so that it can access the pending page store tree without\ninterference from __fscache_write_page().\n\nThis is fixed by splitting the cookie lock, such that the page store tracking\ntree is protected by its own lock, and checking that the cookie pointer is\nnon-NULL before we attempt to follow it whilst holding the object lock.\n\nThe new lock is subordinate to both the cookie lock and the object lock, and so\nshould be taken after those.\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\n" }, { "commit": "4fbf4291aa15926cd4fdca0ffe9122e89d0459db", "tree": "ec2195c39ef8117acea598af4a5c20c77f67aa0b", "parents": [ "440f0affe247e9990c8f8778f1861da4fd7d5e50" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:04 2009 +0000" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:04 2009 +0000" }, "message": "FS-Cache: Allow the current state of all objects to be dumped\n\nAllow the current state of all fscache objects to be dumped by doing:\n\n\tcat /proc/fs/fscache/objects\n\nBy default, all objects and all fields will be shown. This can be restricted\nby adding a suitable key to one of the caller\u0027s keyrings (such as the session\nkeyring):\n\n\tkeyctl add user fscache:objlist \"\u003crestrictions\u003e\" @s\n\nThe \u003crestrictions\u003e are:\n\n\tK\tShow hexdump of object key (don\u0027t show if not given)\n\tA\tShow hexdump of object aux data (don\u0027t show if not given)\n\nAnd paired restrictions:\n\n\tC\tShow objects that have a cookie\n\tc\tShow objects that don\u0027t have a cookie\n\tB\tShow objects that are busy\n\tb\tShow objects that aren\u0027t busy\n\tW\tShow objects that have pending writes\n\tw\tShow objects that don\u0027t have pending writes\n\tR\tShow objects that have outstanding reads\n\tr\tShow objects that don\u0027t have outstanding reads\n\tS\tShow objects that have slow work queued\n\ts\tShow objects that don\u0027t have slow work queued\n\nIf neither side of a restriction pair is given, then both are implied. For\nexample:\n\n\tkeyctl add user fscache:objlist KB @s\n\nshows objects that are busy, and lists their object keys, but does not dump\ntheir auxiliary data. It also implies \"CcWwRrSs\", but as \u0027B\u0027 is given, \u0027b\u0027 is\nnot implied.\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\n" }, { "commit": "440f0affe247e9990c8f8778f1861da4fd7d5e50", "tree": "0012732ac8af0245005403d9a3b0a2b1209edf32", "parents": [ "3bde31a4ac225cb5805be02eff6eaaf7e0766ccd" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:01 2009 +0000" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Thu Nov 19 18:11:01 2009 +0000" }, "message": "FS-Cache: Annotate slow-work runqueue proc lines for FS-Cache work items\n\nAnnotate slow-work runqueue proc lines for FS-Cache work items. Objects\ninclude the object ID and the state. Operations include the object ID, the\noperation ID and the operation type and state.\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\n" }, { "commit": "7394daa8c61dfda4baa687f133748fa0b599b017", "tree": "32d2c55ed60596918ec62ce6ecca186337bf4660", "parents": [ "06b3db1b9bccdc8c2c743122a89745279e5ecc46" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Fri Apr 03 16:42:37 2009 +0100" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Fri Apr 03 16:42:37 2009 +0100" }, "message": "FS-Cache: Add use of /proc and presentation of statistics\n\nMake FS-Cache create its /proc interface and present various statistical\ninformation through it. Also provide the functions for updating this\ninformation.\n\nThese features are enabled by:\n\n\tCONFIG_FSCACHE_PROC\n\tCONFIG_FSCACHE_STATS\n\tCONFIG_FSCACHE_HISTOGRAM\n\nThe /proc directory for FS-Cache is also exported so that caching modules can\nadd their own statistics there too.\n\nThe FS-Cache module is loadable at this point, and the statistics files can be\nexamined by userspace:\n\n\tcat /proc/fs/fscache/stats\n\tcat /proc/fs/fscache/histogram\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\nAcked-by: Steve Dickson \u003csteved@redhat.com\u003e\nAcked-by: Trond Myklebust \u003cTrond.Myklebust@netapp.com\u003e\nAcked-by: Al Viro \u003cviro@zeniv.linux.org.uk\u003e\nTested-by: Daire Byrne \u003cDaire.Byrne@framestore.com\u003e\n" }, { "commit": "0dfc41d1efcc4180abfd32f68f0ade540e636ff6", "tree": "f066d08e2c33d2b475e55c5b18e4e4bff537ee75", "parents": [ "2d6fff637037395cc946ef910a880b5fa67b5370" ], "author": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Fri Apr 03 16:42:36 2009 +0100" }, "committer": { "name": "David Howells", "email": "dhowells@redhat.com", "time": "Fri Apr 03 16:42:36 2009 +0100" }, "message": "FS-Cache: Add the FS-Cache cache backend API and documentation\n\nAdd the API for a generic facility (FS-Cache) by which caches may declare them\nselves open for business, and may obtain work to be done from network\nfilesystems. The header file is included by:\n\n\t#include \u003clinux/fscache-cache.h\u003e\n\nDocumentation for the API is also added to:\n\n\tDocumentation/filesystems/caching/backend-api.txt\n\nThis API is not usable without the implementation of the utility functions\nwhich will be added in further patches.\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\nAcked-by: Steve Dickson \u003csteved@redhat.com\u003e\nAcked-by: Trond Myklebust \u003cTrond.Myklebust@netapp.com\u003e\nAcked-by: Al Viro \u003cviro@zeniv.linux.org.uk\u003e\nTested-by: Daire Byrne \u003cDaire.Byrne@framestore.com\u003e\n" } ] }