)]}' { "commit": "3f6c82728f4e31a97c3a1b32abccb512fed0b573", "tree": "4b577e789a5daef91e40d10bc71c8134b3874ae8", "parents": [ "e325c90ffc13b698fa2814102e05275b21c26bec" ], "author": { "name": "Mel Gorman", "email": "mel@csn.ul.ie", "time": "Mon May 24 14:32:17 2010 -0700" }, "committer": { "name": "Linus Torvalds", "email": "torvalds@linux-foundation.org", "time": "Tue May 25 08:06:58 2010 -0700" }, "message": "mm: migration: take a reference to the anon_vma before migrating\n\nThis patchset is a memory compaction mechanism that reduces external\nfragmentation memory by moving GFP_MOVABLE pages to a fewer number of\npageblocks. The term \"compaction\" was chosen as there are is a number of\nmechanisms that are not mutually exclusive that can be used to defragment\nmemory. For example, lumpy reclaim is a form of defragmentation as was\nslub \"defragmentation\" (really a form of targeted reclaim). Hence, this\nis called \"compaction\" to distinguish it from other forms of\ndefragmentation.\n\nIn this implementation, a full compaction run involves two scanners\noperating within a zone - a migration and a free scanner. The migration\nscanner starts at the beginning of a zone and finds all movable pages\nwithin one pageblock_nr_pages-sized area and isolates them on a\nmigratepages list. The free scanner begins at the end of the zone and\nsearches on a per-area basis for enough free pages to migrate all the\npages on the migratepages list. As each area is respectively migrated or\nexhausted of free pages, the scanners are advanced one area. A compaction\nrun completes within a zone when the two scanners meet.\n\nThis method is a bit primitive but is easy to understand and greater\nsophistication would require maintenance of counters on a per-pageblock\nbasis. This would have a big impact on allocator fast-paths to improve\ncompaction which is a poor trade-off.\n\nIt also does not try relocate virtually contiguous pages to be physically\ncontiguous. However, assuming transparent hugepages were in use, a\nhypothetical khugepaged might reuse compaction code to isolate free pages,\nsplit them and relocate userspace pages for promotion.\n\nMemory compaction can be triggered in one of three ways. It may be\ntriggered explicitly by writing any value to /proc/sys/vm/compact_memory\nand compacting all of memory. It can be triggered on a per-node basis by\nwriting any value to /sys/devices/system/node/nodeN/compact where N is the\nnode ID to be compacted. When a process fails to allocate a high-order\npage, it may compact memory in an attempt to satisfy the allocation\ninstead of entering direct reclaim. Explicit compaction does not finish\nuntil the two scanners meet and direct compaction ends if a suitable page\nbecomes available that would meet watermarks.\n\nThe series is in 14 patches. The first three are not \"core\" to the series\nbut are important pre-requisites.\n\nPatch 1 reference counts anon_vma for rmap_walk_anon(). Without this\n\tpatch, it\u0027s possible to use anon_vma after free if the caller is\n\tnot holding a VMA or mmap_sem for the pages in question. While\n\tthere should be no existing user that causes this problem,\n\tit\u0027s a requirement for memory compaction to be stable. The patch\n\tis at the start of the series for bisection reasons.\nPatch 2 merges the KSM and migrate counts. It could be merged with patch 1\n\tbut would be slightly harder to review.\nPatch 3 skips over unmapped anon pages during migration as there are no\n\tguarantees about the anon_vma existing. There is a window between\n\twhen a page was isolated and migration started during which anon_vma\n\tcould disappear.\nPatch 4 notes that PageSwapCache pages can still be migrated even if they\n\tare unmapped.\nPatch 5 allows CONFIG_MIGRATION to be set without CONFIG_NUMA\nPatch 6 exports a \"unusable free space index\" via debugfs. It\u0027s\n\ta measure of external fragmentation that takes the size of the\n\tallocation request into account. It can also be calculated from\n\tuserspace so can be dropped if requested\nPatch 7 exports a \"fragmentation index\" which only has meaning when an\n\tallocation request fails. It determines if an allocation failure\n\twould be due to a lack of memory or external fragmentation.\nPatch 8 moves the definition for LRU isolation modes for use by compaction\nPatch 9 is the compaction mechanism although it\u0027s unreachable at this point\nPatch 10 adds a means of compacting all of memory with a proc trgger\nPatch 11 adds a means of compacting a specific node with a sysfs trigger\nPatch 12 adds \"direct compaction\" before \"direct reclaim\" if it is\n\tdetermined there is a good chance of success.\nPatch 13 adds a sysctl that allows tuning of the threshold at which the\n\tkernel will compact or direct reclaim\nPatch 14 temporarily disables compaction if an allocation failure occurs\n\tafter compaction.\n\nTesting of compaction was in three stages. For the test, debugging,\npreempt, the sleep watchdog and lockdep were all enabled but nothing nasty\npopped out. min_free_kbytes was tuned as recommended by hugeadm to help\nfragmentation avoidance and high-order allocations. It was tested on X86,\nX86-64 and PPC64.\n\nThs first test represents one of the easiest cases that can be faced for\nlumpy reclaim or memory compaction.\n\n1. Machine freshly booted and configured for hugepage usage with\n\ta) hugeadm --create-global-mounts\n\tb) hugeadm --pool-pages-max DEFAULT:8G\n\tc) hugeadm --set-recommended-min_free_kbytes\n\td) hugeadm --set-recommended-shmmax\n\n\tThe min_free_kbytes here is important. Anti-fragmentation works best\n\twhen pageblocks don\u0027t mix. hugeadm knows how to calculate a value that\n\twill significantly reduce the worst of external-fragmentation-related\n\tevents as reported by the mm_page_alloc_extfrag tracepoint.\n\n2. Load up memory\n\ta) Start updatedb\n\tb) Create in parallel a X files of pagesize*128 in size. Wait\n\t until files are created. By parallel, I mean that 4096 instances\n\t of dd were launched, one after the other using \u0026. The crude\n\t objective being to mix filesystem metadata allocations with\n\t the buffer cache.\n\tc) Delete every second file so that pageblocks are likely to\n\t have holes\n\td) kill updatedb if it\u0027s still running\n\n\tAt this point, the system is quiet, memory is full but it\u0027s full with\n\tclean filesystem metadata and clean buffer cache that is unmapped.\n\tThis is readily migrated or discarded so you\u0027d expect lumpy reclaim\n\tto have no significant advantage over compaction but this is at\n\tthe POC stage.\n\n3. In increments, attempt to allocate 5% of memory as hugepages.\n\t Measure how long it took, how successful it was, how many\n\t direct reclaims took place and how how many compactions. Note\n\t the compaction figures might not fully add up as compactions\n\t can take place for orders other than the hugepage size\n\nX86\t\t\t\tvanilla\t\tcompaction\nFinal page count 913 916 (attempted 1002)\npages reclaimed 68296 9791\n\nX86-64\t\t\t\tvanilla\t\tcompaction\nFinal page count: 901 902 (attempted 1002)\nTotal pages reclaimed: 112599 53234\n\nPPC64\t\t\t\tvanilla\t\tcompaction\nFinal page count: 93 94 (attempted 110)\nTotal pages reclaimed: 103216 61838\n\nThere was not a dramatic improvement in success rates but it wouldn\u0027t be\nexpected in this case either. What was important is that fewer pages were\nreclaimed in all cases reducing the amount of IO required to satisfy a\nhuge page allocation.\n\nThe second tests were all performance related - kernbench, netperf, iozone\nand sysbench. None showed anything too remarkable.\n\nThe last test was a high-order allocation stress test. Many kernel\ncompiles are started to fill memory with a pressured mix of unmovable and\nmovable allocations. During this, an attempt is made to allocate 90% of\nmemory as huge pages - one at a time with small delays between attempts to\navoid flooding the IO queue.\n\n vanilla compaction\nPercentage of request allocated X86 98 99\nPercentage of request allocated X86-64 95 98\nPercentage of request allocated PPC64 55 70\n\nThis patch:\n\nrmap_walk_anon() does not use page_lock_anon_vma() for looking up and\nlocking an anon_vma and it does not appear to have sufficient locking to\nensure the anon_vma does not disappear from under it.\n\nThis patch copies an approach used by KSM to take a reference on the\nanon_vma while pages are being migrated. This should prevent rmap_walk()\nrunning into nasty surprises later because anon_vma has been freed.\n\nSigned-off-by: Mel Gorman \u003cmel@csn.ul.ie\u003e\nAcked-by: Rik van Riel \u003criel@redhat.com\u003e\nCc: Minchan Kim \u003cminchan.kim@gmail.com\u003e\nCc: KOSAKI Motohiro \u003ckosaki.motohiro@jp.fujitsu.com\u003e\nCc: Christoph Lameter \u003ccl@linux-foundation.org\u003e\nCc: KAMEZAWA Hiroyuki \u003ckamezawa.hiroyu@jp.fujitsu.com\u003e\nSigned-off-by: Andrew Morton \u003cakpm@linux-foundation.org\u003e\nSigned-off-by: Linus Torvalds \u003ctorvalds@linux-foundation.org\u003e\n", "tree_diff": [ { "type": "modify", "old_id": "d25bd224d3707532198b1012dc2f4f158ca01875", "old_mode": 33188, "old_path": "include/linux/rmap.h", "new_id": "567d43f29a10f2c3993702ca5ef7fbfccf7bdafa", "new_mode": 33188, "new_path": "include/linux/rmap.h" }, { "type": "modify", "old_id": "5938db54e1d75aa6ec6590cc44ef8bd967c473cd", "old_mode": 33188, "old_path": "mm/migrate.c", "new_id": "b768a1d4fa43384d519c9f4de9778981ece03e96", "new_mode": 33188, "new_path": "mm/migrate.c" }, { "type": "modify", "old_id": "0feeef860a8f5b5d61081234f40e8e149e2696f5", "old_mode": 33188, "old_path": "mm/rmap.c", "new_id": "f522cb008646df170ef0f235a121467409a7c9a0", "new_mode": 33188, "new_path": "mm/rmap.c" } ] }