[RFC][PATCH] make all non-file-backed madvise() calls fail

This is related to this bug:
http://bugme.osdl.org/show_bug.cgi?id=2995

The kernel currently only checks that the memory is file-backed if
MADV_WILLNEED is set.  It's not entirely clear from the manpage at least
that *all* non-file-backed madvise() calls should fail.  

The attached patch returns -EBADF for all non-file-backed madvise()
calls.  I'm not suggesting that this is the absolutely right behavior,
but it certainly does what the bug submitter wants.

Comments?

-- Dave

[PATCH 0/6] CKRM: Memory controller for CKRM

Hello ckrm-tech members,

Here is the latest CKRM Memory controller patch against the patchset Gerrit
released couple of days back.

I applied the feedback I got on/off the list.

It is tested on i386.

Hello linux-mm members,

These are set of patches that provides the control of memory under the CKRM
framework(Details at http://ckrm.sf.net). I eagerly wait for your
feedback/comments/suggestions/concerns etc.,

To All,

I am looking for improvement suggestions
        - to not have a field in the page data structure for the mem
          controller
	- to make vmscan.c cleaner.

--------
Patches are
11-01-mem_base_changes:
        Basic changes to the core kernel to support memory controller.

11-02-mem_base-core:
        To fit in the ckrm framework. No support for guarantee, limit in
this

[PATCH 6/6] CKRM: Documentation for mem controller

-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan <at> us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

Patch 6 of 6 patches to support memory controller under CKRM framework.
Documentaion for the memory controller.

 mem_rc.design |  178 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 mem_rc.usage  |  112 ++++++++++++++++++++++++++++++++++++
 2 files changed, 290 insertions(+)

Index: linux-2.6.12-rc1/Documentation/ckrm/mem_rc.design
===================================================================
--- /dev/null
+++ linux-2.6.12-rc1/Documentation/ckrm/mem_rc.design
 <at>  <at>  -0,0 +1,178  <at>  <at> 
+0. Lifecycle of a LRU Page:
+----------------------------
+These are the events in a page's lifecycle:
+   - allocation of the page
+     there are multiple high level page alloc functions; __alloc_pages()
+	 is the lowest level function that does the real allocation.
+   - get into LRU list (active list or inactive list)
+   - get out of LRU list
+   - freeing the page

[Patch 5/6] CKRM: Add config support for mem controller

-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan <at> us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

Patch 5 of 6 patches to support memory controller under CKRM framework.
Provides some config parameter support. Details about the config parameters
in the Docuemntation patch.

 include/linux/ckrm_mem.h        |   13 +++++
 include/linux/ckrm_mem_inline.h |    4 +
 kernel/ckrm/ckrm_memcore.c      |  101 +++++++++++++++++++++++++++++++++++++---
 kernel/ckrm/ckrm_memctlr.c      |   46 ++++++++++++++++++
 mm/vmscan.c                     |  101 ++++++++++++++++++++++++++++++++++++++--
 5 files changed, 255 insertions(+), 10 deletions(-)

Index: linux-2.6.12-rc1/include/linux/ckrm_mem.h
===================================================================
--- linux-2.6.12-rc1.orig/include/linux/ckrm_mem.h
+++ linux-2.6.12-rc1/include/linux/ckrm_mem.h
 <at>  <at>  -69,17 +69,29  <at>  <at>  struct ckrm_mem_res {
 	int nr_dontcare;		/* # of dont care children */

 	struct ckrm_zone ckrm_zone[MAX_NR_ZONES];
+
+	struct list_head shrink_list;	/* list of classes that are near

[PATCH 3/6] CKRM: Add limit support for mem controller

-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan <at> us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

Patch 3 of 6 patches to support memory controller under CKRM framework.
This patch provides the limit support for teh controller.

 include/linux/ckrm_mem.h   |    2 
 kernel/ckrm/ckrm_memcore.c |  159 +++++++++++++++++++++++++++++++++++++++++++--
 kernel/ckrm/ckrm_memctlr.c |   16 ++++
 3 files changed, 172 insertions(+), 5 deletions(-)

Index: linux-2.6.12-rc1/include/linux/ckrm_mem.h
===================================================================
--- linux-2.6.12-rc1.orig/include/linux/ckrm_mem.h
+++ linux-2.6.12-rc1/include/linux/ckrm_mem.h
 <at>  <at>  -44,6 +44,8  <at>  <at>  struct ckrm_mem_res {
 					 * parent if more than this is needed.
 					 */
 	int hier;			/* hiearchy level, root = 0 */
+	int impl_guar;			/* for classes with don't care guar */
+	int nr_dontcare;		/* # of dont care children */
 };

 extern atomic_t ckrm_mem_real_count;

[PATCH 1/6] CKRM: Basic changes to the core kernel

-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan <at> us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

Patch 1 of 6 patches to support memory controller under CKRM framework.
This patch has the basic changes needed to get the hooks in the appropriate
kernel functions to get control in the controller.

 fs/exec.c                       |    2 +
 include/linux/ckrm_mem_inline.h |   67 ++++++++++++++++++++++++++++++++++++++++
 include/linux/mm_inline.h       |    7 ++++
 include/linux/page-flags.h      |    7 ++++
 include/linux/sched.h           |    8 ++++
 init/Kconfig                    |    9 +++++
 kernel/exit.c                   |    2 +
 kernel/fork.c                   |    6 +++
 mm/page_alloc.c                 |    6 +++
 9 files changed, 114 insertions(+)

Index: linux-2.6.12-rc1/fs/exec.c
===================================================================
--- linux-2.6.12-rc1.orig/fs/exec.c
+++ linux-2.6.12-rc1/fs/exec.c
 <at>  <at>  -49,6 +49,7  <at>  <at> 
 #include <linux/rmap.h>

[PATCH 2/6] CKRM: Core framework support

-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan <at> us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

Patch 2 of 6 patches to support memory controller under CKRM framework.
This patch gets the controller into the CKRM framework. Basically, it
provides all the necessary function points needed by the CKRM framework,
and also provides the filesystem interface.
No support for guarantee/limit is available in this patch.

 include/linux/ckrm_mem.h        |   68 +++++++
 include/linux/ckrm_mem_inline.h |  192 ++++++++++++++++++++
 include/linux/mm.h              |    4 
 kernel/ckrm/Makefile            |    1 
 kernel/ckrm/ckrm_memcore.c      |  376 ++++++++++++++++++++++++++++++++++++++++
 kernel/ckrm/ckrm_memctlr.c      |  205 +++++++++++++++++++++
 6 files changed, 845 insertions(+), 1 deletion(-)

Index: linux-2.6.12-rc1/include/linux/ckrm_mem.h
===================================================================
--- /dev/null
+++ linux-2.6.12-rc1/include/linux/ckrm_mem.h
 <at>  <at>  -0,0 +1,68  <at>  <at> 
+/* include/linux/ckrm_mem.h : memory control for CKRM
+ *

[PATCH 4/6] CKRM: Add guarantee support for mem controller

-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan <at> us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

Patch 4 of 6 patches to support memory controller under CKRM framework.
This patch provides the guarantee support for the controller.

 include/linux/ckrm_mem.h        |   31 ++++
 include/linux/ckrm_mem_inline.h |  244 ++++++++++++++++++++++++--------
 include/linux/mm.h              |    2 
 include/linux/mm_inline.h       |   10 +
 include/linux/mmzone.h          |    2 
 kernel/ckrm/ckrm_memcore.c      |   29 +++
 kernel/ckrm/ckrm_memctlr.c      |  299 +++++++++++++++++++++++++++++++++++++---
 mm/page_alloc.c                 |    3 
 mm/swap.c                       |    3 
 mm/vmscan.c                     |  114 +++++++++++++--
 10 files changed, 631 insertions(+), 106 deletions(-)

Index: linux-2.6.12-rc1/include/linux/ckrm_mem.h
===================================================================
--- linux-2.6.12-rc1.orig/include/linux/ckrm_mem.h
+++ linux-2.6.12-rc1/include/linux/ckrm_mem.h
 <at>  <at>  -26,6 +26,27  <at>  <at> 
 #include <linux/mmzone.h>

AIM9 slowdowns between 2.6.11 and 2.6.12-rc1

While testing the page placement policy patches on 2.6.12-rc1, I noticed
that aim9 is showing significant slowdowns on page allocation-related
tests. An excerpt of the results is at the end of this mail but it shows
that page_test is allocating 18000 less pages.

I did not check who has been recently changing the buddy allocator but
they might want to run a benchmark or two to make sure this is not
something specific to my setup.

root <at> monocle:~# grep _test vmregressbench-2.6.11-standard/aim9/log.txt
     7 page_test           60.01       4420   73.65439       125212.46 System Allocations & Pages/second
     8 brk_test            60.00       1732   28.86667       490733.33 System Memory Allocations/second
     9 jmp_test            60.01     252898 4214.26429      4214264.29 Non-local gotos/second
    10 signal_test         60.00       5983   99.71667        99716.67 Signal Traps/second
    11 exec_test           60.01        788   13.13114           65.66 Program Loads/second
    12 fork_test           60.06        986   16.41692         1641.69 Task Creations/second
    13 link_test           60.00       6302  105.03333         6617.10 Link/Unlink Pairs/second
root <at> monocle:~# grep _test vmregressbench-2.6.12-rc1-standard/aim9/log.txt
     7 page_test           60.01       3784   63.05616       107195.47 System Allocations & Pages/second
     8 brk_test            60.02       1194   19.89337       338187.27 System Memory Allocations/second
     9 jmp_test            60.00     252312 4205.20000      4205200.00 Non-local gotos/second
    10 signal_test         60.00       3731   62.18333        62183.33 Signal Traps/second
    11 exec_test           60.08        762   12.68309           63.42 Program Loads/second
    12 fork_test           60.04        864   14.39041         1439.04 Task Creations/second
    13 link_test           60.01       4723   78.70355         4958.32 Link/Unlink Pairs/second

--

-- 
Mel Gorman
Part-time Phd Student                          Java Applications Developer
University of Limerick                         IBM Dublin Software Lab

Re: AIM9 slowdowns between 2.6.11 and 2.6.12-rc1

On Sun, 2005-04-03 at 15:37 +0100, Mel Gorman wrote:
> While testing the page placement policy patches on 2.6.12-rc1, I noticed
> that aim9 is showing significant slowdowns on page allocation-related
> tests. An excerpt of the results is at the end of this mail but it shows
> that page_test is allocating 18000 less pages.
> 
> I did not check who has been recently changing the buddy allocator but
> they might want to run a benchmark or two to make sure this is not
> something specific to my setup.

Can you get some kernel profiles to see what, exactly, is causing the
decreased performance?  Also, what kind of system do you have?  Does
backing this out help?  If not, can you test some BK snapshots to see
when this started occurring?  

http://linus.bkbits.net:8080/linux-2.5/cset <at> 422de02c1628MP_noKSum9sGlTaC-Q

-- Dave