Memory management

Andrea Arcangeli | 25 May 19:02

Picon

[PATCH 00/35] AutoNUMA alpha14

Hello everyone,

It's time for a new autonuma-alpha14 milestone.

Removed the [RFC] from Subject because 1) this is a release I'm quite
happy with (from the implementation side it allows the same kernel
image to boot optimally on NUMA and not-NUMA hardware and it avoids
altering the scheduler runtime most of the time) and 2) because of the
great benchmark results we got so far, showing this design so far has
been proved to perform best.

I believe (realistically speaking) nobody is going to change
applications to specify which thread is using which memory (for
threaded apps) with the only exception of QEMU and a few others.

For not threaded apps that fits in a NUMA node, there's no way a blind
home node can perform nearly as good as AutoNUMA: AutoNUMA monitor the
whole status of the memory of the running processes and it optimizes
the memory placement and CPU placement dynamically
accordingly. There's a small memory and CPU cost in collecting so much
information to be able to make smart decisions, but the benefits
largely outweight those costs.

If a big idle task was idle for a long while, but it suddenly start
computing, AutoNUMA may totally change the memory and CPU placement of
the other running tasks according to what's best, because it has
enough information to take optimal NUMA placement decisions.

git clone --reference linux -b autonuma-alpha14
git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git autonuma-alpha14

(Continue reading)

Jan Kara | 24 May 18:59

Picon

[PATCH 0/2 v4] Flexible proportions


  Hello,

  here is the next iteration of my flexible proportions code. I've addressed
all Peter's comments.
Changes since v3:
  * changed fprop_fraction_foo() to avoid using percpu_counter_sum()
  * changed __fprop_inc_percpu_max() to avoid 64-bit division (now maximum
    allowed fraction is expressed max_frac/FPROP_FRAC_BASE)
  * avoid drifting of period timer
  * handle better cases where period timer fires long after intended time by
    aging by really passed number of periods
Changes since v2:
  * use timer instead of workqueue for triggering period switch
  * arm timer only if aging didn't zero out all fractions, re-arm timer when
    new event arrives again
  * set period length to 3s

  Some introduction for first time readers:

  The idea of this patch set is to provide code for computing event proportions
where aging period is not dependent on the number of events happening (so
that aging works well both with fast storage and slow USB sticks in the same
system).

  The basic idea is that we compute proportions as:
p_j = (\Sum_{i>=0} x_{i,j}/2^{i+1}) / (\Sum_{i>=0} x_i/2^{i+1})

  Where x_{i,j} is j's number of events in i-th last time period and x_i is
total number of events in i-th last time period.

(Continue reading)

majianpeng | 24 May 15:38

Picon

the max size of block device on 32bit os,when using do_generic_file_read() proceed.

  Hi all:
		I readed a raid5,which size 30T.OS is RHEL6 32bit.
	    I reaed the raid5(as a whole,not parted) and found read address which not i wanted.
		So I tested the newest kernel code,the problem is still.
		I review the code, in function do_generic_file_read()

		index = *ppos >> PAGE_CACHE_SHIFT;
		index is u32.and *ppos is long long.
		So when *ppos is larger than 0xFFFF FFFF *  PAGE_CACHE_SHIFT(16T Byte),then the index is error.

		I wonder this .In 32bit os ,block devices size do not large then 16T,in other words, if block devices
larger than 16T,must parted.

																						Thanks all.
 				
--------------
majianpeng
2012-05-24

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo <at> kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont <at> kvack.org"> email <at> kvack.org </a>

Satoru Moriya | 23 May 22:41

[PATCH RESEND] avoid swapping out with swappiness==0

Hi Andrew,

This patch has been reviewed for couple of months.

This patch *only* improves the behavior when the kernel has
enough filebacked pages. It means that it does not change
the behavior when kernel has small number of filebacked pages.

Kosaki-san pointed out that the threshold which we use
to decide whether filebacked page is enough or not is not
appropriate(*).

(*) http://www.spinics.net/lists/linux-mm/msg32380.html

As I described in (**), I believe that threshold discussion
should be done in other thread because it affects not only
swappiness=0 case and the kernel behave the same way with
or without this patch below the threshold.

(**) http://www.spinics.net/lists/linux-mm/msg34317.html

The patch may not be perfect but, at least, we can improve
the kernel behavior in the enough filebacked memory case
with this patch. I believe it's better than nothing.

Do you have any comments about it?

NOTE: I updated the patch with Acked-by tags

---

(Continue reading)

Christoph Lameter | 23 May 22:34

Picon

Common 00/22] Sl[auo]b: Common functionality V3

V2->V3:
- Incorporate more feedback from Joonsoo Kim and Glauber Costa
- And a couple more patches to deal with slab duping and move
  more code to slab_common.c

V1->V2:
- Incorporate glommers feedback.
- Add 2 more patches dealing with common code in kmem_cache_destroy

This is a series of patches that extracts common functionality from
slab allocators into a common code base. The intend is to standardize
as much as possible of the allocator behavior while keeping the
distinctive features of each allocator which are mostly due to their
storage format and serialization approaches.

This patchset makes a beginning by extracting common functionality in
kmem_cache_create() and kmem_cache_destroy(). However, there are
numerous other areas where such work could be beneficial:

1. Extract the sysfs support from SLUB and make it common. That way
   all allocators have a common sysfs API and are handleable in the same
   way regardless of the allocator chose.

2. Extract the error reporting and checking from SLUB and make
   it available for all allocators. This means that all allocators
   will gain the resiliency and error handling capabilties.

3. Extract the memory hotplug and cpu hotplug handling. It seems that
   SLAB may be more sophisticated here. Having common code here will
   make it easier to maintain the special code.

(Continue reading)

Nathan Zimmer | 23 May 15:28

Picon

[PATCH] tmpfs not interleaving properly


When tmpfs has the memory policy interleaved it always starts allocating at each file at node 0.
When there are many small files the lower nodes fill up disproportionately.
My proposed solution is to start a file at a randomly chosen node.

Cc: Christoph Lameter <cl <at> linux.com>
Cc: Nick Piggin <npiggin <at> gmail.com>
Cc: Hugh Dickins <hughd <at> google.com>
Cc: Lee Schermerhorn <lee.schermerhorn <at> hp.com>
Cc: stable <at> vger.kernel.org
Signed-off-by: Nathan T Zimmer <nzimmer <at> sgi.com>

diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index 79ab255..38eda26 100644
--- a/include/linux/shmem_fs.h
+++ b/include/linux/shmem_fs.h
@@ -17,6 +17,7 @@ struct shmem_inode_info {
 		char		*symlink;	/* unswappable short symlink */
 	};
 	struct shared_policy	policy;		/* NUMA memory alloc policy */
+	int			node_offset;	/* bias for interleaved nodes */
 	struct list_head	swaplist;	/* chain of maybes on swap */
 	struct list_head	xattr_list;	/* list of shmem_xattr */
 	struct inode		vfs_inode;
diff --git a/mm/shmem.c b/mm/shmem.c
index f99ff3e..58ef512 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -819,7 +819,7 @@ static struct page *shmem_alloc_page(gfp_t gfp,

 	/* Create a pseudo vma that just contains the policy */

(Continue reading)

Bartlomiej Zolnierkiewicz | 23 May 09:28

[PATCH 2/2] vmevent: pass right attribute to vmevent_sample_attr()

From: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Subject: [PATCH] vmevent: pass right attribute to vmevent_sample_attr()

Pass "config attribute" (&watch->config->attrs[i]) not "sample
attribute" (&watch->sample_attrs[i]) to vmevent_sample_attr() to
allow use of the original attribute value in vmevent_attr_sample_fn().

Cc: Anton Vorontsov <anton.vorontsov <at> linaro.org>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park <at> samsung.com>
---
Without this patch vmevent_attr_lowmem_pages() always returns 0.

 mm/vmevent.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

Index: b/mm/vmevent.c
===================================================================
--- a/mm/vmevent.c	2012-05-22 17:55:02.000000000 +0200
+++ b/mm/vmevent.c	2012-05-22 18:10:40.075231798 +0200
@@ -31,6 +31,7 @@
 	 */
 	unsigned long			nr_attrs;
 	struct vmevent_attr		*sample_attrs;
+	struct vmevent_attr		*config_attrs[VMEVENT_CONFIG_MAX_ATTRS];

 	/* sampling */
 	struct timer_list		timer;
@@ -104,6 +105,7 @@
 	 */

(Continue reading)

Bartlomiej Zolnierkiewicz | 23 May 09:27

[PATCH] vmevent: add arm support

From: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Subject: [PATCH] vmevent: add arm support

Tested on ARM EXYNOS4210 (Universal C210 board).

Cc: Anton Vorontsov <anton.vorontsov <at> linaro.org>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park <at> samsung.com>
---
 arch/arm/include/asm/unistd.h        |    1 +
 arch/arm/kernel/calls.S              |    1 +
 tools/testing/vmevent/vmevent-test.c |    3 +++
 3 files changed, 5 insertions(+)

Index: b/arch/arm/include/asm/unistd.h
===================================================================
--- a/arch/arm/include/asm/unistd.h	2012-05-22 15:17:15.590826904 +0200
+++ b/arch/arm/include/asm/unistd.h	2012-05-22 15:17:43.990826872 +0200
@@ -404,6 +404,7 @@
 #define __NR_setns			(__NR_SYSCALL_BASE+375)
 #define __NR_process_vm_readv		(__NR_SYSCALL_BASE+376)
 #define __NR_process_vm_writev		(__NR_SYSCALL_BASE+377)
+#define __NR_vmevent_fd		(__NR_SYSCALL_BASE+378)

 /*
  * The following SWIs are ARM private.
Index: b/arch/arm/kernel/calls.S
===================================================================
--- a/arch/arm/kernel/calls.S	2012-05-22 15:16:31.646826898 +0200
+++ b/arch/arm/kernel/calls.S	2012-05-22 15:17:02.850825441 +0200

(Continue reading)

Bartlomiej Zolnierkiewicz | 23 May 09:27

[PATCH 1/2] vmevent: don't leak unitialized data to userspace

From: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Subject: [PATCH] vmevent: don't leak unitialized data to userspace

Remember to initialize all attrs[nr] fields in vmevent_setup_watch().

Cc: Anton Vorontsov <anton.vorontsov <at> linaro.org>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park <at> samsung.com>
---
 mm/vmevent.c |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Index: b/mm/vmevent.c
===================================================================
--- a/mm/vmevent.c	2012-05-22 17:51:13.195231958 +0200
+++ b/mm/vmevent.c	2012-05-22 17:51:40.991231956 +0200
@@ -350,7 +350,10 @@

 		attrs = new;

-		attrs[nr++].type = attr->type;
+		attrs[nr].type = attr->type;
+		attrs[nr].value = 0;
+		attrs[nr].state = 0;
+		nr++;
 	}

 	watch->sample_attrs	= attrs;

--

(Continue reading)

Bartlomiej Zolnierkiewicz | 23 May 09:22

[PATCH] cma: cached pageblock type fixup

From: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Subject: [PATCH] cma: cached pageblock type fixup

CMA pages added to per-cpu pages lists in free_hot_cold_page()
have private field set to MIGRATE_CMA pageblock type .  If this
happes just before start_isolate_page_range() in alloc_contig_range()
changes pageblock type of the page to MIGRATE_ISOLATE it may result
in the cached pageblock type being stale in free_pcppages_bulk()
(which may be triggered by drain_all_pages() in alloc_contig_range()),
page being added to MIGRATE_CMA free list instead of MIGRATE_ISOLATE
one in __free_one_page() and (if the page is reused just before
test_pages_isolated() check) causing alloc_contig_range() failure.

Fix such situation by checking whether pageblock type of the page
changed to MIGRATE_ISOLATE for MIGRATE_CMA type pages in
free_pcppages_bulk() and if so fixup the pageblock type to
MIGRATE_ISOLATE (so the page will be added to MIGRATE_ISOLATE free
list in __free_one_page() and won't be used).

Similar situation can happen if rmqueue_bulk() sets cached pageblock
of the page to MIGRATE_CMA and start_isolate_page_range() is called
before buffered_rmqueue() completes (so the page may used by
get_page_from_freelist() and cause test_pages_isolated() check
failure in alloc_contig_range()).  Fix it in buffered_rmqueue() by
changing the pageblock type of the affected page if needed, freeing
page back to buddy allocator and retrying the allocation.

Please note that even with this patch applied some page allocation
vs alloc_contig_range() races are still possible and may result in
rare test_pages_isolated() failures.

(Continue reading)

Bartlomiej Zolnierkiewicz | 23 May 09:23

[PATCH] cma: retry on test_pages_isolated() failure

From: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Subject: [PATCH] cma: retry on test_pages_isolated() failure

Retry (once) migration on test_pages_isolated() failure.

Cc: Michal Nazarewicz <mina86 <at> mina86.com>
Cc: Marek Szyprowski <m.szyprowski <at> samsung.com>
Cc: Mel Gorman <mgorman <at> suse.de>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie <at> samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park <at> samsung.com>
---
 mm/page_alloc.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Index: b/mm/page_alloc.c
===================================================================
--- a/mm/page_alloc.c	2012-05-15 12:40:54.199127705 +0200
+++ b/mm/page_alloc.c	2012-05-15 12:41:25.335127686 +0200
@@ -5796,7 +5796,7 @@
 {
 	struct zone *zone = page_zone(pfn_to_page(start));
 	unsigned long outer_start, outer_end;
-	int ret = 0, order;
+	int ret = 0, order, retry = 0;

 	/*
 	 * What we do here is we mark all pageblocks in range as
@@ -5826,7 +5826,7 @@
 				       pfn_max_align_up(end), migratetype);
 	if (ret)

(Continue reading)

Group

gmane.linux.kernel.mm.

Search Archive

Page

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11

May 2012

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane

Recent entries

[PATCH 00/35] AutoNUMA alpha14 (25 May 19:02)
[PATCH 0/2 v4] Flexible proportions (24 May 18:59)
the max size of block device on 32bit os,when using do_generic_file_re (24 May 15:38)
[PATCH RESEND] avoid swapping out with swappiness==0 (23 May 22:41)
Common 00/22] Sl[auo]b: Common functionality V3 (23 May 22:34)
[PATCH] tmpfs not interleaving properly (23 May 15:28)
[PATCH 2/2] vmevent: pass right attribute to vmevent_sample_attr() (23 May 09:28)
[PATCH] vmevent: add arm support (23 May 09:27)
[PATCH 1/2] vmevent: don't leak unitialized data to userspace (23 May 09:27)
[PATCH] cma: cached pageblock type fixup (23 May 09:22)
[PATCH] cma: retry on test_pages_isolated() failure (23 May 09:23)
[PATCH] mm: fix NULL ptr deref when walking hugepages (23 May 09:20)
[PATCH 1/2 v2] zsmalloc: zsmalloc: use unsigned long instead of void * (23 May 03:43)
[PATCH v4] mm: Fix slab->page flags corruption. (23 May 01:25)
[RFC] Common code 07/12] slabs: Move kmem_cache_create mutex handling (22 May 19:51)
memcg-devel git tree updated to 3.4 (22 May 17:57)
[PATCH] memcg/hugetlb: Add failcnt support for hugetlb extension (22 May 13:43)
[PATCH 1/2] mm/memblock: cleanup on duplicate VA/PA conversion (22 May 10:31)
[PATCH] hugetlb: fix resv_map leak in error path (21 May 22:28)
[PATCH] slab+slob: dup name string (21 May 17:18)
[RFC] Common code 08/12] slabs: list addition move to slab_common (21 May 11:27)

Archives

1193	May 2012
1140	April 2012
1265	March 2012
1181	February 2012
1091	January 2012
962	December 2011
1154	November 2011
766	October 2011
954	September 2011
1065	August 2011
728	July 2011
1496	June 2011
1385	May 2011
1362	April 2011
1638	March 2011
814	February 2011
761	January 2011
790	December 2010
1370	November 2010
1063	October 2010
867	September 2010
1152	August 2010
1048	July 2010
1114	June 2010
929	May 2010
1403	April 2010
1374	March 2010
881	February 2010
1211	January 2010
928	December 2009
919	November 2009
717	October 2009
1042	September 2009
1146	August 2009
1008	July 2009
1520	June 2009
1294	May 2009
1069	April 2009
1204	March 2009
783	February 2009
563	January 2009
674	December 2008
867	November 2008
1056	October 2008
569	September 2008
380	August 2008
770	July 2008
768	June 2008
812	May 2008
794	April 2008
1116	March 2008
872	February 2008
1128	January 2008
295	December 2007
543	November 2007
665	October 2007
482	September 2007
864	August 2007
716	July 2007
445	June 2007
955	May 2007
662	April 2007
602	March 2007
448	February 2007
420	January 2007
176	December 2006
222	November 2006
304	October 2006
239	September 2006
426	August 2006
142	July 2006
279	June 2006
309	May 2006
317	April 2006
307	March 2006
117	February 2006
333	January 2006
244	December 2005
566	November 2005
292	October 2005
207	September 2005
69	August 2005
164	July 2005
280	June 2005
348	May 2005
212	April 2005
181	March 2005
269	February 2005
364	January 2005
348	December 2004
442	November 2004
304	October 2004
189	September 2004
250	August 2004
115	July 2004
106	June 2004
163	May 2004
133	April 2004
129	March 2004
375	February 2004
240	January 2004
81	December 2003
132	November 2003
85	October 2003
133	September 2003
188	August 2003
326	July 2003
214	June 2003
381	May 2003
307	April 2003
206	March 2003
191	February 2003
198	January 2003
107	December 2002
166	November 2002
401	October 2002
397	September 2002

Design

Zawodny | Right Menu | Left Menu

Your Own Design

Paste the URL of your CSS below. Download CSS template