headers
Adam G Litke | 1 Aug 2002 01:12
Picon
Favicon

Kernel profiling tools ported to 2.5.29

Ok.  Quite a few patch releases to announce today (5 to be exact):

linux-2.5.29+gcov-1.patch - Gcov is finally ported to something newer than
2.5.18 :)  To accomodate the latest kernels, I had to make a few
modifications to the kernel module.  I saw this as a great oppertunity to
provide the module and kernel code together in one patch.  The module
(gcov.o) is now built and installed with the rest of the modules as it
should be.

linux-2.5.29+kernprof-1.patch - This has been found to work with gcc 3.0.4
only!  Your mileage may vary depending on the hardware you have but it
doesn't work on some IBM SMP boxes when compiled with 2.9*.  There is a bit
more information about the gcc bug over at the SGI kernprof page.

linux-2.5.29+lckm-1.patch - Latest port of lockmeter.  Nothing more to say
about this one.

linux-2.5.29+k+l-1.patch - Kernprof and lockmeter support.

linux-2.5.29+g+k+l-1.patch - Gcov, kernprof, and lockmeter support.

My apologies for not releasing all of these sooner.  I just completed my
transition to BitKeeper which really helps me keep these up to date and
makes it simple to integrate patches.

As usual these are available at the Sourceforge lse page -
http://www.sf.net/projects/lse.  Find them in the profiler toolbox section
with the above names.  Please test them out, comments are welcome :)

--Adam Litke
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mala Anand | 1 Aug 2002 14:42
Picon
Favicon

RE: [RFC] per cpu slab fix to reduce freemiss


Tony Luck wrote..
>> No I am using the object(beginning space) to store the links. When
>> allocated, I can initialize the space occupied by the link address.

>You can't use the start of the object (or any other part) in this way,
>you'll have no way to restore the value you overwrote.

>Take a look at Jeff Bonwick's paper on slab allocators which explains
>this a lot better than I can:

>
http://www.usenix.org/publications/library/proceedings/bos94/full_papers/bon

>wick.a

 I read the document and see that I cannot use the object space even
when the object is free.  However I would like to know how much of this
assumption is used in the Linux Kernel code.  Looking through the tcpip
stack code I realized that most of the caches(tcp_opne_request,
tcp_bind_bucket, tcp_tw_bucket, inet_peer_cache etc.,) do not have
constructors and destructors.Some of the caches have them ofcourse.
Skbs have constructor, however the code executes constructor before
freeing the object and initializes the rest of the fields during
allocation.  In this case, this feature of preserving the object
between uses do not eliminate any of the object initialization code.
Having said that I would like to know if in any part of the Linux code
is taking advantage of this assumption. That is, the object is preserved
by the slab allocator between uses so the user does not have to
reinitialize.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dipankar Sarma | 1 Aug 2002 15:22
Picon

Re: [RFC] per cpu slab fix to reduce freemiss

On Thu, Aug 01, 2002 at 07:42:10AM -0500, Mala Anand wrote:
> 
> 
> Tony Luck wrote..
> >> No I am using the object(beginning space) to store the links. When
> >> allocated, I can initialize the space occupied by the link address.
> 
> >You can't use the start of the object (or any other part) in this way,
> >you'll have no way to restore the value you overwrote.
> 
> >Take a look at Jeff Bonwick's paper on slab allocators which explains
> >this a lot better than I can:
> 
> >
> http://www.usenix.org/publications/library/proceedings/bos94/full_papers/bon
> 
> >wick.a
> 
> In the present design there is a limit on how many free objects are held
> in the per cpu array. So when an object is freed it might end in another
> cpu more often.  The main cost lies in memory latency than execution of
> initializing the fields.  I doubt if we get the same gain as explained in
> the paper by preserving the fields between uses on an SMP/NUMA machines.
> 
> I agree that preserving read only variables that can be used between uses
> will help performance. We still can do that by revising the assumption to
> leave the first 4 or whatever bytes needed to store the links. What do you
> think?

Mala,
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dipankar Sarma | 1 Aug 2002 15:44
Picon

Re: [RFC] per cpu slab fix to reduce freemiss

On Thu, Aug 01, 2002 at 08:31:45AM -0500, Mala Anand wrote:
> 
> Dipankar wrote..
> >Isn't it possible to tune the cpucache limit by writing to
> >/proc/slabinfo so that you avoid frequent draining of free objects ?
> >Am I missing something here ?
> 
> Are you referring to raising the per cpu array limit? I don't think you
> tune that using /proc/slabinfo.  However that does not solve the problem,

Hmm... then what does slabinfo_write()->kmem_tune_cpucache() do ?

> it only delays it.  It needs to grow/shrink dynamically based on need. I
> am not only referring to frequently draining of free objects but also
> as a result of this refilling the object array due to subsequent
> allocations and so on.

If draining of free objects become rare, shouldn't refilling of the
object also become rare ?

Thanks
--

-- 
Dipankar Sarma  <dipankar <at> in.ibm.com> http://lse.sourceforge.net
Linux Technology Center, IBM Software Lab, Bangalore, India.

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mala Anand | 1 Aug 2002 15:31
Picon
Favicon

Re: [RFC] per cpu slab fix to reduce freemiss


Dipankar wrote..
>Isn't it possible to tune the cpucache limit by writing to
>/proc/slabinfo so that you avoid frequent draining of free objects ?
>Am I missing something here ?

Are you referring to raising the per cpu array limit? I don't think you
tune that using /proc/slabinfo.  However that does not solve the problem,
it only delays it.  It needs to grow/shrink dynamically based on need. I
am not only referring to frequently draining of free objects but also
as a result of this refilling the object array due to subsequent
allocations and so on.

Regards,
    Mala

   Mala Anand
   IBM Linux Technology Center - Kernel Performance
   E-mail:manand <at> us.ibm.com
   Phone:838-8088; Tie-line:678-8088

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Permalink | Reply | 0 comments |
headers
Luck, Tony | 1 Aug 2002 23:31
Picon
Favicon

RE: [RFC] per cpu slab fix to reduce freemiss

> Furthermore I think this design does not take into consideration of
> multiprocessor issues such as cache-bouncing, cache-warmthness etc.,
> And also the original implementation of the slab cache in the Linux
> kernel did not have per cpu support (I am not sure if the paper takes
> into consideration of SMP etc., also). So this assumption needs to be
> examined in the light of SMP, NUMA etc., I would like to explore the
> possibility of changing this assumption if possible in lieu of SMP/NUMA
> cache effects.
>
> In the present design there is a limit on how many free objects are held
> in the per cpu array. So when an object is freed it might end in another
> cpu more often.  The main cost lies in memory latency than execution of
> initializing the fields.  I doubt if we get the same gain as explained in
> the paper by preserving the fields between uses on an SMP/NUMA machines.

Bonwick has a newer paper
(http://www.usenix.org/events/usenix01/bonwick.html)
that describes how per cpu support can be added.  I've forgotten my Usenix
password, so I can't get the full text of the paper online at the moment.
But, if I recall correctly his magazine layer included support to
dynamically
adjust the size of the per-cpu lists.

The question becomes: Are the performance benefits high enough to justify
this extra code complexity?  Especially as tuning using /proc/slabinfo is
already available to mitigate problems that are bad enough for people to
notice.

Can you quantify the SMP/NUMA benefits?  I took some measurements a while
ago that showed that a huge percentage of slab allocations were freed by the
(Continue reading)

Permalink | Reply | 6 comments |
headers
Patrick J. Dempsey | 2 Aug 2002 05:29
Favicon

Newly created BitKeeper trees for LSE Patches

Hi everyone, For your programming enjoyment I have created a
BitKeeper repository for all of the LSE patches.

I have created clones of Linus's bkbits tree and populated them
with some of the profiling tools.  I'd like to see some more trees, like 
the RCU stuff.

If any LSE hacker wants to have admin access to create your own trees 
please let me know and I will arrange it.

--

-- 
Patrick J. Dempsey
rock <at> sr71.net

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Permalink | Reply | 1 comment |
headers
Andrew Theurer | 2 Aug 2002 20:35
Picon
Favicon

readprofile and redhat 2.4.9-e3smp

I am trying to use readprofile and cannot get any output other than:

     4 _stext                                     0.0500
     4 total                                      0.0000

I have profile=2 and the System.map should be correct.  I have not had this 
problem with any kernel.org kernel.  Any ideas?  Did I miss something? The 
kernel is the one from RH Advanced Server 2.1, 2.4.9-e3smp.  

Thanks,

Andrew Theurer

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Permalink | Reply | 2 comments |
headers
Benjamin LaHaise | 2 Aug 2002 20:52
Picon
Favicon

Re: readprofile and redhat 2.4.9-e3smp

On Fri, Aug 02, 2002 at 01:35:13PM -0500, Andrew Theurer wrote:
> I have profile=2 and the System.map should be correct.  I have not had this 
> problem with any kernel.org kernel.  Any ideas?  Did I miss something? The 
> kernel is the one from RH Advanced Server 2.1, 2.4.9-e3smp.  

Boot with nmi_watchdog=1

		-ben
--

-- 
"You will be reincarnated as a toad; and you will be much happier."

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Permalink | Reply | 1 comment |
headers
Andrew Theurer | 3 Aug 2002 01:00
Picon
Favicon

Re: readprofile and redhat 2.4.9-e3smp

That worked, thanks.

-Andrew

On Friday 02 August 2002 1:52 pm, Benjamin LaHaise wrote:
> On Fri, Aug 02, 2002 at 01:35:13PM -0500, Andrew Theurer wrote:
> > I have profile=2 and the System.map should be correct.  I have not had
> > this problem with any kernel.org kernel.  Any ideas?  Did I miss
> > something? The kernel is the one from RH Advanced Server 2.1,
> > 2.4.9-e3smp.
>
> Boot with nmi_watchdog=1
>
> 		-ben

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Permalink | Reply | 0 comments |

Gmane