Ted Unangst | 1 Apr 01:23 2006
Picon

Re: How to find memory leak in library/OS?

On 3/30/06, Claus Assmann <ca+OpenBSD_misc <at> zardoc.endmail.org> wrote:
> Is there some "simple" way to find a memory leak in some OS supplied
> library? I have a (constantly running) application that grows in a
> week from 5MB to 15MB in size (VSZ and RSS as reported by ps). The
> application can be compiled with an optional debugging memory
> allocator that tracks all (de)allocations to check whether any of
> its malloc()/free() calls leak memory; according to that tool the
> application behaves fine.  Hence I'm wondering whether there is a
> memory leak in some library or the OS, which also could be triggered
> by the way my application uses it (see the recent thread about
> telldir()/seekdir()). My application uses pthreads and the DNS
> resolver, the latter by contacting it via UDP: sendto(), recvfrom().
> Note: the memory leak seems to be unique to OpenBSD (3.8 and earlier),
> I can't reproduce it on SunOS 5.9 and others. That's why I'm asking
> for hints where to look for the leak: is there some "simple" way
> to show the allocated memory in the debugger or via system calls
> and to find out which functions made those allocations?

just to confirm something, this happens with openbsd 3.7?  3.6?

Anton Karpov | 1 Apr 02:04 2006
Picon

Re: security hole in sendmail

does that mean that we've got a second remote hole? Don't kick my ass.

AFAIK, even if this is a remote hole in sendmail, OpenBSD exploits
mitigation techniques makes this hole hardly (if even possible) exploitable
in OpenBSD. Am I right? Although this is an integer overflow, not buffer
overflow...

Claus Assmann | 1 Apr 02:29 2006

Re: How to find memory leak in library/OS?

On Fri, Mar 31, 2006, Ted Unangst wrote:

> > Note: the memory leak seems to be unique to OpenBSD (3.8 and earlier),

> just to confirm something, this happens with openbsd 3.7?  3.6?

3.7: yes; 3.6 probably yes, but I don't have statistics from
that time. Here's one from last year:

Tue Sep  6 19:51:43 PDT 2005
smxm      4793  0.0  0.6  2792  3016 ??  Ss     9:26AM    0:07.07 smar

Wed Sep  7 21:46:21 PDT 2005
smxm      4793  0.0  0.7  3580  3800 ??  Ss    Tue09AM    0:23.23 smar

Thu Sep  8 07:47:37 PDT 2005
smxm      4793  0.0  0.8  4016  4236 ??  Ss    Tue09AM    0:30.31 smar

Fri Sep  9 21:36:37 PDT 2005
smxm      4793  0.0  1.0  5084  5304 ??  Ss    Tue09AM    0:51.28 smar

Sat Sep 10 08:32:25 PDT 2005
smxm      4793  0.0  1.1  5300  5520 ??  Ss    Tue09AM    0:55.82 smar

(this wasn't important enough back then because I couldn't reproduce
it in a test environment and no user complained about it).

BTW: it does not seem to be a problem with mutex/cond: I "saved"
those in an array for reuse (instead of calling _init()/_destroy()
for every invocation) and even then the size grows. I'll try to
(Continue reading)

Greg Thomas | 1 Apr 02:41 2006
Picon

Re: Sys-Admin vs Network Admin

On 3/31/06, Karsten McMinn <tenyou <at> gmail.com> wrote:
> On 3/30/06, Greg Thomas <get.lists <at> gmail.com> wrote:
> >
> > Huh?  I'm not talking about any of the above and I'm not really
> > talking talking about official sysadmins, either.  I'm talking about
> > security-ignorant non-computer engineers that have root and no one's
> > going to take root away from them.
>
>
> why don't you do it?
>

I'd love to.  But it's been made clear to me in no uncertain terms
that I need to just report security issues rather than change them. 
And in most cases I understand as it could affect our on-air
operations if I went around booting people or locking down accounts.

Greg

David Higgs | 1 Apr 02:45 2006
Picon

Re: How to find memory leak in library/OS?

> BTW: it does not seem to be a problem with mutex/cond: I "saved"
> those in an array for reuse (instead of calling _init()/_destroy()
> for every invocation) and even then the size grows. I'll try to
> build a debugging version of libc (with some malloc checks) over
> the weekend.

Another old trick is to let your program eat memory for a good while,
and then break into its execution.  Randomly inspect some of the
allocated memory your program still holds; there is a very good chance
you're looking at leaked structures.  Hopefully you can figure out
what those structures are, and track down what's allocating so many.

--david

Claus Assmann | 1 Apr 02:58 2006

Re: How to find memory leak in library/OS?

On Fri, Mar 31, 2006, David Higgs wrote:

> Another old trick is to let your program eat memory for a good while,
> and then break into its execution.  Randomly inspect some of the
> allocated memory your program still holds; there is a very good chance
> you're looking at leaked structures.  Hopefully you can figure out
> what those structures are, and track down what's allocating so many.

Do you know how to find the allocated memory? Is there some "map"
(linked list?) to access from the debugger?

PS: please note that this leak doesn't show up on four other OSs,
not even testing with Purify shows a leak. Hence it seems most
likely that the leak is somewhere in an OpenBSD library, which means
I wouldn't find any of "my" structures on the heap (except for those
that are still in use).

Hannah Schroeter | 1 Apr 03:35 2006
Picon

Re: How to find memory leak in library/OS?

Hello!

On Fri, Mar 31, 2006 at 04:45:55PM -0800, David Higgs wrote:
>> BTW: it does not seem to be a problem with mutex/cond: I "saved"
>> those in an array for reuse (instead of calling _init()/_destroy()
>> for every invocation) and even then the size grows. I'll try to
>> build a debugging version of libc (with some malloc checks) over
>> the weekend.

>Another old trick is to let your program eat memory for a good while,
>and then break into its execution.  Randomly inspect some of the
>allocated memory your program still holds; there is a very good chance
>you're looking at leaked structures.  Hopefully you can figure out
>what those structures are, and track down what's allocating so many.

I've done that in a somewhat more sophisticated way: Wrap malloc and
friends to print their allocation size, address, *and a backtrace*
(just the addresses in hex), to a separate file descriptor. Then, a
separate program sets up that FD with a pipe, reads from that, matches
up malloc/calloc and free, taking realloc into account properly, of
course. And that separate program can display the top N allocation sizes
sorted by their backtrace. The separate program can also read nm output
so the backtraces are printed symbolically.

However, I'm not sure whether I could open-source that stuff, because
I wrote it at and for work.

And of course, it needs a little architecture dependent code for the
backtrace stuff. And a bit of hacking around (toolchain specific) so I
can intercept malloc etc., and at the same time use the original malloc
(Continue reading)

Nick Holland | 1 Apr 03:35 2006
Picon

Re: kernel

Alex Stamatis wrote:
> Hallo guys.
> 
> I have 1 question.
> I turned the 3.7 system in the stable batch and everything went fine. But
> what makes me wonder is that in dmesg or in uname-a the kernel doesnt say
> STABLE. In 2 other openbsd's that I have seen being in stable batch the
> STABLE word is shown. The best part is that in dmesg the kernels date is the
> new one (the day that I did the build on the new kernel).
> Do you know why this happens ???
> 
> Best Regards
> Alex
> 
> Uname :
> OpenBSD hercules 3.7 GENERIC#0 i386
> 
> 
> dmesg :
> OpenBSD 3.7 (GENERIC) #0: Wed Mar 29 04:41:11 EEST 2006
>     root <at> hercules:/usr/src/sys/arch/i386/compile/GENERIC
...

What is in your /usr/src/sys/CVS/Tag file?
(that tells us what you checked out)

Note: if you simply "apply patches" from errata37.html, you will not see 
the banner change, because 1) you really aren't running the stable 
branch, but rather a patched version of release (there are differences),
2) you don't alter the file that changes this by applying patches.
(Continue reading)

Hannah Schroeter | 1 Apr 03:39 2006
Picon

Re: How to find memory leak in library/OS?

Hi!

On Fri, Mar 31, 2006 at 04:58:28PM -0800, Claus Assmann wrote:
>On Fri, Mar 31, 2006, David Higgs wrote:

>> Another old trick is to let your program eat memory for a good while,
>> and then break into its execution.  Randomly inspect some of the
>> allocated memory your program still holds; there is a very good chance
>> you're looking at leaked structures.  Hopefully you can figure out
>> what those structures are, and track down what's allocating so many.

>Do you know how to find the allocated memory? Is there some "map"
>(linked list?) to access from the debugger?

I'm not sure whether there is a map in OpenBSD's malloc. However,
you could of course change it to output trace stuff similar to what I
described in my other mail, and then couple that trace stuff with gdb
debugging.

How complicated is your code? Perhaps I could try to build and test it
with my malloc tracing stuff.

Kind regards,

Hannah.

mrzehak | 1 Apr 01:35 2006
Picon

3.8 006_sendmail.patch make install problem.

Welcome,

I am relatively new to OpenBSD, still playing with it and exploring.
Even than, i already love that system. It's just the best quality
software i know. Perfect development policy - don't abandon it. But
back to the meritum. I have freshly installed OpenBSD
3.8-release/stable on old P100 box acting as a router, with all
previous patches applied without any problems. As always, i strictly
follow instructions included in patch file. Everything seems to be
O.K. untill make install command execution. I get following error
message:

install: Target: /usr/share/doc/smm/08.sendmailop
*** Error code 71

Stop in /usr/src/gnu/usr.sbin/sendmail/doc/op 
(line 47 of /usr/share/mk/bsd.doc.mk).
*** Error code 1

Stop in /usr/src/gnu/usr.sbin/sendmail.

After examining given error message and /usr/share/mk/bsd.doc.mk
file, i've decided to manualy make /usr/share/doc/smm/08.sendmailop
directory. After that, make install command was executed without any
error message, with followimg last five lines of output:

===> doc/op
install -c -o root -g bin -m 444  Makefile op.me 
/usr/share/doc/smm/08.sendmailop
===> cf
(Continue reading)


Gmane