Adam Tauno Williams | 7 Mar 16:34 2009

ZideStore & odd leak behaviour

I understand that ZideStore monitors it's own size and then restarts
itself after having reached the configured limit (default 200MB, it
starts at about 36MB).   But we've done some benchmarks and found that
as ZideStore grows larger it actually gets slower;  which makes one
doubt the effectiveness of whatever caching it is supposedly doing.  So
I poked around further and found something really weird.  In this test
I'm performing the exact same folder enumeration over and over - so
theoretically I'm not adding anything to any cache after the first
request.  ZideStore answers the first 21 (yep, exactly 21) responses
without growing in size beyond the first request.  Then on the 22nd
request it starts to allocate additional memory on every request.

Anyone have a wild guess were to start looking as to why that is?  It
seems baffling to me.  Was the original author of ZideStore a Black Jack
dealer?

Also, as an aside: On my servers I've gone in to ZSBackend and removed
the SxRecordCacheManager & SxSetCacheManager and rebuilt, and have the
same results although the sizes differ slightly [of course].  From
looking at the code I don't think the cache actually worked, and my
little battery of tests shows no performance change upon removing
SxRecordCacheManager/SxSetCacheManager.  Actually the performance got
very slightly better although the difference is hard to be confident in
since it is so very small (close to being noise).

<log>
ZideStore dispatchRequest BEGIN (vmem size is 45182976)
ZideStore dispatchRequest END (vmem size is 45182976)

... 21 requests without an increase in vmem, then ...
(Continue reading)

Helge Hess | 7 Mar 16:50 2009

Re: ZideStore & odd leak behaviour

On 07.03.2009, at 16:34, Adam Tauno Williams wrote:
> ZideStore answers the first 21 (yep, exactly 21) responses
> without growing in size beyond the first request.  Then on the 22nd
> request it starts to allocate additional memory on every request.

Well, I would assume that this is just the way the glibc malloc  
allocated memory (first a big block, then in smaller increments). The  
size reported by top etc is the size allocated from the memory, not  
the actual size required by the application.

But sure, ZideStore might have a leak somewhere. I suppose if GNUstep  
declares gcc+gstep-base GC as stable, it would be another big reason  
to switch to gnustep-base.

> From looking at the code I don't think the cache actually worked,  
> and my
> little battery of tests shows no performance change upon removing
> SxRecordCacheManager/SxSetCacheManager.

Well, maybe the caches are not really used. Not sure what you are  
testing, but the eg vcard retrieval commands AFAIK have their own  
cache. The caches might have been intended for ZideLook like usage  
(WebDAV property retrievals), but can't say w/o revisiting the code.

Greets,
   Helge

PS: I'm not sure who did most of the work on the ZS, but I'm most  
likely responsible for the design in any case ;-)
--

-- 
(Continue reading)

Adam Tauno Williams | 8 Mar 20:51 2009

valgrind & Obj-C ? [Was: ZideStore & odd leak behaviour]

On Sat, 2009-03-07 at 16:50 +0100, Helge Hess wrote:
> On 07.03.2009, at 16:34, Adam Tauno Williams wrote:
> > ZideStore answers the first 21 (yep, exactly 21) responses
> > without growing in size beyond the first request.  Then on the 22nd
> > request it starts to allocate additional memory on every request.
> Well, I would assume that this is just the way the glibc malloc  

Yea, that makes sense.

> But sure, ZideStore might have a leak somewhere. I suppose if GNUstep  
> declares gcc+gstep-base GC as stable, it would be another big reason  
> to switch to gnustep-base.

Does anyone know if valgrind is useful with Objective-C?  I've googled
and can't find much if anything.   Aiming it at ZideStore, or just the
OGo utilities, is interesting but I'm still trying to figure out what it
means exactly.

For example I always get a reference to PrintfFormatScanner like:

==14519== 7,454 (5,860 direct, 1,594 indirect) bytes in 290 blocks are
definitely lost in loss record 23 of 33
==14519==    at 0x4025E92: calloc
(in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==14519==    by 0x45AD2FB: objc_calloc (in /usr/lib/libobjc.so.2.0.0)
==14519==    by 0x446E518: _i_NSDefaultZone__calloc_byteSize_
(NSDefaultZone.m:129)
==14519==    by 0x4465E3E: NSAllocateObject (NSZone.h:136)
==14519==    by 0x4497631: _c_NSTemporaryString__allocWithZone_
(NSString.m:1967)
(Continue reading)

Helge Hess | 8 Mar 21:10 2009

Re: valgrind & Obj-C ? [Was: ZideStore & odd leak behaviour]

On 08.03.2009, at 20:51, Adam Tauno Williams wrote:
> Does anyone know if valgrind is useful with Objective-C?

I've used KCacheGrind a lot to profile SOPE/OGo. But more for  
profiling, not for leak detection. Don't know about this.

In theory libFoundation supports BoehmGC PLUS retain counting in a  
leak-detection mode. Not sure whether this still works ... most likely  
not :-/
Its quite nice to find ObjC leaks.

Well, then of course one could try OmniObjectMeter with OGo on MacOSX  
to find leaks.

Thanks,
   Helge
-- 
Helge Hess
http://helgehess.eu/
--

-- 
OpenGroupware.org Developer
developer@...
http://mail.opengroupware.org/mailman/listinfo/developer

Adam Tauno Williams | 9 Mar 18:52 2009

Re: Build error with SOPE 4.6

On Mon, 2009-03-09 at 14:02 +0000, Antony Gelberg wrote:
> Hi,
> I'm trying to build some Debian packages for SOPE 4.6 from SVN, to head
> towards the point of making them publically available.  However I have a
> slight issue in sope-core/NGStreams - config.h doesn't seem to be picked up.
> It works if I change 
> #include "config.h"
> to
> #include "x86_64/linux-gnu/config.h"

Perhaps this in the configure script is relevant
<quote>
  # Note: GNUSTEP_TARGET_CPU is not yet available (set by common.make),
so we
  #       only have environment variables
  # Note: we can't set SYSTEM_LIB_DIR in this location, it gets
overridden by
  #       common.make
  cfgwrite "ifeq (\$(findstring _64, \$(GNUSTEP_HOST_CPU)), _64)"
  cfgwrite "CONFIGURE_64BIT:=yes"
  cfgwrite "CGS_LIBDIR_NAME:=lib64"
  cfgwrite "else"
  cfgwrite "CGS_LIBDIR_NAME:=lib"
  cfgwrite "endif"
</quote>

I've never built x86_64 packages so I don't know.  However, Whooha has,
so it must work somehow.

> But surely the build system should pick this up - I know little about gs_make,
(Continue reading)

Sebastian Reitenbach | 10 Mar 17:08 2009
Picon

Fwd: Re: [Urgent] GNUstep at Google Summer of Code'2009

Hi,

just read this message on the gnustep-discuss list
do we want to participate there?

I've no real idea, what the participation of SoC means, in tearms of what kind 
of work this means for someone participating, but I guess, I'll not have much 
time. Otherwise maybe it would be a chance to tackle the switch from 
libFoundation to gnustep-base.

Sebastian
Picon
From: Adam Fedor <fedor@...>
Subject: Re: [Urgent] GNUstep at Google Summer of Code'2009
Date: 2009-03-10 14:46:21 GMT


On Mar 10, 2009, at 7:05 AM, Quentin Mathé wrote:


Great, thanks Adam :-)

I just reorganized the GNUstep SoC page a bit by dividing Ideas into four project-specific sections: http://wiki.gnustep.org/index.php/Summer_Of_Code_2009


The SoC page mentions Opengroupware as participating with GNUstep, but I don't remember hearing from anyone from opengroupware about that.  Does anyone know if they want to participate or if they are applying separately?
(Continue reading)

Adam Tauno Williams | 10 Mar 19:02 2009

Re: Fwd: GNUstep at Google Summer of Code'2009

> just read this message on the gnustep-discuss list
> do we want to participate there?

I have no idea what it is.

> I've no real idea, what the participation of SoC means, in tearms of what kind 
> of work this means for someone participating, but I guess, I'll not have much 
> time. Otherwise maybe it would be a chance to tackle the switch from 
> libFoundation to gnustep-base.

That would be sweet.  I'd be more than willing to help someone what with
OGo/SOPE issues to the best I could; but I wouldn't consider myself
qualified to be a "mentor" regarding GNUstep.

> Sebastian
> email message attachment (forwarded message), "Adam Fedor
> <fedor@...>: Re: [Urgent] GNUstep at Google Summer of
> Code'2009"
> > -------- Forwarded Message --------
> > From: Adam Fedor <fedor@...>
> > To: Developer GNUstep <gnustep-dev@...>, Discuss-gnustep Discuss
> > <discuss-gnustep@...>
> > Cc:
> >
=?ISO-8859-1?Q?Discussion_list_for_=C9toil=E9_desktop_environmen?=@...,
t <etoile-discuss@...>
> > Subject: Re: [Urgent] GNUstep at Google Summer of Code'2009
> > Date: Tue, 10 Mar 2009 08:46:21 -0600
> > On Mar 10, 2009, at 7:05 AM, Quentin Mathé wrote:

(Continue reading)

Adam Tauno Williams | 10 Mar 19:33 2009

Re: Fwd: GNUstep at Google Summer of Code'2009

On Tue, 2009-03-10 at 14:02 -0400, Adam Tauno Williams wrote:
> > just read this message on the gnustep-discuss list
> > do we want to participate there?
> I have no idea what it is.
> > I've no real idea, what the participation of SoC means, in tearms of what kind 
> > of work this means for someone participating, but I guess, I'll not have much 
> > time. Otherwise maybe it would be a chance to tackle the switch from 
> > libFoundation to gnustep-base.
> That would be sweet.  I'd be more than willing to help someone what with
> OGo/SOPE issues to the best I could; but I wouldn't consider myself
> qualified to be a "mentor" regarding GNUstep.
> > Sebastian
> > email message attachment (forwarded message), "Adam Fedor
> > <fedor@...>: Re: [Urgent] GNUstep at Google Summer of
> > Code'2009"
> > > -------- Forwarded Message --------
> > > From: Adam Fedor <fedor@...>
> > > To: Developer GNUstep <gnustep-dev@...>, Discuss-gnustep Discuss
> > > <discuss-gnustep@...>
> > > Cc:
> > >
=?ISO-8859-1?Q?Discussion_list_for_=C9toil=E9_desktop_environmen?=@...,
t <etoile-discuss@...>
> > > Subject: Re: [Urgent] GNUstep at Google Summer of Code'2009
> > > Date: Tue, 10 Mar 2009 08:46:21 -0600
> > > On Mar 10, 2009, at 7:05 AM, Quentin Mathé wrote:

So if they want ideas (I have no idea how large/small they mean):
(a) port SOPE/OGo to from libFoundation to gnustep-base
(b) implement the CalDAV/CardDAV REPORT operations in ZideStore
(Continue reading)

Adam Tauno Williams | 11 Mar 13:28 2009

Re: Re: Fwd: GNUstep at Google Summer of Code'2009

On Wed, 2009-03-11 at 12:00 +0000, Antony Gelberg wrote:
> On 2009-03-10, Adam Tauno Williams <adam@...> wrote:
> > So if they want ideas (I have no idea how large/small they mean):
> > (a) port SOPE/OGo to from libFoundation to gnustep-base
> Not to hijack the thread, but I thought libFoundation was needed no longer -
> as http://sope.opengroupware.org/en/build/thirdparty.html states:
> "For SOPE 4.3.9 we reduced ThirdParty dependencies to just two packages
> (libobjc and libFoundation) and none of them are strictly required
> (libobjc/libFoundation only if you aren't using gstep-base or Cocoa). "
> I suppose the above is slightly unclear, especially the last parentheses - can
> you clarify?

You confusing SOPE with OGo.  I believe you can build (and use) SOPE on
gnustep-base;  but OGo with gnustep-base (and thus OGo with SOPE built
with gnustep-base) is untested.  I believe it builds but you'll run into
errors.  Whooha has done some fixes related this this [I think] and at
least once upon a time there was a branch of OGo for testing with gstep
base [it would probably be pretty out of date now]/

--

-- 
OpenGroupware.org Developer
developer@...
http://mail.opengroupware.org/mailman/listinfo/developer

Sebastian Reitenbach | 11 Mar 13:48 2009
Picon

Re: Re: Fwd: GNUstep at Google Summer of Code'2009

On Wednesday 11 March 2009 01:28:38 pm Adam Tauno Williams wrote:
> On Wed, 2009-03-11 at 12:00 +0000, Antony Gelberg wrote:
> > On 2009-03-10, Adam Tauno Williams <adam@...> wrote:
> > > So if they want ideas (I have no idea how large/small they mean):
> > > (a) port SOPE/OGo to from libFoundation to gnustep-base
> >
> > Not to hijack the thread, but I thought libFoundation was needed no
> > longer - as http://sope.opengroupware.org/en/build/thirdparty.html
> > states: "For SOPE 4.3.9 we reduced ThirdParty dependencies to just two
> > packages (libobjc and libFoundation) and none of them are strictly
> > required (libobjc/libFoundation only if you aren't using gstep-base or
> > Cocoa). " I suppose the above is slightly unclear, especially the last
> > parentheses - can you clarify?
>
> You confusing SOPE with OGo.  I believe you can build (and use) SOPE on
> gnustep-base;  but OGo with gnustep-base (and thus OGo with SOPE built
> with gnustep-base) is untested.  I believe it builds but you'll run into
> errors.  Whooha has done some fixes related this this [I think] and at
> least once upon a time there was a branch of OGo for testing with gstep
> base [it would probably be pretty out of date now]/

sope works fine linked against gnustep-base, as sogo works that way very well.

there are branches for ogo against gsmake2 and gnustep-base:
http://svn.opengroupware.org/cgi-bin/viewvc.cgi/branches/gsmake2/?root=SOPE
http://svn.opengroupware.org/cgi-bin/viewvc.cgi/branches/ogo-gsmake2/?root=OGo
after fosdem, znek fixed svn access, so that I was able to update both 
branches to trunk.
I'll merge trunk changes into those branches from time to time.

(Continue reading)


Gmane