headers
Alistair Bayley | 1 Sep 2005 16:36
Picon

Profiling and analysing space usage

Hello all.

Below is a (typically pointless) program, which is a small slice from
a larger one I'm profiling. I'm interested in getting the memory usage
as small as possible. The loop function (and its analogue in the real
program) contributes significantly to the allocation stats. AFAICT,
this would be mainly due to a closure built for the if-stmt.

The heap profile graph for this program shows an initial peak, and
then the graph is flat at 8Mbytes, which I think is the space
allocated to the two STArrays (2 arrays, 1 million chars each, 4 bytes
per char?). So it looks as though any allocation for the loop function
is GC'd very soon after it's allocated.

So my questions are:

 - is my analysis of the space usage correct i.e. allocation in the
loop function is very short-lived and nothing to worry about?

 - is there anything I can do to reduce the memory usage, or is it
already minimal, assuming that I'm happy with the size of the
STArrays? I realise I could use, say, PackedStrings, but the original
program works on lists of (Eq a); I've just chosen Chars for this
example so as to eliminate polymorphism from the profiling.

Notes:
 - exporting only main makes a *big* difference to both space and time
usage, as does adding type signatures, to eliminate polymophism.

This is the only retainer set shown on the profiling graph:
(Continue reading)

Permalink | Reply | 2 comments |
headers
Frederik Eaton | 1 Sep 2005 17:14

hat, ghc

Is there a way to use Hat with GHC, without 'hmake'? The Hat Tutorial
shows 'hmake' being used, but 'hmake' doesn't work for me. The problem
is that 'hmake' seems to be looking for 'ghc' based on something other
than PATH:

 $ ghc --make pointtracker.hs
 Chasing modules from: pointtracker.hs
 Compiling Util             ( ./Util.hs, ./Util.o )
 Compiling XSimpleIO        ( ./XSimpleIO.hs, ./XSimpleIO.o )
 Compiling Main             ( pointtracker.hs, pointtracker.o )
 Linking ...
 $ which ghc
 /home/frederik/.toast-i386/armed/bin/ghc
 $ hmake -hat pointtracker.hs

 Fail: Can't find module Graphics.X11.Xlib in user directories
         .
   Or in installed libraries/packages at
         /usr/lib/ghc-6.2.2/imports
   Asked for by: pointtracker.hs
   Fix using the -I, -P, or -package flags.

 Stop - hmake dependency error.

i.e., I'm not sure how 'hmake' found /usr/lib/ghc-6.2.2, but this
directory is not mentioned in my environment, and it was not my
intention to use this version of ghc.

Thanks,
(Continue reading)

Permalink | Reply | 5 comments |
headers
Olaf Chitil | 1 Sep 2005 17:42
Picon
Favicon

Re: hat, ghc

Frederik Eaton wrote:

>Is there a way to use Hat with GHC, without 'hmake'? 
>
Sure, you can make all calls to hat-trans and ghc directly yourself, but 
that is rather cumbersome. You cannot use anything like ghc --make, 
because ghc doesn't know about Hat.

>The Hat Tutorial
>shows 'hmake' being used, but 'hmake' doesn't work for me. The problem
>is that 'hmake' seems to be looking for 'ghc' based on something other
>than PATH:
>  
>
hmake uses its own little database of compilers. You can tell hmake 
about the compiler through

hmake-config add ghc
hmake-config default ghc

http://www.haskell.org/hmake/hmake-config.html gives more details.

>...
> 
> Fail: Can't find module Graphics.X11.Xlib in user directories
> 
>
This indicates that you will run into a more serious problem. Hat 
currently only supports a limited set of libraries and Graphics.X11.Xlib 
is not among them.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Malcolm Wallace | 1 Sep 2005 17:46
Picon

Re: hat, ghc

Frederik Eaton <frederik <at> a5.repetae.net> writes:

> Is there a way to use Hat with GHC, without 'hmake'?

You could transform each module in your project individually with
hat-trans, before running ghc --make over the traced version.
Unfortunately, this means you will need to find some other way to
do the dependency analysis to ensure that modules are transformed
in the right order.  It is a known deficiency of the ghc --make
implementation that it cannot handle pre-processors nicely - this
applies equally to Happy, Alex, generic Haskell, etc.

One way would be to write a Makefile, with rules something like

    $(patsubst %.hs, Hat/%.hs, ${SRCS}) : Hat/%.hs : %.hs
		hat-trans $<
    $(patsubst %.hs, Hat/%.o, ${SRCS}) : Hat/$.o : Hat/%.hs
		ghc -c -package hat $<

and then use ghc -M to generate the dependencies somehow.

>  $ which ghc
>  /home/frederik/.toast-i386/armed/bin/ghc
>  $ hmake -hat pointtracker.hs
> 
>  Fail: Can't find module Graphics.X11.Xlib in user directories
>          .
>    Or in installed libraries/packages at
>          /usr/lib/ghc-6.2.2/imports
>    Asked for by: pointtracker.hs
(Continue reading)

Permalink | Reply | 3 comments |
headers
Frederik Eaton | 1 Sep 2005 22:34

Re: hat, ghc

On Thu, Sep 01, 2005 at 04:46:38PM +0100, Malcolm Wallace wrote:
> Frederik Eaton <frederik <at> a5.repetae.net> writes:
> 
> > Is there a way to use Hat with GHC, without 'hmake'?
> 
> You could transform each module in your project individually with
> hat-trans, before running ghc --make over the traced version.
> Unfortunately, this means you will need to find some other way to
> do the dependency analysis to ensure that modules are transformed
> in the right order.  It is a known deficiency of the ghc --make
> implementation that it cannot handle pre-processors nicely - this
> applies equally to Happy, Alex, generic Haskell, etc.
> 
> One way would be to write a Makefile, with rules something like
> 
>     $(patsubst %.hs, Hat/%.hs, ${SRCS}) : Hat/%.hs : %.hs
> 		hat-trans $<
>     $(patsubst %.hs, Hat/%.o, ${SRCS}) : Hat/$.o : Hat/%.hs
> 		ghc -c -package hat $<
> 
> and then use ghc -M to generate the dependencies somehow.

OK, I might try that.

> ...
> > i.e., I'm not sure how 'hmake' found /usr/lib/ghc-6.2.2, but this
> > directory is not mentioned in my environment, and it was not my
> > intention to use this version of ghc.
> 
> You can select which version of ghc is used by hmake, either on the
(Continue reading)

Permalink | Reply | 2 comments |
headers
Ketil Malde | 2 Sep 2005 08:57
Picon
Picon

Re: Profiling and analysing space usage

Alistair Bayley <abayley <at> gmail.com> writes:

I'm no expert, but since nobody else seems to have answered:

>  - is my analysis of the space usage correct i.e. allocation in the
> loop function is very short-lived and nothing to worry about?

IME, that would be the typical case.

>  - is there anything I can do to reduce the memory usage, or is it
> already minimal, assuming that I'm happy with the size of the
> STArrays? I realise I could use, say, PackedStrings

Slightly more general, you could use unboxed STUArrays.  Still limited
to a few built-in data types, and strict.  E.g. a STUArray Int Word8
would only take up one byte per element, an STArray would store a
(4 byte) pointer in each cell, pointing either to a Word8 value, or to
a lazy thunk for computing it.

At least as far as I understand it.

(Manuel's paper notwithstanding, I still have the feeling that
*UArrays should be *StrictArrays, and unboxing should be left as a
compiler optimization (which then could apply to arbitrary element
types, as long as they were strict.) 

-k
--

-- 
If I haven't seen further, it is by standing in the footprints of giants
(Continue reading)

Permalink | Reply | 0 comments |
headers
Malcolm Wallace | 2 Sep 2005 11:22
Picon

Re: hat, ghc

Frederik Eaton <frederik <at> a5.repetae.net> writes:

> By the way, can I make a request to have the hmake default be to use
> the environment?

Ian Lynagh recently added this capability to hmake, but it wasn't
documented (until yesterday).  The relevant option is
    hmake-config add-dyn ghc
which causes the version of ghc to be probed every time hmake is run,
rather than only once at configure-time.  Thus, it should pick up
the current environment settings as needed.

> Now, it appears to be looking for 'ghc-pkg' in the library directory
> of 'ghc' (rather, again, than in $PATH)

Well, hmake absolutely needs to use the ghc-pkg that belongs to the
particular version of ghc.  The heuristic used to find it might be
slightly fragile, but it is the best available, and indeed it is
based on the information given by ghc itself.  If the executable
has been moved such that querying ghc for its location is no longer
reliable, that's tough to fix.

Regards,
    Malcolm
Permalink | Reply | 1 comment |
headers
Simon Peyton-Jones | 2 Sep 2005 14:58
Picon
Favicon
Gravatar

RE: Profiling under 6.4.1 and Solaris segfaults

Does this happen on anything other than Solaris?  We don't have local
access to Solaris hardware at the moment.

Simon

| -----Original Message-----
| From: glasgow-haskell-users-bounces <at> haskell.org
[mailto:glasgow-haskell-users-
| bounces <at> haskell.org] On Behalf Of Nils Anders Danielsson
| Sent: 29 August 2005 18:04
| To: glasgow-haskell-users <at> haskell.org
| Subject: Profiling under 6.4.1 and Solaris segfaults
| 
| Running a simple Hello World program compiled with a GHC 6.4.1
| snapshot (20050820) and profiling turned on (-prof -auto-all) under
| Solaris results in a segfault. The core dump is similar to a core dump
| posted earlier
|
(http://www.mail-archive.com/glasgow-haskell-bugs <at> haskell.org/msg04621.h
tml):
| 
| (gdb) where
| #0  0x0003df8c in EnterFunCCS ()
| #1  0x0004a714 in stg_PAP_info ()
| #2  0x00045538 in schedule ()
| #3  0x00045ff8 in waitThread_ ()
| #4  0x00045f48 in scheduleWaitThread ()
| #5  0x0004258c in rts_evalLazyIO ()
| #6  0x0003bea8 in main ()
|
(Continue reading)

Permalink | Reply | 0 comments |
headers
Bulat Ziganshin | 2 Sep 2005 18:08

Re: Profiling and analysing space usage

Hello Alistair,

Thursday, September 01, 2005, 6:36:09 PM, you wrote:

AB> The heap profile graph for this program shows an initial peak, and
AB> then the graph is flat at 8Mbytes, which I think is the space
AB> allocated to the two STArrays (2 arrays, 1 million chars each, 4 bytes
AB> per char?). So it looks as though any allocation for the loop function
AB> is GC'd very soon after it's allocated.

Char uses 4 bytes because it must hold any Unicode character, which is
about 1 million :)  use Word8 or CChar

--

-- 
Best regards,
 Bulat                            mailto:bulatz <at> HotPOP.com
Permalink | Reply | 0 comments |
headers
Frederik Eaton | 2 Sep 2005 21:00

Re: hat, ghc

On Fri, Sep 02, 2005 at 10:22:06AM +0100, Malcolm Wallace wrote:
> Frederik Eaton <frederik <at> a5.repetae.net> writes:
> 
> > By the way, can I make a request to have the hmake default be to use
> > the environment?
> 
> Ian Lynagh recently added this capability to hmake, but it wasn't
> documented (until yesterday).  The relevant option is
>     hmake-config add-dyn ghc
> which causes the version of ghc to be probed every time hmake is run,
> rather than only once at configure-time.  Thus, it should pick up
> the current environment settings as needed.

OK.

> > Now, it appears to be looking for 'ghc-pkg' in the library directory
> > of 'ghc' (rather, again, than in $PATH)
> 
> Well, hmake absolutely needs to use the ghc-pkg that belongs to the
> particular version of ghc.  The heuristic used to find it might be
> slightly fragile, but it is the best available, and indeed it is
> based on the information given by ghc itself.  If the executable
> has been moved such that querying ghc for its location is no longer
> reliable, that's tough to fix.

If 'ghc' could always tell you where the correct 'ghc-pkg' lives, then
I guess that might be a reasonable way of doing things. However,
nothing is strange about my installation, it is the default but with a
different prefix, so something else must be broken. (And if 'hmake'
had just gone with $PATH then it would have worked :) )
(Continue reading)

Permalink | Reply | 0 comments |

Gmane