headers
Mark Wayland | 1 Sep 2005 01:13
Picon
Favicon

RE: The Quest for the (More) Perfect Build Tool

> Our typical projects compile all of the files in specified
> directories, so you can just create a new .cpp file in the
> appropriate directory and the file will get compiled
> automatically (no messing with .dsps or adding files to a
> makefile).

How did you set this up with JAM?

As far as I could tell using my in-depth 1 day look at it
(tongue in cheek) it doesn't do this out of the box - our
makefile solution does exactly that using

S_FILES := $(wildcard $(SRC_DIR)*.s) $(wildcard
$(SRC_DIR)$(PLATFORM)/*.s)
C_FILES := $(wildcard $(SRC_DIR)*.c) $(wildcard
$(SRC_DIR)$(PLATFORM)/*.c)
CPP_FILES := $(wildcard $(SRC_DIR)*.cpp) $(wildcard
$(SRC_DIR)$(PLATFORM)/*.cpp)

I also found that if I was to move over to JAM, I'd have to
pretty much rewrite the jambase to support cross compiling for
our multiple platforms and build targets.

I've set up our makefile stuff to be modular to the point where
a new project only requires a small, simple makefile which in
turn includes our common build makefile stuff. I had to write my
own dependency generator, as every single one on the web was
broken in one form or another, but that wasn't too difficult.

Regards,
(Continue reading)

Permalink | Reply | 0 comments |
headers
Paul Du Bois | 1 Sep 2005 02:28
Picon
Gravatar

Re: The Quest for the (More) Perfect Build Tool

On 8/30/05, Scott Shumaker <sshumaker <at> naughtydog.com> wrote:
> [...] functionality, as well as optimizations like header file dependency caching,

I remember using jam at a previous job back in '94, and it was rather slow
until we added dependency caching. Has this not made its way into the
mainline yet? And if not, is there a standard patch that people use? Google
seems to indicate not.

Paul Du Bois
Double Fine Productions
_______________________________________________
Sweng-gamedev mailing list
Sweng-gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 4 comments |
headers
Scott Shumaker | 1 Sep 2005 04:46
Favicon

RE: The Quest for the (More) Perfect Build Tool

Yeah, we added dependency caching, which gave us dramatic speedups.  It's
not standard, I'm afraid, but fortunately it wasn't that difficult to add,
even though we're using a newer version of JAM.  You can check out Craig
McPheeter's (sp?) JAM additions if you're interested.

Scott

-----Original Message-----
From: sweng-gamedev-midnightryder.com-admin <at> lists.midnightryder.com
[mailto:sweng-gamedev-midnightryder.com-admin <at> lists.midnightryder.com] On
Behalf Of Paul Du Bois
Sent: Wednesday, August 31, 2005 5:29 PM
To: sweng-gamedev <at> midnightryder.com
Subject: Re: [Sweng-gamedev] The Quest for the (More) Perfect Build Tool

On 8/30/05, Scott Shumaker <sshumaker <at> naughtydog.com> wrote:
> [...] functionality, as well as optimizations like header file 
> dependency caching,

I remember using jam at a previous job back in '94, and it was rather slow
until we added dependency caching. Has this not made its way into the
mainline yet? And if not, is there a standard patch that people use? Google
seems to indicate not.

Paul Du Bois
Double Fine Productions
_______________________________________________
Sweng-gamedev mailing list
Sweng-gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
(Continue reading)

Permalink | Reply | 3 comments |
headers
Alen Ladavac | 1 Sep 2005 17:00

Re: The Quest for the (More) Perfect Build Tool

Multireply...

Nick:

Yes, you are right... after looking at it a bit longer, SCons appears so 
beautifully engineered that it would be a shame to just let it go like that. 
But how do you think adding parts from Jam would help Scons? Scons is pure 
Python. I will try the Python alternatives you mentioned, though.

And, thanks for the link on the ml! It went under my radar. I have mailed 
Steven to ask him whether he still takes help on this. :)

Mark:

GLOB built in does that. It takes some custom rules to make it work 
recursively, but its not overly complicated.

Paul:

Base Jam hasn't changed much recently. The best way is to grab a set of 3rd 
party patches for teh Jam's source and apply them all. It is pretty straight 
forward. Dependency caching is one of the most important.

--

-- 
Alen 

_______________________________________________
Sweng-gamedev mailing list
Sweng-gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
(Continue reading)

Permalink | Reply | 2 comments |
headers
Paul Du Bois | 1 Sep 2005 19:22
Picon
Gravatar

Re: The Quest for the (More) Perfect Build Tool

On 9/1/05, Alen Ladavac <alenl-ml <at> croteam.com> wrote:
> Base Jam hasn't changed much recently. The best way is to grab a set of 3rd
> party patches for teh Jam's source and apply them all. It is pretty straight
> forward. Dependency caching is one of the most important.

Alen,

I took Scott's suggestion and looked up Craig McPheeters, which led me
to a little bit of ML discussion but no patches. I noticed you
complimented him on his patch format, but my google skills must be
getting rusty because I can't find patches anywhere :-/. But he has an
area on the perforce public depot -- were you just pulling his source
from there?

Paul Du Bois
Double Fine Productions
_______________________________________________
Sweng-gamedev mailing list
Sweng-gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 1 comment |
headers
Alen Ladavac | 2 Sep 2005 09:01

Re: The Quest for the (More) Perfect Build Tool

Paul,

Here's the list of things I did to our version of Jam. (A snippet from 
updated readme, so it has this kind of formulation.)

-------------------------

The version contained here is based on Jam2.5rc3, or more exactly, the 
contents of the perforce.com public depot at changelist 03174, in other 
words //public/jam/src/... <at> 03174 on server public.perforce.com:1666.

Modifications done so far are:
- Added changes that allow Jam to self-compile under .NET 2003.
- Applied Jam patch by Miklos Fazekas, fixing problems with dependencies on 
actions that have multiple targets.
 "If an action has more than one target, before executing the action, jam
 should update all dependents of the targets.
 However jam will update only the dependets of the first target!"
- Modified Jam so that its output messages don't contain leading elipsis 
"...", because it confuses the .NET IDE.
- Applied header cache patch, by Craig McPheeters with modifications by Matt 
Armstrong. Originating from
  //guest/matt_armstrong/jam/hdrscan_cache/... <at> 03174.
- Modified Jam to self-compile with .pdb debug info, so that it can be 
profiled.
- Made Jam split long error messages because they can crash .NET IDE.
- Optimized the fix for dependencies on actions that have multiple targets 
to only touch an action once.

--------------------------
(Continue reading)

Permalink | Reply | 0 comments |
headers
Nick Trout | 6 Sep 2005 23:13

RE: The Quest for the (More) Perfect Build Tool


The traditional way of optimizing scripting languages is to profile
them, work out the bottlenecks and then write optimized (C/asm) code
with a binding back to the scripting language.

You were trying to profile SCons I think. This is pretty easy, you just
add a flag "--profile=scons.prof" and then use the standard Python
pstats package to process the output. E.g. :-

	python -c "import pstats; s=pstats.Stats('scons.prof');
s.strip_dirs(); s.sort_stats('calls'); s.print_stats();
s.sort_stats('cumulative'); s.print_stats()" > bgsprof.txt

will give you number of calls, ordered and cumulative time in calls,
ordered (there are other options). There are some example outputs below.
Just looking quite superficially at the data it looks like the SCons 10%
of it's time in the following memoizing function alone:

<<<<
def ALT8_MeMoIZeR_gen_key(argtuple, kwdict):
    ret = []
    for A in argtuple:
        X = getattr(A, '_MeMoIZeR_Key', None)
        if X:
            ret.append(X)
        else:
            ret.append(repr(A))
    for K,V in kwdict.items():
        ret.append(str(K))
        X = getattr(V, '_MeMoIZeR_Key', None)
(Continue reading)

Permalink | Reply | 1 comment |
headers
Alen Ladavac | 7 Sep 2005 08:45

Re: The Quest for the (More) Perfect Build Tool

In the mean time, I did some profiling on SCons using project generation 
scripts from Noel's benchmark. Here are the conclusions in short:

1) I was trying exactly what you said for profiling, but it didn't work. 
Then Steven told me that Scons profiling doesn't work with Python 2.4 and 
up. I moved to 2.3 and it works now.
2) Tried Psyco and IronPython. Psyco worked out of the box but didn't give 
any performance improvements. IronPython didn't want to run SCons. Seems 
like it is not a fully functioning Python implementation, rather an 
experiment.
3) The speed is linear according to the number of files. I was hoping for 
some kind of a hidden O(n^2) plot, but there isn't any. It's just slow 
because of a constant. It takes about 6 millisec to process one file (in the 
nothing-to-do situation). For 1000 files, it means 6 seconds. :( The number 
of 6 milliseconds would be over 10 million cycles on my CPU. It sounds like 
it really has quite an overhead.
4) Regarding the memoizer... it is a system they use to cache results of 
functions that are called very often. When profiling, it is better to turn 
that off, otherwise you don't really see what happens.
5) The first real bottleneck I have tracked down seems to be in the var 
substitition. (for e.g. "$CC $SOURCE -o $TARGET", ...) It takes about 40% of 
processing time.  (Note that I turn off the memoizer for call-based 
profiling, but not for real timings, sothis is 40% even with memoizer 
_enabled_!) Note that besides plain vars, it supports some strange 
constructs in there, so it is a very complex system, involving lot of 
recursion. It's all written in Python, and I'm not sure that rewriting it in 
C would be trivial, as it is interweaved with other systems I think, to get 
the var values and functions from them. I have asked Steven for his opinion 
on that subsystem, but didn't get a reply yet.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Drizzt | 7 Sep 2005 11:36
Picon
Favicon

How to find who has the rights?

What if I want to know who has the right over the source code of a certain game? Is there a way to find it?

Thanks,
Mauro.

____________________________________________________________
Libero Flat, sempre a 4 Mega a 19,95 euro al mese! 
Abbonati subito su http://www.libero.it

_______________________________________________
Sweng-gamedev mailing list
Sweng-gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 0 comments |
headers
Ken Noland | 7 Sep 2005 16:33

RE: How to find who has the rights?

Yeah, ask!

Either that or look at the back of the box of the game and go through
each company listed there and see what they do.

-Ken Noland

-----Original Message-----
From: sweng-gamedev-midnightryder.com-admin <at> lists.midnightryder.com
[mailto:sweng-gamedev-midnightryder.com-admin <at> lists.midnightryder.com]
On Behalf Of Drizzt
Sent: Wednesday, September 07, 2005 5:36 AM
To: sweng-gamedev
Subject: [Sweng-gamedev] How to find who has the rights?

What if I want to know who has the right over the source code of a
certain game? Is there a way to find it?

Thanks,
Mauro.

____________________________________________________________
Libero Flat, sempre a 4 Mega a 19,95 euro al mese! 
Abbonati subito su http://www.libero.it

_______________________________________________
Sweng-gamedev mailing list
Sweng-gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.
com
(Continue reading)

Permalink | Reply | 0 comments |

Gmane