headers
Roesler, Randy | 1 Aug 2001 21:38
Picon

RE: Anyone else using Jam for BIG projects?

Glen:

> -----Original Message-----
> From: Glen Darling [mailto:gdarling <at> cisco.com]
> 
>      Is anyone else out there using JAM for anything as 
> complex as this?

Our situation is not as large as you, but its comparable,
I think.

Our Jamrules is 4000 lines, builds on 4 targets (HP, Aix, Sun,
Nt) with 6 variants (various levels of debug, purify, quantify)

We have some 6000 source files with about 20 file types.

We have appoximately 500 Jamfiles with 12000 lines of rule
invocations. (Rules are never defined in Jamfiles, only Jamrules)

Our Jambase is so minimal, it contains juts 37 lines. All it really
does is call $TOP/Jamrules and ./Jamfile.  We have no recompiled
or relines jam executable for the last 2 years.  We last recompiled
to fix a bug in jam AIX archive support (the bug still exists 
in the official jam release).

Our Jam system as some 30,000 targets.

It takes about 8 hours to build the system on our fastest
machines, 1 hour is we use -j10.  On Solaris it takes
3 days ! (4 hours if we use multiple machines, -j 6, and
(Continue reading)

Permalink | Reply | 2 comments |
headers
Roesler, Randy | 1 Aug 2001 21:49
Picon

RE: Unnecessary recompilation

Main a : a.cpp helper.cpp

calls Objects a.cpp helper.cpp

which calls Object helper.o : helper.cpp

which calls C++ helper.o : helper.cpp

Main b : ... 

also makes a similar call sequence,
including C++ helper.o : helper.cpp

Since the C++ rules was invoked on
helper.o twice, Jam feels the need to 
invoke the C++ action twice.

Thats just the way that Jam was written.

You can see lots of evidence of this
behavior in the distrubuted jam rules.
Look for lines like

	if ! $($(<)-included))
      {
         $(v)-included = true ;
      }

> -----Original Message-----
> From: Arnt Gulbrandsen [mailto:arnt <at> gulbrandsen.priv.no]
(Continue reading)

Permalink | Reply | 6 comments |
headers
Roesler, Randy | 1 Aug 2001 22:09
Picon

RE: Problems with: custom rule; cross-directory depende ncies

I would add extension .wd to you Objects rule
(or UserObjects rule)

somthing like

	case .wd:
         switch $(<:S=)
         {
            case *_parser : C++ $(<) : $(>:S=_parser.cpp} ; Whale $(>); 
            case *_lexer : C++ $(<) : $(>:S=_lexer.cpp} ; Whale $(>); 
         }

Modify your Objects rule like (do't take me too literally

rule Objects
{
	local i j s x ;

	makeGrsistedName s : $(<) ;

	for i in $(s)
	{
		objectFiles x : $i ;

		for j in $(x)
		{
			Object $(j) : $(i);
		}
	}
}
(Continue reading)

Permalink | Reply | 0 comments |
headers
David Abrahams | 2 Aug 2001 00:05
Favicon
Gravatar

Re: Anyone else using Jam for BIG projects?

----- Original Message -----
From: "Roesler, Randy" <rroesler <at> mdsi.bc.ca>

> >      Has anyone working on easing the pain of header scanning
> > large numbers  of source files?
>
> No, and I don't think it is really required.  And something
> like the Boost Build System is the wrong way to go, in my
> oppion.

It seems a little unfair for you to toss that remark off without an
explanation. I respect your experience in building large projects. What
about Boost.Build do you think is inappropriate?

-Dave
Permalink | Reply | 0 comments |
headers
Roesler, Randy | 2 Aug 2001 01:55
Picon

RE: Anyone else using Jam for BIG projects?

Yes, Sorry, I guess its a bit unfair.  A few weeks ago (months ago :)
I intended to send you a more formal (and private) critism 
about the Boost system, but one things and then another, ...

By the time I sat down to write a reply, I had convinced my self
that you were far enough along that major critism could add no value.
(From my frequent job as code reviewer, I know that most often, 
going back five steps is not an option people will entertain).

If I was wrong, then I spend a Saturday and give you some feed back
that you can use.

What do I think is inappropriate ?  In a nut shell I think that 
the tact of adding "requirements" in the way that you have makes
the user's job of specifing jam files more difficult.  What
would I propose as a solution -- well, thats what I would need
a day to formulate.

> -----Original Message-----
> From: David Abrahams [mailto:david.abrahams <at> rcn.com]
> Sent: Wednesday, August 01, 2001 3:06 PM
> To: Roesler, Randy; 'Glen Darling'; jamming <at> perforce.com
> Cc: Roesler, Randy
> Subject: Re: [jamming] Anyone else using Jam for BIG projects?
> 
> 
> 
> ----- Original Message -----
> From: "Roesler, Randy" <rroesler <at> mdsi.bc.ca>
>
(Continue reading)

Permalink | Reply | 1 comment |
headers
Glen Darling | 2 Aug 2001 02:39
Picon
Favicon

Re: Anyone else using Jam for BIG projects?

Hi David,

Thanks very much for your prompt and thought-provoking reply!  Comments 
below....

At 03:21 PM 07/31/2001 -0400, David Abrahams wrote:

>----- Original Message -----
>From: "Glen Darling" <gdarling <at> cisco.com>
>
> > Hi,
> >
> > I work for Cisco Systems where JAM has been used for about 3 years as the
> > primary build tool on an embedded systems project I have been involved
>with
> > for 1 year.  We have been working in somewhat of a vacuum with respect to
> > JAM (though we have communicated directly with Christopher Seiwald, we
> > haven't really participated on this mailing list much since Karl
>Klashinsky
> > left our team).  As a first step to changing that isolation, I have read
> > all recent traffic on the archive for this list (since June 1, 2001 and
> > selected bits prior) and I have peeked at both Boost.Build and ntjam --
>way
> > cool stuff by the way -- which makes me regret being so insular over here.
> >
> > I have three questions for you.  Any answers will be appreciated...
> >
> > Our project currently consists of about 15,000 source files (several
> > million LoC) including 18 different source suffix types, two different
> > target CPUs, many different code variants for different
(Continue reading)

Permalink | Reply | 2 comments |
headers
Glen Darling | 2 Aug 2001 03:22
Picon
Favicon

RE: Anyone else using Jam for BIG projects?

Hi Randy,

Thanks for your response!  Your environment looks very similar to ours in 
complexity (and I think also in the architecture of your JAM code).

My comments are below...

At 12:38 PM 08/01/2001 -0700, Roesler, Randy wrote:

>Glen:
>
> > -----Original Message-----
> > From: Glen Darling [mailto:gdarling <at> cisco.com]
> >
> >      Is anyone else out there using JAM for anything as
> > complex as this?
>
>Our situation is not as large as you, but its comparable,
>I think.
>
>Our Jamrules is 4000 lines, builds on 4 targets (HP, Aix, Sun,
>Nt) with 6 variants (various levels of debug, purify, quantify)
>
>We have some 6000 source files with about 20 file types.
>
>We have appoximately 500 Jamfiles with 12000 lines of rule
>invocations. (Rules are never defined in Jamfiles, only Jamrules)
>
>Our Jambase is so minimal, it contains juts 37 lines. All it really
>does is call $TOP/Jamrules and ./Jamfile.  We have no recompiled
(Continue reading)

Permalink | Reply | 0 comments |
headers
Roesler, Randy | 2 Aug 2001 06:44
Picon

RE: Anyone else using Jam for BIG projects?

I just timed the header file scan portion of a jam invokation on
a local file system on HPUX, less than 1 minute.

2 minutes, 10 seonds, when everything is over NFS.

So the question would be, why does would you scan times be 
some much higher as to be a problem for you.

Remember, people who need to build the who tree will be 
waiting hours anyways.

To speed up dependency analysis without changing jam itself,
you could use makedepend and process the output to create 
a jam.deps file in each compoment which you then include into
your jam files.  You could use jam itself to rebuild the
dependencies, but that would definitely be a pre-invokation of 
jam.  This would also be error prone unless jam was invoked
from within a wrapper script.

(This was one of the reasons we went to jam in the first place.
Please happly live with systems like this using make, but Ive
been burnt too many times by builds that did not refresh their
dependencies.)

If you want the makedepends like step to be automatically run
within the same build of jam, then you would be looking at 
more extensive changes to jam.  

In make0, before headers is called, call a new function,
dependencies say.  In dependencies you would have to
(Continue reading)

Permalink | Reply | 4 comments |
headers
Glen Darling | 2 Aug 2001 07:23
Picon
Favicon

Whitespace As Delimiter -- Yuk!

Hey,

At 12:49 PM 08/01/2001 -0700, Roesler, Randy wrote:
> > From: Arnt Gulbrandsen [mailto:arnt <at> gulbrandsen.priv.no]
> > > Consider a simple Jamfile
> > > Main        a : a.cpp helper.cpp ;
> > > Main        b : b.cpp helper.cpp;
> > (I suppose the missing space on the second line is just a typo.)

You can kill that "missing-space-before-semicolon" bug once and for all by 
tweaking your JAM source as follows:

In scan.c, in yylex(), around line 300 it has:

       if( ( c = yychar() ) == EOF || !inquote &&  ( isspace( c ) ) )
           break;
        }
        /* Check obvious errors. */

You can tweak that to:

       if( ( c = yychar() ) == EOF || !inquote &&  ( isspace( c )  || c == 
';' ) )
           break;
        }
        /* Check obvious errors. */

Unfortunately that's a bit like trying to soak up a flood with a single 
bread crumb and you *cannot* kill all other such token merging problems 
this easily.  Just try adding in a similar check for ':' and you'll see
(Continue reading)

Permalink | Reply | 2 comments |
headers
Arnt Gulbrandsen | 2 Aug 2001 12:17
Picon
Favicon
Gravatar

Re: Unnecessary recompilation

"Roesler, Randy" <rroesler <at> mdsi.bc.ca>
> Since the C++ rules was invoked on
> helper.o twice, Jam feels the need to 
> invoke the C++ action twice.
> 
> Thats just the way that Jam was written.

I understand the logic that makes it happen, but fail to see any argument
for this behaviour.

> You can see lots of evidence of this
> behavior in the distrubuted jam rules.
> Look for lines like
> 
> 	if ! $($(<)-included))
>       {
>          $(v)-included = true ;
>       }

That sounds like the author realized what was happening... but I still
don't see any reason for it. Shouldn't the action execution code think
"Oh, I've done this exact action, no need to do it again"?

--Arnt
Permalink | Reply | 2 comments |

Gmane