headers
Vladimir Prus | 1 Sep 2003 10:05
Picon
Favicon

Buglet in :P and :D docs.

Hi,

a Boost.Build user noted the following problem with Jam documentation which 
is present in main Jam line as well.

Jurko "GospodentiФ" wrote:
> 3. Boost.Jam docs (Make(1)-redux) say that on Unix and NT machines
> $(var:P) and $(var:D) are the same. But that is not true because on my
> machine $(var:P) contains the grist while $(var:D) doesn't.
>   Example:
>   aaa = "<xxx>yyy/shoot.me" ;
>   ECHO Without grist: $(aaa:D) ;   ECHO With grist: $(aaa:P) ;

- Volodya
Permalink | Reply | 0 comments |
headers
Paul Forgey | 9 Sep 2003 21:29

precompiled headers using msvc and working with -j

I've seen requests bounced around the list for such a thing.  I've come 
up with something that's been working quite well here for a while, 
although it is kind of ugly.  Then again, vc's handling of pre-compiled 
headers is a mess if you try to make it work in an smp build 
environment anyway.

Hopefully, we'll be able to keep the same external Jamfile interface to 
it when we wire up precompiled headers to gcc-3.4 in our build system.

To use it, invoke the PreCompiledHeaders rule, with $(1) being the list 
of source files to be injected with the precompiled header, and $(2) to 
be the header file (single header file!).  Two pch files will be 
generated; one for C and the other for C++.  For all C sources, the 
compiler forces in an #include of $(2) using the C compiled pch output, 
and likewise for C++.  That means you want proper #ifdef guarding and 
#ifdef/#ifndef __cplusplus sections in your pch and any header files it 
includes.

Assuming your header files are properly guarded, your code can continue 
to include the same files your precompiled header did with little 
impact on build performance.  This is important if the same code is to 
be built where precompiled headers aren't supported.  In this case, 
that's presently everything not built with msvc.

This isn't quite the Microsoft paradigm for handling precompiled 
headers, but it seems closer to the way most other compilers do it.  
Since we build on lots of platforms using different compilers, we 
wanted something that would translate well to non msvc environments.

I don't like how I added .h to the UserObject rule to support this.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Peter Björklund | 14 Sep 2003 00:40
Picon
Favicon

Automatic inclusion of source files

Hi!

Since Jam automatically parses the dependencies of the header files, how can
I get it to automatically include add the source files as dependencies? I
have a lot of source files (several hundred) in sub directories. Now, I
obviously don't want to add them all in my jamfile. How can I tell Jam to do
it for me?

Best regards,
Peter Bjorklund
Permalink | Reply | 0 comments |
headers
Peter Björklund | 14 Sep 2003 11:11
Picon
Favicon

Re: Automatic inclusion of source files

Thanks Alen!

Your code solved most of my jam problems!

However, I have a couple óf libraries (again with several hundred source
files in subdirectories) that doesn't come with any jam or make files. The
problem with those are that they are muilti platform (e.g. GameCube,
Playstation, Palm, MacOS etc) and those obviously fails to compile on the
"wrong" platform.

So therefore I am interested in only having those .cpp files that have been
determined that there is an include dependency with. Jam must be able to
correctly parse "#ifdef" and "#if defined(xxx)" macro statements.

The solution for me now is that everytime I get a new version of the library
sources, I delete the directories and source files that contain code for the
"wrong" platform. This is *very* time consuming.

Maybe there is a program which can correctly parse the dependencies and then
feed that list to jam?

Any help would be much appreciated!

Best regards,
Peter Bjorklund

----- Original Message ----- 
From: "Alen Ladavac" <alen <at> croteam.com>
To: "Peter Björklund" <piot <at> hotmail.com>
Sent: Sunday, September 14, 2003 12:13 PM
(Continue reading)

Permalink | Reply | 7 comments |
headers
Alen Ladavac | 14 Sep 2003 13:31

Re: Automatic inclusion of source files

I don't understand how could include dependencies between .cpp files be determined automatically?
You don't #include .cpp files, do you?

We also build for multiple platforms here, but we solve that problem in a specific way (we used that
even before Jam, to improve readability and keep our sanity :)). We put all platform-specific parts
of each library in subdirectory of a Sys subdir. Like SomeLibrary/Sys/Win32, SomeLibrary/Sys/XBox,
SomeLibrary/Sys/SDL, etc. My original version of RGLOB has additional case in ListDir to skip */Sys
when searching directories. Then I can manually add needed per-platform subdirs, like this:

FILES = [ RGLOB $(PRJDIR) : *.cpp *.c ] ;

if $(CONFIG) = Win32 {
    FILES += [ RGLOB $(PRJDIR)\\Sys\\Win32 : *.cpp ] ;
} else # .... etc.

Main MyApp : $(FILES) ;

Cheers,
Alen

----- Original Message -----
From: "Peter Björklund" <piot <at> hotmail.com>
To: "Alen Ladavac" <alen <at> croteam.com>; <jamming <at> perforce.com>
Sent: Sunday, September 14, 2003 9:11 AM
Subject: Re: [jamming] Automatic inclusion of source files

> Thanks Alen!
>
> Your code solved most of my jam problems!
>
(Continue reading)

Permalink | Reply | 0 comments |
headers
Peter Björklund | 14 Sep 2003 14:58
Picon
Favicon

Re: Automatic inclusion of source files

Hi again!

> I don't understand how could include dependencies between .cpp files be
determined automatically?
> You don't #include .cpp files, do you?
No, but in most cases each .h corresponds to  a .cpp file in the same
directory. There are of course exceptions to this rule, sometimes the
definition is located elsewhere, sometimes there isn't a definition at all
(pure virtual interface) and sometimes it is a different source type (.inc,
.asm etc). However, it is easier to set up a dependency for those files that
doesn't follow this rule, than the other way around.

> We put all platform-specific parts
> of each library in subdirectory of a Sys subdir. Like
SomeLibrary/Sys/Win32, SomeLibrary/Sys/XBox,
> SomeLibrary/Sys/SDL, etc.

That is what I would like to have as well. The designers of those libraries
thought it was better to do it like this:
SomeLibrary/Input/Win32/...
SomeLibrary/Input/MacOs/...
SomeLibrary/Graphics/Win32/...
SomeLibrary/Graphics/MacOs/...
.....

The best solution might be to ask them to change their directory structure.
I certainly would try to do that, cause it would make things a lot easier :)

Thanks again Alen!
(Continue reading)

Permalink | Reply | 1 comment |
headers
Alen Ladavac | 14 Sep 2003 18:00

Re: Automatic inclusion of source files

> That is what I would like to have as well. The designers of those libraries
> thought it was better to do it like this:
> SomeLibrary/Input/Win32/...
> SomeLibrary/Input/MacOs/...
> SomeLibrary/Graphics/Win32/...
> SomeLibrary/Graphics/MacOs/...
> .....
>
> The best solution might be to ask them to change their directory structure.
> I certainly would try to do that, cause it would make things a lot easier :)

You could modify ListDir so that it enters special dirs (Win32, MacOs, ...) only if a particular var
is set. That would work as well, even though the directory structure is not as tidy.

> PS.  We are in the same business, I work for Dice (creators of Battlefield
> 1942, Midtown Madness 3, Rallisport Challenge etc) and I think Sam is
> serious fun :o)

Thanks. I hear that BF1942 is a great game as well, I wish I could say that from personal
experience, but I don't get to play a lot games lately. Isn't that ironic? :)

Anyway, I wish you luck with Jam, and let me know if you find out some useful tricks in it.

Alen

----- Original Message -----
From: "Peter Björklund" <piot <at> hotmail.com>
To: <jamming <at> perforce.com>
Cc: "Alen Ladavac" <alen <at> croteam.com>
Sent: Sunday, September 14, 2003 12:58 PM
(Continue reading)

Permalink | Reply | 0 comments |
headers
Alen Ladavac | 14 Sep 2003 18:40

Batching compile actions _does_ make a difference! _Huge_ difference!

Hi all,

I'd like to dust off this topic again. I searched the archives and found that
it was already discussed before, but with no result.

To reiterate.... Some (most?) compilers are able to compile more than one
source file in one invocation. As the things are now, Jam is able to batch
several updates of same library archive together, but it cannot batch C++
compilations because the target files are different.

The previous discussion has, AFAIK, ended with a claim that this ability would
not gain any significant performance improvements. I would like to show you
results of tests I've made which show that improvements are _very_
significant!

library A:
  separate: 31 seconds
  batched: 22 seconds
library B:
  separate: 351 seconds
  batched: 195 seconds

Horrors... this is almost _2 times_ slower than it should be!

Does anyone have some solutions or workarounds for this problem? Or at least
some pointers on what would be the simplest wat to add this functionality to
Jam source, or even better to Jambase. I thought of maybe writing a special
rule that will collect sources with same flags and create a pseudo target for
them. But I don't think I could do that without forcing recompilation even on
those that haven't changed.
(Continue reading)

Permalink | Reply | 5 comments |
headers
Matt Armstrong | 14 Sep 2003 16:51

Re: Batching compile actions _does_ make a difference! _Huge_ difference!

Alen Ladavac <alen <at> croteam.com> writes:
[...]
> library A:
>   separate: 31 seconds
>   batched: 22 seconds
> library B:
>   separate: 351 seconds
>   batched: 195 seconds
> 
> Horrors... this is almost _2 times_ slower than it should be!
>
> "Batched" case was measured as being compiled using the Visual's
> proprietary .vcproj format (for non-Win32 people, .vcproj is a very
> weird concept of holding a list of .cpp files and their settings in an
> .xml file).

I would be interested to see how your tests turned out if they were done
in the same way jam might batch compile.  E.g. compile groups of 10 or
20 source files together.  This is the kind of test I ran 9 months ago
and did not see the performance gain you see (this was with VC++ 6.0).

Also, I wonder if pre-compiled headers entered the picture when
compiling with VC++.NET.  They usually don't with jam.
Permalink | Reply | 4 comments |
headers
Alen Ladavac | 14 Sep 2003 20:01

Re: Batching compile actions _does_ make a difference! _Huge_ difference!

Hi Matt,

> I would be interested to see how your tests turned out if they were done
> in the same way jam might batch compile.  E.g. compile groups of 10 or
> 20 source files together.

Why only 10 or 20? Why not all files with same flags? And even if it is only
10 files at the time, it will be much better than 1 at a time.

> This is the kind of test I ran 9 months ago
> and did not see the performance gain you see (this was with VC++ 6.0).

Yes, I know, I've seen your posts, but but the key points are:

1) Large project.
2) Lot of includes (DX9 and windows.h are evil).
3) Must use precompiled headers, and must use the manual version (/Yc, /Yu),
not automatic one (/YX) as it is not good enough.
4) There are no tricks, the compiler used by the .vcproj is same command
line cl.exe, the GUI front end is just sugar coating.
5) Batching does matter. A lot.

The thing is that for a lot of includes, you _must_ have precompiled headers,
and must have them tuned well. Without it, it is so slow that nothing makes a
difference. But if you have precompiled headers for a large project, the .pch
file is several megabytes (cca 7MB in the library B). Each invocation of the
compiler must load that file. I don't know exactly what is going on, but I
speculate that even though the .pch stays in file cache, compiler must parse
its data and recreate its internal lists from it.
(Continue reading)

Permalink | Reply | 3 comments |

Gmane