Bob Lee | 1 Mar 06:49 2010

RE: Using build.sh for cross builds


	Some of you expressed interest if I came up with a solution.  I have a hack that works.  It allows for a
centralized read-only TOOLDIR, I look for an environment variable TRUST_TOOLDIR to be "yes".  I didn't
take the time to make it a build.sh option, and it has a side effect.  The nbmake wrapper build I came up with a
manageable hack.  It results in a local version of the nbmake-* being created in the NETBSDSRCDIR.
	It would be nice if this was a flag in build.sh, and to skip the nbmake wrapper build if the tools aren't being
built.  The ability to over-ride OBJDIR and DESTDIR from the local environment/build.sh input --
currently, I'm changing those by hand right now.  The only reason I mention it is, that I may complete these
features.  Right now is not possible.
	I've attached a context diff of build.sh and Makefile.  Hope it helps someone else.

- bob
Attachment (ROtooldir.diff): application/octet-stream, 3502 bytes
David Holland | 1 Mar 08:52 2010
Picon

Re: updating a piece of the running system from build.sh is too hard

On Fri, Feb 26, 2010 at 01:36:44PM -0500, Greg Troxel wrote:
 > > I think that it would be useful to have a simple way of telling
 > > ${TOOLDIR}/bin/nbmake-${MACHINE} to install as if MKUNPRIVED had not
 > > been set at the time the wrapper was created.  I haven't thought enough
 > > about how to do that.
 > 
 > "sudo nbmake-$arch DESTDIR=/ MKUNPRIVED=no install" seems to do exactly
 > this but I'm not 100% certain.  I propose
 > 
 >   sudo nbmake-$arch live-install
 > 
 > to be an equivalent shortcut.

ISTM (if we're talking about rearranging stuff) that this should be
"install", and the install that goes into $OBJ/destdir.$ARCH/...
should be called something else, like stage-install or install-staging
or whatnot.

That is, we may as well admit that (like pkgsrc) we have two
installation trees, one used during the build and one that the
completed build is dropped into later.

A full build should always build into the local destdir (and IMO it
should always use METALOG, but that's a separate discussion) and not
touch the ultimate destdir (which is usually /) until its final
install phase. However, if I manually do 'make && make install'
somewhere it normally means I want to install directly into the
ultimate destdir.

Our build system is way too complicated and doesn't need 7/8 of the
(Continue reading)

Masao Uebayashi | 1 Mar 09:20 2010
Picon

Re: updating a piece of the running system from build.sh is too hard

> ISTM (if we're talking about rearranging stuff) that this should be
> "install", and the install that goes into $OBJ/destdir.$ARCH/...
> should be called something else, like stage-install or install-staging
> or whatnot.
> 
> That is, we may as well admit that (like pkgsrc) we have two
> installation trees, one used during the build and one that the
> completed build is dropped into later.

Understood, but I'm looking forward more; syspkgs.  If we always create
packages from the staged (dest)dir and install them via pkg_install(8), we
can concentrate installation path in single point; we use the same method to
install NetBSD in sysinst(8), line install, or embedded target image creation.
This is a huge win.

> Our build system is way too complicated and doesn't need 7/8 of the
> combinations it supports.

Fully agreed.  If we could agree that the current design is complicated and
should be simplified to improve further, only that we agree, that'd be a BIG
one step forward...

Masao

--

-- 
Masao Uebayashi / Tombi Inc. / Tel: +81-90-9141-4635

Greg Troxel | 1 Mar 14:05 2010
Picon

Re: updating a piece of the running system from build.sh is too hard


David Holland <dholland-tech <at> netbsd.org> writes:

> On Fri, Feb 26, 2010 at 01:36:44PM -0500, Greg Troxel wrote:
>  > > I think that it would be useful to have a simple way of telling
>  > > ${TOOLDIR}/bin/nbmake-${MACHINE} to install as if MKUNPRIVED had not
>  > > been set at the time the wrapper was created.  I haven't thought enough
>  > > about how to do that.
>  > 
>  > "sudo nbmake-$arch DESTDIR=/ MKUNPRIVED=no install" seems to do exactly
>  > this but I'm not 100% certain.  I propose
>  > 
>  >   sudo nbmake-$arch live-install
>  > 
>  > to be an equivalent shortcut.
>
> ISTM (if we're talking about rearranging stuff) that this should be
> "install", and the install that goes into $OBJ/destdir.$ARCH/...
> should be called something else, like stage-install or install-staging
> or whatnot.

I disagree, because there is historically one notion of install and the
destdir setup as part of build.sh and make variables says where that
goes.  install should go into DESTDIR, always.

What I was trying to do is say

  I know the build was set up to go to DESTDIR, and UNPRIVED, and I know
  that I could set USETOOLS=no and redo the build and install normally,
  but what I want now is to reuse the existing build and do an
(Continue reading)

Masao Uebayashi | 2 Mar 19:26 2010
Picon

config(5) break down

I want to slowly start breaking down config(5) files (sys/conf/files,
sys/**/files.*) into small pieces.  The goal is to clarify ownership of files;
lines like "file aaa.c bbb | ccc" are to be changed into "file aaa.c"
(ownership) and "ccc: bbb" (dependency).  Because in the modular world one
file belongs to one module.  Otherwise you end up to have two copies of aaa.o
in module "bbb" and "ccc".

Broken config(5) files will be named like "<module>.conf", because files.*
namespace is insufficient.  For example pci.kmod can't use files.pci.

sys/net*, sys/dev/scsipi, and sys/dev/wscons need serious rework...

Masao

--

-- 
Masao Uebayashi / Tombi Inc. / Tel: +81-90-9141-4635

Masao Uebayashi | 3 Mar 13:27 2010
Picon

Re: config(5) break down

config(5) has 4 ways to define a collection of sources:

			initialization	interface	dependency	configurable
	device		autoconf(9)	bdev/cdev	y		y
	defpseudodev	xxxattach	bdev/cdev	n		y
	defpseudo	xxxattach	-		n		y
	define		-		-		y		n

I think "device" part is fine.  Device drivers are configured by autoconf(9),
and defined as "device".  Used with interface attributes (define xxx {}),
most of problems can be resolved.  (Some complex ones like scsipi / wscons
are their problems, not config(5).)

In sys/net* and some sys/kern there're some code that can be modular.  Most
of them are defined as either defpseudo or define.  If it's user configurable,
it should be defpseudo.  If it's simply a shared code, define.

There're lots of confusions in sys/conf/files; some are defflag or defparam.
Anything which _owns_ *.c, AND user-configurable should be defpseudo.  We'll
have "defpseudo inet" or "defpseudo ipsec" eventually.

defpseudo lacks some points:

- No dependency.
- No detach.

I think defpseudo's initialization and module's one can be merged.  Dependency
is also addressed naturally if defpseudo is always handled as modules.  config(1)
is taught about defpseudo's dependency.  If config fragment like:

(Continue reading)

Masao Uebayashi | 3 Mar 13:30 2010
Picon

Re: config(5) break down

> 	defpseudo inet
> 	:
> 	defpseudo gif
> 	file if_gif.c gif

Should have been read as:

 	defpseudo inet
 	:
 	defpseudo gif: inet <---
 	file if_gif.c gif

David Holland | 4 Mar 06:16 2010
Picon

Re: config(5) break down

On Wed, Mar 03, 2010 at 03:26:20AM +0900, Masao Uebayashi wrote:
 > I want to slowly start breaking down config(5) files
 > (sys/conf/files, sys/**/files.*) into small pieces.  The goal is to
 > clarify ownership of files; lines like "file aaa.c bbb | ccc" are
 > to be changed into "file aaa.c" (ownership) and "ccc: bbb"
 > (dependency).

I'm not entirely convinced that makes sense in all cases, but I
suppose it probably does in most.

 > Because in the modular world one file belongs to one module.

Perhaps a first step would be using config(1) and files.* to generate
the module makefiles instead of maintaining them by hand...

 > Broken config(5) files will be named like "<module>.conf", because files.*
 > namespace is insufficient.  For example pci.kmod can't use files.pci.

Huh? I don't understand.

--

-- 
David A. Holland
dholland <at> netbsd.org

David Holland | 4 Mar 06:23 2010
Picon

Re: updating a piece of the running system from build.sh is too hard

On Mon, Mar 01, 2010 at 05:20:35PM +0900, Masao Uebayashi wrote:
 > > ISTM (if we're talking about rearranging stuff) that this should be
 > > "install", and the install that goes into $OBJ/destdir.$ARCH/...
 > > should be called something else, like stage-install or install-staging
 > > or whatnot.
 > > 
 > > That is, we may as well admit that (like pkgsrc) we have two
 > > installation trees, one used during the build and one that the
 > > completed build is dropped into later.
 > 
 > Understood, but I'm looking forward more; syspkgs.  If we always
 > create packages from the staged (dest)dir and install them via
 > pkg_install(8), we can concentrate installation path in single
 > point; we use the same method to install NetBSD in sysinst(8), line
 > install, or embedded target image creation.  This is a huge win.

It may be (I remain unconvinced) but it's also completely irrelevant
to what I (and most everybody else) wants to have happen if I go into
src/bin/sh and type 'make && make install'.

If that would require grinding around and installing some whole syspkg
instead of just a new sh, that would be a big big regression. If it
doesn't require that, syspkgs are irrelevant to what I'm talking
about.

 > > Our build system is way too complicated and doesn't need 7/8 of the
 > > combinations it supports.
 > 
 > Fully agreed.  If we could agree that the current design is complicated and
 > should be simplified to improve further, only that we agree, that'd be a BIG
(Continue reading)

David Holland | 4 Mar 06:46 2010
Picon

Re: updating a piece of the running system from build.sh is too hard

On Mon, Mar 01, 2010 at 08:05:32AM -0500, Greg Troxel wrote:
 > > ISTM (if we're talking about rearranging stuff) that this should be
 > > "install", and the install that goes into $OBJ/destdir.$ARCH/...
 > > should be called something else, like stage-install or install-staging
 > > or whatnot.
 > 
 > I disagree, because there is historically one notion of install and the
 > destdir setup as part of build.sh and make variables says where that
 > goes.  install should go into DESTDIR, always.

Yes, historically there's one notion of install. Historically, the
system build installs a new libc into /usr/lib and then recompiles the
rest of the tree against it, and if it turns out to be no good you'd
better have an /altroot or similar handy. This is not a good model,
and therefore we don't do that any more; we build into a private
DESTDIR and then install separately to another DESTDIR afterwards.

Given that there are two DESTDIRs it doesn't help anything to say that
install should always go into DESTDIR. The whole point of my previous
post is that sometimes you want one and sometimes you want the other,
and therefore you should select what you want (stage-install vs.
install, for example) rather than muck around changing DESTDIR and
hoping the right things happen.

(Note also that right now the depend info becomes incorrect if you
change DESTDIR on the fly; this theoretically shouldn't affect install
targets but it's nonetheless a potential risk.)

 > What I was trying to do is say
 > 
(Continue reading)


Gmane