mylex eisa dac 960

1.6D sysinst... [long]

Folks,

trying to upgrade a NetBSD 1.5something installation on a Macintosh Quadra
700, I've taken some notes. I will 'cross-post' this list to tech-install
and port-mac68k since it contains general as well as mac68k specific items;
a discussion should probably move to tech-install.

Yes, I've managed to upgrade the machine with sysinst and not cause
permanent damage - but, oh, the pain...

And -- please take this list as constructive criticism, not flamage of
those who worked on sysinst.

The first part is somewhat chronological, while the second lists some more
random points.

==1==

# Sysinst offers to upgrade the root ffs; this is probably not appropriate
for mac68k. Does the booter understand the current FFS? The MacOS installer
tooldefinitely doesn't, and the upgrade cannot be undone.

--> Skip this message (for the root ffs at least) on mac68k.

# Message "...first part... is finished. Sysinst has written a disklabel to
the target disk, ..." is bogus and annoying for mac68k.

--> By all means skip this message on mac68k. Don't make my heart jump by
falsely telling me you've just trashed the partition table.

# In the FTP menu, I got control sequences from the cursor keys, and a
messed up screen. A ^L did not help. (Cursor keys worked elsewhere; over
the 15 or so re-iterations of the process, problem did not reoccur.)

# The dhcp client option is not working ('dhclient failed').

Then, serious problems began...

# xbase.tgz could not be unpacked, because the directories
/usr/X11R6/lib/X11/rstart/commands/{x,x11} could not be removed.  I had the
option of installing the remaining sets, then the upgrade was aborted, and
sysinst told me to manually reinstall the missing set(s).
I tried that, and the reinstall was aborted because of a missing
/etc/fstab. Right -- the previous attempt had dropped everything on the
floor because of a missing xbase.tgz, leaving a half-done /etc.

--> Keep state and history. Let me back out, one step at a time. Give me
the chance to recover and resume. If you feel you must abort, either undo
any changes, or at least list the changes made so far.

# On the other hand, failing to remove /usr/include/mac68k does not cause
sysinst to abort unconditionally. That's good.

--> Be consistent if you can.

# Since sysinst had left the disk in an unknown state, I had to start the
upgrade procedure from scratch. Shell prompt, mount sd0a, restore etc,
unmount. Start upgrade procedure. Upgrade aborted unconditionally because
sysinst cannot move /usr/X11R6/bin/X (a symlink!) to X.old. Excuse me?

--> Distinguish between fatal and non-fatal issues, and, in the latter
case, give me a chance to clean up from a shell prompt and resume. Don't
just assume, and do your thing. Offer alternatives, give me rope.

# If for whatever reason a fscked partition was unclean, the install
aborts. Having sysinst unconditionally quit on me because I forgot to
unmount a partition... has bitten me a hundred times, and still hurts like
the first time.

--> Stop tripping over your own feet: Unmount disks properly before fscking
them.

# sysinst fscked /dev/sd0d, had it marked clean and then aborted the
install -- repeatedly. Worked again after yet another reboot; happened
again later with /dev/sd0a.

# sysinst 'Full Installation' defaults to _not_ installing a kernel during
an upgrade. Reboot and be surprised.

--> Respect the principle of least surprise.

# Mis-type the target dir path for an ftp install, and you have to start
from scratch.

# sysinst remembers domain name, hostname, IP address and netmask from a
network setup, but not the nameserver and gateway IPs.

# Why does "install additional sets" unconditionally re-make devices and
mess with /etc?

==2==

# Why does 'umount -a' need /etc/fstab around? What is unclear about 'all'?

# US kbd layout on a non-US kbd _and_ no cli editor is just too much of a PITA.

--> For the not floppy-size-constrained install kernels, add command line
history in sh, and add switchable key maps.

# For installation via ftp, show|check available space on local
filesystems. Let me recover from disk space shortage.

# Treat /usr/X11R6/lib/X11 like /etc, don't just overwrite its contents.

# Factor out actions like transition a.out -> elf, upgrading /etc or
/usr/X11R6/lib/X11, and make them available in a menu.

# INSTALL.mac68k has nothing about /etc/postinstall or /usr/sbin/etcupdate.
postinstall does not even have a manpage.

# How does /etc/postinstall relate to the sysinst /etc upgrade procedure?
Looks like you can do one, or the other.

Overall impression: While sysinst has become quite polished in some areas,
its structure is still very rigid and unforgiving. Stroll from the Right
Path, and you're done for.

At this point, I'd advise to ignore sysoinst and upgrade the traditional
way: Boot with the new kernel to single user, unpack the tarballs, upgrade
/etc with /etc/postinstall and /usr/sbin/etcupdate.

	hauke

--
/~\  The ASCII Ribbon Campaign       "They that can give up essential liberty
\ /    No HTML/RTF in email          to obtain a little temporary safety
 X     No Word docs in email         deserve neither liberty nor safety."
/ \  Respect for open standards                    -- Benjamin Franklin, 1759

Re: 1.6D sysinst... [long]

I can't address all this issues, but I might be able to offer some 
insight on some of them.

On Saturday, July 20, 2002, at 04:41 AM, Hauke Fath wrote:

> # Sysinst offers to upgrade the root ffs; this is probably not 
> appropriate
> for mac68k. Does the booter understand the current FFS? The MacOS 
> installer
> tooldefinitely doesn't, and the upgrade cannot be undone.

The filesystems created with Mkfs/Installer are "old" type; the 
filesystems created with sysinst are "new" type.  I believe the Booter 
only knows about "new" type filesystems, but the two filesystems are 
similar enough that the Booter works on "new" filesystems that are kept 
fairly small.  This has been discussed on the port-mac68k list in the 
past.  I don't think the filesystem is converted during an upgrade, so 
I'm not sure why this is even an issue for an upgrade.  As for a full 
install this issue is covered in the INSTALL documentation that was 
recently committed.  Preliminary copies were made available for download 
and this was posted on the port-mac68k list.

> --> Skip this message (for the root ffs at least) on mac68k.
>
> # Message "...first part... is finished. Sysinst has written a 
> disklabel to
> the target disk, ..." is bogus and annoying for mac68k.

The message is in the MI portion of sysinst and isn't necessary totally 
bogus.  Since the disklabel for the mac68k port is dynamically 
constructed from the Apple Disk Partition Map which could have been 
modified by the sysinst process, the message is mostly correct.

> # In the FTP menu, I got control sequences from the cursor keys, and a
> messed up screen. A ^L did not help. (Cursor keys worked elsewhere; over
> the 15 or so re-iterations of the process, problem did not reoccur.)

This is an MI problem.  There are a number of PRs written against 
sysinst and some cover subjects like this.

> # The dhcp client option is not working ('dhclient failed').

Sysinst has in the past had problems configuring and handling the 
network connection. This isn't a mac68k issue though since it happens in 
many of the other ports as well.  The dhcp option was working for the 
person who tested it on the mac68k port though.

> # xbase.tgz could not be unpacked, because the directories
> /usr/X11R6/lib/X11/rstart/commands/{x,x11} could not be removed.  I had 
> the
> option of installing the remaining sets, then the upgrade was aborted, 
> and
> sysinst told me to manually reinstall the missing set(s).
> I tried that, and the reinstall was aborted because of a missing
> /etc/fstab. Right -- the previous attempt had dropped everything on the
> floor because of a missing xbase.tgz, leaving a half-done /etc.

This sounds like an MI problem with sysinst.

> # Since sysinst had left the disk in an unknown state, I had to start 
> the
> upgrade procedure from scratch. Shell prompt, mount sd0a, restore etc,
> unmount. Start upgrade procedure. Upgrade aborted unconditionally 
> because
> sysinst cannot move /usr/X11R6/bin/X (a symlink!) to X.old. Excuse me?

I think this is caused by not having the right option on the "pax" 
command.  We found a situation like this in the MD part of sysinst and 
fixed it so I assume this is in the MD part.

> # If for whatever reason a fscked partition was unclean, the install
> aborts. Having sysinst unconditionally quit on me because I forgot to
> unmount a partition... has bitten me a hundred times, and still hurts 
> like
> the first time.

Probably a MI issue.  Sysinst has gotten better with things like this 
though.

> # sysinst fscked /dev/sd0d, had it marked clean and then aborted the
> install -- repeatedly. Worked again after yet another reboot; happened
> again later with /dev/sd0a.

Haven't a clue.

> # sysinst 'Full Installation' defaults to _not_ installing a kernel 
> during
> an upgrade. Reboot and be surprised.

This has been fixed in -current and 1.6 although you shouldn't have been 
surprised since there was an error generated that your system didn't 
have a kernel.  You probably blew past that like I did when it first 
happened to me.

> # Mis-type the target dir path for an ftp install, and you have to start
> from scratch.
>
> # sysinst remembers domain name, hostname, IP address and netmask from a
> network setup, but not the nameserver and gateway IPs.
>
> # Why does "install additional sets" unconditionally re-make devices and
> mess with /etc?

> # Why does 'umount -a' need /etc/fstab around? What is unclear about 
> 'all'?
>
> # US kbd layout on a non-US kbd _and_ no cli editor is just too much of 
> a PITA.
>
> --> For the not floppy-size-constrained install kernels, add command 
> line
> history in sh, and add switchable key maps.
>
> # For installation via ftp, show|check available space on local
> filesystems. Let me recover from disk space shortage.
>
> # Treat /usr/X11R6/lib/X11 like /etc, don't just overwrite its contents.
>
> # Factor out actions like transition a.out -> elf, upgrading /etc or
> /usr/X11R6/lib/X11, and make them available in a menu.
>
> # INSTALL.mac68k has nothing about /etc/postinstall or 
> /usr/sbin/etcupdate.
> postinstall does not even have a manpage.

The 1.6D INSTALL docs were not up to date.  Changes were committed a few 
days ago and preliminary documentation was available for download.

> # How does /etc/postinstall relate to the sysinst /etc upgrade 
> procedure?
> Looks like you can do one, or the other.
>
>
> Overall impression: While sysinst has become quite polished in some 
> areas,
> its structure is still very rigid and unforgiving. Stroll from the Right
> Path, and you're done for.
>
> At this point, I'd advise to ignore sysoinst and upgrade the traditional
> way: Boot with the new kernel to single user, unpack the tarballs, 
> upgrade
> /etc with /etc/postinstall and /usr/sbin/etcupdate.

Re: 1.6D sysinst... [long]

On Sat, 20 Jul 2002, Hauke Fath wrote:

> # Treat /usr/X11R6/lib/X11 like /etc, don't just overwrite its contents.

Why? It's a bad idea, in general, to customize installed files. Nor is
it required. For "xdm" for example, you could copy
/usr/X11R6/lib/X11/xdm/ to /etc/X11/xdm/, customize the files in /etc,
and set `xdm_flags="-config /etc/X11/xdm/xdm-config"' in /etc/rc.conf.

> # Factor out actions like transition a.out -> elf, upgrading /etc or
> /usr/X11R6/lib/X11, and make them available in a menu.

What's the issue with the a.out -> ELF transition? It should be a
one-time thing, and be unobtrusive for systems which are ELF already.

For "/etc", you could always decline to install the "etc.tgz" set, and
let "etcupdate" take care of it after booting into the new system.

> Overall impression: While sysinst has become quite polished in some areas,
> its structure is still very rigid and unforgiving. Stroll from the Right
> Path, and you're done for.
>
> At this point, I'd advise to ignore sysoinst and upgrade the traditional
> way: Boot with the new kernel to single user, unpack the tarballs, upgrade
> /etc with /etc/postinstall and /usr/sbin/etcupdate.

While most of the problems you point out are valid (and they all look
fixable), I think your overall assessment is overly harsh. You can get
used to sysinstall's warts: If the install bombs because of a typo in
the nfs path, for example, you can just hit ^D and start again -- no
need to reboot. I could never get used to the old Mac OS installer
taking 24 hours to unpack the sets on a Quadra, and what you call "the
traditional way" is hard to support (though I do that myself on the
one headless Quadra, it's not a very nice thing to present that
procedure to a new user), and won't do at all for new installs.

Frederick

Re: 1.6D sysinst... [long]

At 7:27 Uhr -0500 20.7.2002, Bob Nestor wrote:
>I believe the Booter only knows about "new" type filesystems,
>but the two filesystems are similar enough that the Booter
>works on "new" filesystems that are kept fairly small.

What is 'fairly small' here?

>This has been discussed on the port-mac68k list in the
>past.  I don't think the filesystem is converted during an upgrade,

Early during the upgrade procedure, sysinst offered to upgrade my root ffs.
2002-07-06 snapshot, to be precise.

>so I'm not sure why this is even an issue for an upgrade.

If you accept the upgrade offer, and your root filesystem does not match
the above 'fairly small', your only way back is to 'newfs -O' the partition
and restore from a backup.

Since this operation is (a) not reversible and can (b) trivially be done
with a 'fsck -c' in single user mode, I propose to not offer the option in
sysinst at that point.  IMHO, a notice would be enough.

>> # Message "...first part... is finished. Sysinst has written a
>> disklabel to
>> the target disk, ..." is bogus and annoying for mac68k.
>
>The message is in the MI portion of sysinst and isn't necessary totally
>bogus.  Since the disklabel for the mac68k port is dynamically
>constructed from the Apple Disk Partition Map which could have been
>modified by the sysinst process, the message is mostly correct.

Errm... For me at least, on mac68k there is a _big_ difference between
updating a disklabel (which sysinst has no business with during an upgrade,
either) and _writing_ a disklabel. Where 'disklabel', at least for me,
denotes a BSD disklabel, which has no place on a mac68k boot disk.

I have seen sysinst trash my HFS partition before, so my trust is limited.

	hauke

--
/~\  The ASCII Ribbon Campaign       "They that can give up essential liberty
\ /    No HTML/RTF in email          to obtain a little temporary safety
 X     No Word docs in email         deserve neither liberty nor safety."
/ \  Respect for open standards                    -- Benjamin Franklin, 1759

Re: 1.6D sysinst... [long]

At 10:31 Uhr -0500 20.7.2002, Frederick Bruckman wrote:
>On Sat, 20 Jul 2002, Hauke Fath wrote:
>
>> # Treat /usr/X11R6/lib/X11 like /etc, don't just overwrite its contents.
>
>Why? It's a bad idea, in general, to customize installed files. Nor is
>it required.

Like '/etc/*'? Frederick, you've just sounded like Microsoft.  ;)

This is a really old installation, significantly older than NetBSD-pkg. I
set it up shortly after Allen enabled X11 support for the Quadra 700.
[hauke <at> q700] ~ # ls -lt /usr/X11R6/lib/X11/ | tail
-r--r--r--   1 root  wheel   4767 Jul  6 21:00 XKeysymDB
drwxr-xr-x   2 root  wheel    512 Jun  3 21:42 wmx
drwxr-xr-x   2 root  wheel    512 Nov 18  1999 etc
drwxr-xr-x   2 root  wheel    512 Nov 18  1999 doc
drwxr-xr-x   5 root  wheel    512 Nov 18  1999 XF86Setup
drwxr-xr-x   3 root  wheel    512 Jun  7  1997 afterstep
drwxr-xr-x   2 root  wheel    512 May 22  1997 sample.fvwmrc
drwxr-xr-x   2 root  wheel    512 May 22  1997 fvwm2
drwxr-xr-x   2 root  wheel    512 May 22  1997 fvwm
drwxr-xr-x   2 root  wheel    512 May 22  1997 chimera

A vanilla X11 app will stuff its global preferences in
'/usr/X11R6/lib/X11', since this is the traditional /etc equivalent of X11.
And so will X11 based packages unless you go with xpkgwedge.

NetBSD/mac68k has shipped the same barebones X11 setup for seven years or
so. Its xdm setup does not deal with an Xmodmap ("everyone has a US kbd"),
it does not deal with ssh keys (Xsession)... I'd have to look up what else
I have changed there. X11 application resource defaults come to mind.

>For "xdm" for example, you could copy
>/usr/X11R6/lib/X11/xdm/ to /etc/X11/xdm/, customize the files in /etc,
>and set `xdm_flags="-config /etc/X11/xdm/xdm-config"' in /etc/rc.conf.

Thanks for the pointer. I will do that.

>> # Factor out actions like transition a.out -> elf, upgrading /etc or
>> /usr/X11R6/lib/X11, and make them available in a menu.
>
>What's the issue with the a.out -> ELF transition? It should be a
>one-time thing, and be unobtrusive for systems which are ELF already.

Finer control? The ability to retry/re-do specific steps instead of doing
it all over? You're right, though, that transition is a bad example since
sysinst realizes when it has already done that and does not retry. But
there are other things like the /etc update, making devices or whatever
that you might want to factor out. Minor point, agreed.

>For "/etc", you could always decline to install the "etc.tgz" set, and
>let "etcupdate" take care of it after booting into the new system.

Sigh... That was exactly my summary: Don't trust sysinst for the upgrade,
do it manually. *If* you tell sysinst to upgrade an existing installation,
one of the  first things it does is move the existing /etc aside and
install etc.tgz which is then partly adjusted during the upgrade procedure.
And if sysinst drops the ball, you have to find out what has been done so
far and clean up after it. In my case, not even fstab was in place, which
makes it impossible to e.g. reinstall selected sets.

The etcupdate/postinstall combo is vastly superior to what sysinst offers
and incompatible with the latter. As I said, the INSTALL document of the
20020706 snapshot does not mention it at all, and postinstall for all its
beauty and power is undocumented other than by its source and folklore on
current-users.

In case I haven't made that clear enough: My issue is with sysinst, mainly.
The fact that I can boot a ramdisk kernel on mac68k and work from it is way
cool, and the performance of /etc/postupdate is the one major pleasant
surprise in this experience.

>> Overall impression: While sysinst has become quite polished in some areas,
>> its structure is still very rigid and unforgiving. Stroll from the Right
>> Path, and you're done for.
>>
>> At this point, I'd advise to ignore sysoinst and upgrade the traditional
>> way: Boot with the new kernel to single user, unpack the tarballs, upgrade
>> /etc with /etc/postinstall and /usr/sbin/etcupdate.
>
>While most of the problems you point out are valid (and they all look
>fixable), I think your overall assessment is overly harsh.

Due to these valid problems, I needed ~15 sysinst runs to get the upgrade
done. And it is not even the first time I went through this. 'Overly harsh?'

>You can get used to sysinstall's warts: If the install bombs because
>of a typo in the nfs path, for example, you can just hit ^D and
>start again -- no need to reboot.

I rebooted _once_ because fsck claimed to mark a fs clean and did not, and
sysinst did see this as an error. The other fourteen sysinst runs resulted
of simply 'starting again'. Bad enough, though, since you have to manually
clean up after sysinst (and find out the hard way _what_ you have to take
care of) before you can restart it.

>I could never get used to the old Mac OS installer

The MacOS installer is irrelevant for this discussion - I haven't even
mentioned it.

>taking 24 hours to unpack the sets on a Quadra, and what you call "the
>traditional way" is hard to support (though I do that myself on the
>one headless Quadra, it's not a very nice thing to present that
>procedure to a new user), and won't do at all for new installs.

I am not talking about new installs, either. I am talking about the sysinst
upgrade procedure, only. While new installs have been just as problematic
for me in the past, at least there is no danger of trashing existing data
(unless sysinst trashes the wrong partition, that is...).

	hauke

[Btw.: I went through this three days ago. Fifteen times, each time kicked
out with a different trivial error, cleaning up, re-starting from zero. I
can be pretty stubborn in such cases, and I decided to make sort of a
project out of it. When did _you_ (or Bob) do your last NetBSD upgrade
through sysinst? Just curious...]

--
/~\  The ASCII Ribbon Campaign       "They that can give up essential liberty
\ /    No HTML/RTF in email          to obtain a little temporary safety
 X     No Word docs in email         deserve neither liberty nor safety."
/ \  Respect for open standards                    -- Benjamin Franklin, 1759

Re: 1.6D sysinst... [long]

On Sat, 20 Jul 2002, Hauke Fath wrote:

> At 10:31 Uhr -0500 20.7.2002, Frederick Bruckman wrote:
> >On Sat, 20 Jul 2002, Hauke Fath wrote:
>
> The etcupdate/postinstall combo is vastly superior to what sysinst offers
> and incompatible with the latter. As I said, the INSTALL document of the
> 20020706 snapshot does not mention it at all, and postinstall for all its
> beauty and power is undocumented other than by its source and folklore on
> current-users.

Yes, running "etcupdate/postinstall" should be on the list of things
for "sysinstall" to do -- in lieu of moving "/etc" to "/etc.old",
agreed. The last time we were in release mode, "etcupdate" did not
exist, so it's understandable that it's not in "sysinstall" yet.

> >While most of the problems you point out are valid (and they all look
> >fixable), I think your overall assessment is overly harsh.
>
> Due to these valid problems, I needed ~15 sysinst runs to get the upgrade
> done. And it is not even the first time I went through this. 'Overly harsh?'

That's pretty bad. I never had to do it more than 4 or 5 times. 

> >I could never get used to the old Mac OS installer
>
> The MacOS installer is irrelevant for this discussion - I haven't even
> mentioned it.

But it was the supported way that "sysinstall" replaced, so it is
relevant to the question of what the recommended procedure in INSTALL*
should be (since it sounded like you were suggesting that the INSTALL*
docs be reverted).

> I am not talking about new installs, either. I am talking about the sysinst
> upgrade procedure, only. While new installs have been just as problematic
> for me in the past, at least there is no danger of trashing existing data
> (unless sysinst trashes the wrong partition, that is...).

I believe that's fixed, and I'd like to thank Bob Nestor for pushing
through and getting the fix in time for NetBSD 1.6. [That was my big
complaint with "sysinstall", and now I'm completely sold.]

> [Btw.: I went through this three days ago. Fifteen times, each time kicked
> out with a different trivial error, cleaning up, re-starting from zero. I
> can be pretty stubborn in such cases, and I decided to make sort of a
> project out of it. When did _you_ (or Bob) do your last NetBSD upgrade
> through sysinst? Just curious...]

I've done it three times in the past month or two -- to 1.5ZC, and
1.6A and 1.6D. There's a learning curve. The latest one was absolutely
painless (though as you said, DHCP doesn't fully work, it only gets
half of the information, but it still saves a little typing).

Frederick

Re: 1.6D sysinst... [long]

On Sat, 20 Jul 2002, Hauke Fath wrote:

> At 7:27 Uhr -0500 20.7.2002, Bob Nestor wrote:
>
> Early during the upgrade procedure, sysinst offered to upgrade my root ffs.
> 2002-07-06 snapshot, to be precise.
>
> >so I'm not sure why this is even an issue for an upgrade.
>
> If you accept the upgrade offer, and your root filesystem does not match
> the above 'fairly small', your only way back is to 'newfs -O' the partition
> and restore from a backup.
>
> Since this operation is (a) not reversible and can (b) trivially be done
> with a 'fsck -c' in single user mode, I propose to not offer the option in
> sysinst at that point.  IMHO, a notice would be enough.

I believe Scott Reynolds noted the same problem, with "fsck -c"
damaging a partition. I agree that "sysinstall" should simply give
notice. The last time I'd tried "fsck -c", on NetBSD 1.3?, it was a
bear, and so I'd simply given up on the old Installer, "newfs"'ing and
"restore/dump"'ing a series of new disks in the intervening period.

> >> # Message "...first part... is finished. Sysinst has written a
> >> disklabel to
> >> the target disk, ..." is bogus and annoying for mac68k.
> >
> >The message is in the MI portion of sysinst and isn't necessary totally
> >bogus.  Since the disklabel for the mac68k port is dynamically
> >constructed from the Apple Disk Partition Map which could have been
> >modified by the sysinst process, the message is mostly correct.
>
> Errm... For me at least, on mac68k there is a _big_ difference between
> updating a disklabel (which sysinst has no business with during an upgrade,
> either) and _writing_ a disklabel. Where 'disklabel', at least for me,
> denotes a BSD disklabel, which has no place on a mac68k boot disk.

I agree. It's a lie, it's alarming; you can learn to ignore it -- but
it should go away.

> I have seen sysinst trash my HFS partition before, so my trust is limited.

There was massive brain damage in the paritioning code (and still is
in netbsd-1-5, in which INSTALL notes "sysinstall" is still labeled as
experimental). See PR port-mac68k 15528, now closed.

Frederick

help

how can i make a bootable CD with ahead nero burning room with win 2k
professional