Edgar Fuß | 27 Jan 19:36 2016

Two improvements to etcupdate(8)

Should this go to tech-install?

I wrote two improvements (or so I think) to etcupdate(8) for which I attach
two separate patches. I must admit I did't test them too hard yet.

1. I never used automatic (-a) mode because I never used it before.
The problem is that unless you used -a last time, you lack the reference 
checksums in /var/etcupdate.
So I added a ``bootstrap'' mode (-A) that only builds the checksums from its 
-s arguments and doesn't do anything else.
The idea is that you first etcupdate -A -s /path/to/old/etc.tgz and then 
etcupdate -a -s /path/to/new/etc.tgz afterwards.

2. I regularily update a whole set of machines. I usually did this by 
running etcupdate on a development machine, noted the per-file decisions 
I made (on paper) and then used that list to know what to type to further 
etcupdate invocations on the production machines.
I think computers are better at recording and playback than humans, so I added 
two switches, -W and -R, that record the (hand-made) keep/overwrite/merge 
decisions to a file resp. use that file to follow the same decisions again.
The file doesn't record the details of a merge operations, only the fact,
but manually re-merging three files is an improvement over typing d/i/m 
for a hundred files. Or so I think.

Comments/suggestions/improvements (and better names than ``bootstrap'', 
``read'' and ``write'') welcome.
--- etcupdate	2016-01-27 11:26:42.000000000 +0100
+++ etcupdate-A	2016-01-27 16:12:36.000000000 +0100
(Continue reading)

David Young | 26 Jan 21:06 2016

monitoring cache performance of user programs?

I'm wondering if anybody is using pmc(1) to monitor cache performance of
user programs on modern x86?  Maybe there is a better way to do this in
NetBSD, today, such as a facility provided by DTrace?

Ultimately I would like to have a performance profile of some of my
programs that tells not only how long is spent in each line, but roughly
how the time was spent, so that I can make better decisions about
whether/how to optimize the code.  Even if time is ascribed to causes
(cache misses, page faults) in a very approximate way, I think that it
would be more illuminating than simulating the code in my head.



David Young
dyoung <at> pobox.com    Urbana, IL    (217) 721-9981

Christos Zoulas | 22 Jan 20:00 2016

-D_NETBSD_SOURCE makes headers uncompilable


A few of the traditional BSD origin headers, use u_{char,int,long} which
are not defined in <sys/types.h> if _NETBSD_SOURCE is defined. There are
four choices:

1. Leave things alone; it's been broken forever and seems harmless. Autoconf
2. Fix all the headers to use the full types; intrusive will introduce diffs
   with other Unixes and upstreams.
3. Provide local definitions of the needed types on each header. Messy, error
4. Put the definition of those types in a separate header (<sys/utypes.h>)
   and include this header from other headers that require it. This is what
   I've chosen to implement in this patch and I'd like to commit.
5. ???



Index: include/bm.h
RCS file: /cvsroot/src/include/bm.h,v
retrieving revision 1.5
diff -u -u -r1.5 bm.h
--- include/bm.h	3 Feb 2005 04:39:32 -0000	1.5
+++ include/bm.h	22 Jan 2016 18:53:53 -0000
 <at>  <at>  -37,6 +37,8  <at>  <at> 
 #ifndef _BM_H_
(Continue reading)

Thor Lancelot Simon | 18 Jan 21:09 2016

stty erase WTF

What's been done to terminal handling in NetBSD 7.0?

I now see the following, grossly broken behavior:

1) On my ssh client, stty shows erase is ^?

2) I ssh to a NetBSD-7 machine; stty shows erase is ^?

3) I "su -" or "sudo -i".  stty shows erase is now ^H

The culprit appears to be these lines in /root/.profile:

	if [ -x /usr/bin/tset ]; then
        	eval $(tset -sQrm 'unknown:?unknown')

But whatever the reason is, this needs to be fixed.  My ssh session
explicitly propagated erase=^? to the netbsd-7 system, and nothing
we ship in the system defaults has any business overriding that.


  Thor Lancelot Simon	                                     tls <at> panix.com

  "We cannot usually in social life pursue a single value or a single moral
   aim, untroubled by the need to compromise with others."      - H.L.A. Hart

Greg Troxel | 18 Jan 19:20 2016

non-POSIX behavior from wc(1)

On NetBSD 6, one gets wc output with pleasing spaces

$ wc -l unison.log cmake.diff
       1 unison.log
      58 cmake.diff
      59 total
$ wc unison.log cmake.diff
       1       7      45 unison.log
      58     168    2461 cmake.diff
      59     175    2506 total

But POSIX says:

   "%d %d %d %s\n", <newlines>, <words>, <bytes>, <file>

which does not have extra spaces.  This only matters if you write code
to parse the output based on the spec:


So it seems we should adjust wc(1) to note the extra spaces, since it
seems unlikely that we would come to consensus on fixing it.

James K. Lowden | 14 Jan 00:03 2016

asn1parse & man warts

Is asn1parse(1) supposed to be in 7.0?  It's mentioned under the wrong
name in apropos(1), but is not included in any set.  

$ for F in *.tgz; \
do pax -zf $F | grep asn1parse; done 

$ man -k -l asn 
RSA_sign_ASN1_OCTET_STRING(3) - RSA signatures
ASN1_generate_nconf(3) - ASN1 generation functions
ASN1_STRING_new(3) - ASN1_STRING allocation functions
ASN1_STRING_dup(3) - ASN1_STRING utility functions
ASN1_OBJECT_new(3) - allocation functions
ASN1_STRING_print_ex(3) - ASN1_STRING output routines.
asn1parse(1) - ASN.1 parsing tool

$ man -w asn1parse
man: no entry for asn1parse in the manual.

$ man -w openssl_asn1parse

In passing I'd like to mention a couple of minor issues: 

1.  The synopsis for "man -k" in the manual is incorrect.  Current
behavior is to drop "-k" from argv and exec apropos.  That means all
other flags are passed to apropos, and 

(Continue reading)

Christos Zoulas | 9 Jan 04:07 2016

regsub() proposal

I would like to have sed -e 's/foo\(bar\)foo/\1/'. Most interpretive
languages have easy access to this, but there is no c function that
does this. I propose:

regsub(char *buf, size_t len, const char *sub, const regmatch_t *rm,
    const char *str);

aregsub(char **buf, const char *sub, const regmatch_t *rm, const char *str);

regsub() is like snprintf() where aregsub() is like asprintf().

The rest of the arguments are:

	sub = the right hand side of the sed substitution containing
		\<n> or & escapes.
	rm = the regexec() match list. It should be 10 elements long.
	str = the input string to be used

They return either the number of characters for the full conversion,
or -1 for error.

Here's the implementation, together with a main program...


/*	$NetBSD$	*/

(Continue reading)

Thomas Klausner | 9 Jan 00:58 2016

UTF-8 capable fmt(1)

I still use fmt(1), and when I do, I often note that it breaks
non-ASCII chars.

NetBSD contains a fmt(1) that's still from 4.xBSD days (IIUC) and has
basically no comments on what it does nor why.

1997 Gareth McCaughan rewrote it completely; that version is used in
OpenBSD and FreeBSD. Both versions have wide character support
nowadays, but it looks like the FreeBSD version is based on the
OpenBSD one from long ago.

I've replaced the fmt sources and man page in my local tree with the
version from OpenBSD and only had to ifdef out two calls to pledge()
to make it compile without warnings. And it does indeed support wide

I suggest we follow the lead of other and replace our 30 year old fmt
with a 15 year old one :)


Simon J Gerraty | 8 Jan 17:27 2016

sh: Cannot vfork

I get a lot of email...

I've been having increasing problems with mail delivery due to:

Dec 17 11:41:07 chaos postfix/local[28962]: 0396A580A9:
to=<sjg <at> chaos.jnpr.net>, relay=local, delay=396, delays=396/0.02/0/0.01,
dsn=5.3.0, status=bounced (Command died with status 2:
"/usr/pkg/bin/procmail || exit 75 #sjg". Command output: sh: Cannot
vfork )

I'm not sort of ram,swap, or disk
even when I see fork errors on console, top shows <= 200 processes running
similar to the output below.

Last night I was unable to login to this box - ssh failed to get a

I have increased the max number of processes that can run both globally
(3k) and per user (1k)

I have tweaked /etc/postfix/master.cf to limit the number of instances
of local (21) and smtp (42)

I even upgraded the OS from  5 stable (otherwise very solid) to
NetBSD chaos 6.1.5 NetBSD 6.1.5 (GENERIC) amd64
but the problem continues.

$ ulimit -a
time(cpu-seconds)    unlimited
file(blocks)         unlimited
(Continue reading)

Carsten Kunze | 7 Jan 17:07 2016

S_IFMT not mentioned in any man page


constant S_IFMT (and related constants) is not mentioned in any man page (except dev_mkdb).  Shouldn't it
be mentioned in stat(2)?  Of course one can look in sys/stat.h, but this file is source code and not a specification.


Martin Husemann | 28 Dec 13:16 2015

Per thread locales

New firefox uses a newer version of skia that uses a  helper class
"GrAutoLocaleSetter" to temporarily switch the locale to "C" while parsing
GL shaders.

IIUC Jörg strongly prefers to fix all locale sensitiv calls to use variants
explicitly passing the locale (which is most simple for "C").

However, this is not always trivial (when the code uses lots of deeply nested
library code) so patches are hard to verify and even harder to upstream.

I would argue that besides very special situations the usage sketched above
is the only valid reason to temporarily switch locales at runtime, and it may
make sense to add special case support for it.

A relatively simple & cheap hack would be a per-thread flag "override process
locale with C" and a uselocale() function that recognizes "C" and the
process locale and then sets/clears the flag accordingly, failing in all
other cases.