Charles Cui | 1 Jul 09:32 2016


Hi Christos,

I have spend several days of thinking how to design a unit test to prove
the usefulness of
the priority protect patch and here are some updates of the new unit tests.
After investigating several ways of designing the tests, I realized that if
can prove low priority thread runs first after raising priority due to
mutex blocking,
then it indicates scheduler takes the new priority into considerations.

Here are some steps,
1. main thread sets itself to be a realtime task and launched two tasks,
one has higher priority and
the other has lower priority.
2. each child thread(low and high priority thread) sets its scheduler and
3. each child thread did several rounds of computation, after each round it
sleep 1 second.
4. the child thread with low priority will call _sched_protect to increase
its protect priority.
5. We verify the thread with low priority runs first.

Why does it work?
For main thread, we launched thread with high priority first, this actually
gives this thread
some benefits of starting first. And if the thread with low priority did
not call _sched_protect,
the high priority thread should finish task first. After each round of
computation, we call
(Continue reading)

coypu | 24 Jun 15:34 2016

Useless programs in base.tgz - bpm, ekermit...


There's a lot of programs sticking around since a long time, and are
less useful today. at the very least, they do not need to be in
base.tgz, which means they take up space even in minimal installs.

ekermit in rescue, base.tgz
bpm in base.tgz
atf tests utilities in base.tgz

I want to:
delete ekermit, bpm (or at the very least move them to another set).
move atf things (and rump?) out of base.tgz to a more optional set.

Edgar Fuß | 21 Jun 20:46 2016

Shell Command Substitution and fork()

I have a shell script that makes heavy use of Command Substitution, i.e.
The script takes several seconds to execute, mainly because Command 
Substitution takes place in a Subshell Environment and that usually means 
a fork().
However, the OS X^W^WmacOS ksh only takes tens of milliseconds for the same 
job, as does shells/ast-ksh from pkgsrc.
Unsurprisingly, the latter (I'm unable test for the former due to lack of 
ktrace or the like) doesn't fork.

Does anyone know how those ksh's achieve that? Are there any drawbacks? 
Could ash be taught the same thing?

Or can someone think of a POSIX-compliant way to put the output of a command 
(think printf) into a variable without Command Substitution? Unfortunately
	printf ... | read x
doesn't do the trick because POSIX allows (and most shells indeed do) 
execution of the tail-of-pipe in a Subshell Environment.

Charles Cui | 17 Jun 19:45 2016


Yeah, that makes sense. I will see how to construct such a unit test(or
call it benchmark)

Thanks, Charles.

2016-06-17 10:44 GMT-07:00 Christos Zoulas <christos <at>>:

> On Jun 16,  9:28pm, charles.cui1984 <at> (Charles Cui) wrote:
> -- Subject: Re: _SC_SIGQUEUE_MAX
> | Hi Christos,
> |
> |     I have completed some unit tests for ad's work and run these unit
> tests
> | on two versions (my version and your version).
> | Your initial thinking is correct. There are bugs when getting and setting
> | different fields and these bugs can be
> | verified via my unit tests (it does not pass in my version, but pass in
> | your version).
> | Here are these patches.
> |
> |
> Thanks; these look great but only test the basic functionality of
> getting and setting fields. They don't test any of the actual
> functionality of the feature being added. What I would like you to
> do is to think how to construct test cases which involve multiple
> threads that create the conditions for the priority inheritance
(Continue reading)

Abhinav Upadhyay | 12 Jun 01:20 2016

Learning words from man pages

Hi All,

I came across an interesting paper from Google on machine learning[1],
where they came up with an efficient representation for words from a
corpus. These representations are called word embeddings in general,
and they have titled their method as word2vec.

It is a two layer neural network which given a corpus as input,
produces a set of word vectors as its output. These vectors represent
each word in the corpus in a vector space, where words with similar
semantics lie nearer to each other in that space.

There are two methods of training the data:
1. Bag of words: here the ordering of the words in the corpus is not
considered. It can be thought of like, given a word, what are the
other words similar to this.
2. Skip grams: It considers the ordering of the words, it can be
thought of like, if given word w1, what is the probability of word w2
appearing next.

They have shown interesting implications of this, for example,
"France" and "Italy" are closer to each other in the model that they
trained. Another interesting observation is the application of vector
algebra here, for example they show that:

vector(king) - vector(man) + vector(woman) = vector(queen).

This technique is becoming widely popular and has applications in
areas like search, question answering, summarization. I've trained
this on our man page corpus data (plus some man pages from pkgsrc) and
(Continue reading)

Christos Zoulas | 11 Jun 01:29 2016


On Jun 10,  2:37pm, charles.cui1984 <at> (Charles Cui) wrote:
-- Subject: Re: _SC_SIGQUEUE_MAX

| Thanks, Christos, I will read FreeBSD related code carefully.

In the meantime also:

- there are no man pages or tests for PRIOPROTECT_AND_GETCLOCK
- there are no man pages or tests for PTHREAD (barrierattr calls)
- how did the SIGPOLL patch work?

I committed:



Paul Goyette | 10 Jun 08:18 2016

Possible enhancement to find(1)

I often find myself (no pun intended!) wanting to determine which files 
were accessed before/since a particular timestamp.  There currently 
exist the -{a,c}{newer,min,time} primitives, but none of these allow me 
to say "since today at 5 AM".

I'd like to suggest adding new primitives

 	-asince "timestamp"
 	-csince "timestamp"

where the timestamp argument would be converted to an actual time using 

Does anyone else think this is a good idea?  Is there anyone who would 
object to adding these primitives?

| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:      |
| (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at   |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at |

Charles Cui | 10 Jun 02:35 2016


Hi guys,

   I am trying to add the support of SIGQUEUE_MAX, which is the maximal
number of real time signals to the target process. There are several ways
send a signal to a process like, kill, raise, etc. I am wondering whether
all real
time signals should be only sent via sigqueue? (in which case, establishing
a queue in the sigqueue code path and manage the queue should be enough to
support _SC_SIGQUEUE_MAX).

Thanks, Charles
Brooks Davis | 8 Jun 15:19 2016

typo fix for vis.3

Kurt Jaeger submitted the following patch to vis.3 which we import into
FreeBSD from NetBSD.

-- Brooks

--- contrib/libc-vis/vis.3	2016-06-04 06:46:24.234726000 +0200
+++ contrib/libc-vis/vis.3	2016-06-04 06:47:13.515516000 +0200
 <at>  <at>  -513,7 +513,7  <at>  <at>  
 .Nx 6.0
 .Fx 9.2 .
-Myltibyte character support was added in
+Multibyte character support was added in
 .Nx 7.0
 .Fx 9.2 .

Charles Cui | 8 Jun 04:33 2016


Hi Christos,

   I have PTHREAD_STACK_MIN supported and attached
patch for your review. My next step is to focus on the remaining
Seems we need much more significant changes in this part.

Thanks, Charles
Attachment (0003-add-PTHREAD_STACK_MIN.patch): application/octet-stream, 2873 bytes
James K. Lowden | 7 Jun 03:35 2016

TRE regex

Back in 2009, Matthias-Christian Ott ported Ville Laurikari's regex
library, apparently with the intention of replacing the one in base,
originally from Henry Spencer.

What happened?  Other than the announcement, I find no discussion about
it.  I see it's in pkgsrc, all well and good, but why was  the project
was undertaken and the work not brought into base?  

In case you are feeling complacent about NetBSD's regex, the awk
documentation relies on it, and falls short.  Awk claims to implement
regex per egrep(1) -- providing no further description -- but that's
just docurot: 

	$ echo aaa | egrep 'a{3}' | wc -l
	$ echo aaa | awk '/a{3}/' | wc -l

As far as I know, we have 3 regex definitions in base: GNU grep, NetBSD
sed (with regex(3), defined by re_format(7)), and NetBSD awk.  It would
be an improvement IMO to use one implementation for all utilities in
base, to make them internally consistent and dependable (and
reproducible), even at the expense of compatibilitly with GNU's


(Continue reading)