Re: humanize_number(3) for dd(1) summary?

On 04.12.2010 19:05, Jean-Yves Migeon wrote:
> On 04.12.2010 17:54, Matt Thomas wrote:
>> I use dd to test performance and 2 digits of data is not neough.
>> I actually like the current output.  I also support the idea if
>> you want humanized numbers, a conv= option would be best.
>
> I'll do as Christos suggested: leave the output as is, but implement a
> msgfmt parameter where you can either:
> - pass options:
> 	'msgfmt=human' (human readable output)
> 	'msgfmt=quiet' (silent mode -- if you don't care about output)
> - pass a string format, similar to date(1) with strftime(3):
> 	msgfmt="%i records in\n%B bytes\nfoo %bar baz\n"
>
> Default output will remain the same. Those that want to use dd(1) for
> specific stuff like benchmarks or testing could use the msgfmt string to
> tailor output to their liking.

Wow, I forgot I wrote this a long time ago and kept it sleeping in part 
of my tree *sigh*

The attached patch extends dd(1) in a way that permits specifying either:
- a msgfmt, with "human", "quiet", "posix" currently in place
- an arbitrary format, when passed as msgfmt=... to command line by user.

Of course the default behavior of dd(1) remains POSIX. It's only when 
the user passes msgfmt as argument that dd(1) will modify its summary() 
message.

Examples:

Re: humanize_number(3) for dd(1) summary?

> The attached patch extends dd(1) in a way that permits specifying either:
> - a msgfmt, with "human", "quiet", "posix" currently in place
> - an arbitrary format, when passed as msgfmt=... to command line by user.

It's better to have #ifdef option which disables the new feature
(as current #ifdef NO_CONV via SMALLPROG in Makefile)
for crunched binaries on poor install media.

---
Izumi Tsutsui

Re: humanize_number(3) for dd(1) summary?

On 02.11.2011 15:39, Izumi Tsutsui wrote:
>> The attached patch extends dd(1) in a way that permits specifying either:
>> - a msgfmt, with "human", "quiet", "posix" currently in place
>> - an arbitrary format, when passed as msgfmt=... to command line by user.
>
> It's better to have #ifdef option which disables the new feature
> (as current #ifdef NO_CONV via SMALLPROG in Makefile)
> for crunched binaries on poor install media.

Done. The option is NO_MSGFMT.

Total binary size:

new dd: 29920
new dd, with SMALLPROG: 24864
old dd: 26677
old dd, with SMALLPROG: 22028

The increase is therefore ~2.5kiB with the modifications when compared 
with the old binary.

--

-- 
Jean-Yves Migeon
jeanyves.migeon <at> free.fr

? .gdbinit
? dd
? dd.cat1
? dd.html1

Parallel tasking library

Hi, would there be any benefit in having some form of parallel tasking
library (something like Intel TBB, but simpler and developed in C) in
NetBSD?

The idea is to parallelize userlevel applications, e.g. data
compression like gzip, bzip, maybe some others...

I'm not volunteering to rewrite those applications, but I've been
working on such tasking library, and my initial benchmarks seem to
indicate parallel for loops perform up to twice as fast, compared to
Intel TBB. It does need more benchmarking and testing.

I can provide code + documentation if there is interest to have it
integrated into NetBSD.

Re: humanize_number(3) for dd(1) summary?

> Done. The option is NO_MSGFMT.
> 
> Total binary size:
> 
> new dd: 29920
> new dd, with SMALLPROG: 24864
> old dd: 26677
> old dd, with SMALLPROG: 22028
 :
>  <at>  <at>  -105,6 +108,7  <at>  <at>  void
>  jcl(char **argv)
>  {
>  	struct arg *ap, tmp;
> +	//struct dd_msgfmt *fmt;
>  	char *oper, *arg;
>  
>  	in.dbsz = out.dbsz = 512;

Leftover?

> --- misc.c	5 Oct 2007 07:23:09 -0000	1.21
> +++ misc.c	2 Nov 2011 22:21:55 -0000
 :
> +static void
> +posix_summary(void)
> +{
> +	char buf[100];
> +
>  	/* Use snprintf(3) so that we don't reenter stdio(3). */
> -	(void)snprintf(buf, sizeof(buf),

Re: humanize_number(3) for dd(1) summary?

On 03.11.2011 14:55, Izumi Tsutsui wrote:
>>  <at>  <at>  -105,6 +108,7  <at>  <at>  void
>>   jcl(char **argv)
>>   {
>>   	struct arg *ap, tmp;
>> +	//struct dd_msgfmt *fmt;
>>   	char *oper, *arg;
>>
>>   	in.dbsz = out.dbsz = 512;
>
> Leftover?

Yep, thanks

>> --- misc.c	5 Oct 2007 07:23:09 -0000	1.21
>> +++ misc.c	2 Nov 2011 22:21:55 -0000
>   :
>> +static void
>> +posix_summary(void)
>> +{
>> +	char buf[100];
>> +
>>   	/* Use snprintf(3) so that we don't reenter stdio(3). */
>> -	(void)snprintf(buf, sizeof(buf),
>> -	    "%llu+%llu records in\n%llu+%llu records out\n",
>> -	    (unsigned long long)st.in_full,  (unsigned long long)st.in_part,
>> -	    (unsigned long long)st.out_full, (unsigned long long)st.out_part);
>> +	(void)dd_snprintf(buf, sizeof(buf),
>> +	    "%I+%i records in\n%O+%o records out\n");
>>   	(void)write(STDERR_FILENO, buf, strlen(buf));

Re: Parallel tasking library

On Thu, Nov 03, 2011 at 12:48:21PM +0000, Sad Clouds wrote:
> Hi, would there be any benefit in having some form of parallel tasking
> library (something like Intel TBB, but simpler and developed in C) in
> NetBSD?

A few years ago I remember murmurs about Apple's Grand Central Dispatch,
which was imported into FreeBSD.  Does that do what you are looking for?
It would be nice to have that in NetBSD.

Dave

--

-- 
David Young             OJC Technologies is now Pixo
dyoung <at> pixotech.com     Urbana, IL   (217) 344-0444 x24

Re: Parallel tasking library

On Thu, Nov 03, 2011 at 01:10:38PM -0500, David Young wrote:
> A few years ago I remember murmurs about Apple's Grand Central Dispatch,
> which was imported into FreeBSD.  Does that do what you are looking for?
> It would be nice to have that in NetBSD.

What would use it in base?

Martin

Re: Parallel tasking library

On Thu, 3 Nov 2011 20:34:39 +0100
Martin Husemann <martin <at> duskware.de> wrote:

> On Thu, Nov 03, 2011 at 01:10:38PM -0500, David Young wrote:
> > A few years ago I remember murmurs about Apple's Grand Central
> > Dispatch, which was imported into FreeBSD.  Does that do what you
> > are looking for? It would be nice to have that in NetBSD.
> 
> What would use it in base?
> 
> Martin

Libraries that do compression/decompression, encryption/decryption,
email/web servers in base. For example if C compiler could be modified
to execute functions in parallel when building a project (this is quite
different from parallel Make), then waiting time on SMP hardware would
be drastically cut down.

It's just a case of looking for potential applications and see if
certain sections can be parallelized.

Re: Parallel tasking library

On Thu, 3 Nov 2011 13:10:38 -0500
David Young <dyoung <at> pobox.com> wrote:

> On Thu, Nov 03, 2011 at 12:48:21PM +0000, Sad Clouds wrote:
> > Hi, would there be any benefit in having some form of parallel
> > tasking library (something like Intel TBB, but simpler and
> > developed in C) in NetBSD?
> 
> A few years ago I remember murmurs about Apple's Grand Central
> Dispatch, which was imported into FreeBSD.  Does that do what you are
> looking for? It would be nice to have that in NetBSD.

It's not very portable, i.e. I don't think you can use it with plain C
compiler.