headers
Poul-Henning Kamp | 3 Jan 2005 11:47
Picon
Favicon

making nmdm(4) emulate actual speed.


I participated in an "Editor Celebrity Death Match" recently and
being the senior combatant my weapon of choice was ed(1).  To
properly show off ed(1)'s main weakness I wanted to run my slides
in ed(1) on a 300 bps line.

Rather than use two USB-serial dongles and a usb-hub, I hacked nmdm(4)
up to actually respect the baud-rate set with stty.

Would this be considered generally useful ?

--

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk <at> FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 6 comments |
headers
Luigi Rizzo | 3 Jan 2005 15:57

Re: making nmdm(4) emulate actual speed.

On Mon, Jan 03, 2005 at 11:47:39AM +0100, Poul-Henning Kamp wrote:
> 
> I participated in an "Editor Celebrity Death Match" recently and
> being the senior combatant my weapon of choice was ed(1).  To
> properly show off ed(1)'s main weakness I wanted to run my slides
> in ed(1) on a 300 bps line.
> 
> Rather than use two USB-serial dongles and a usb-hub, I hacked nmdm(4)
> up to actually respect the baud-rate set with stty.
> 
> Would this be considered generally useful ?

being nmdm(4) an emulation tool, i'd say definitely yes, probably
even more useful if you provide a knob to enable/disable the speed
emulation -- i see a point in actually emulating the wire speed,
but also one in not doing so when the application is not speed-sensitive
and you just want it to run quickly.

cheers
luigi

> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk <at> FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
> _______________________________________________
> freebsd-arch <at> freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-arch
> To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
_______________________________________________
(Continue reading)

Permalink | Reply | 3 comments |
headers
Julian Elischer | 3 Jan 2005 19:51

Re: making nmdm(4) emulate actual speed.


Poul-Henning Kamp wrote:

>I participated in an "Editor Celebrity Death Match" recently and
>being the senior combatant my weapon of choice was ed(1).  To
>properly show off ed(1)'s main weakness I wanted to run my slides
>in ed(1) on a 300 bps line.
>
>Rather than use two USB-serial dongles and a usb-hub, I hacked nmdm(4)
>up to actually respect the baud-rate set with stty.
>
>Would this be considered generally useful ?
>  
>

maybe as an option...

I want faster tansfer speed when I use it.. but emulated speed could be 
fun..
and even useful as a way of pacing data in some cases..

_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 0 comments |
headers
M. Warner Losh | 3 Jan 2005 22:00

Re: making nmdm(4) emulate actual speed.

In message: <18962.1104749259 <at> critter.freebsd.dk>
            Poul-Henning Kamp <phk <at> phk.freebsd.dk> writes:
: Rather than use two USB-serial dongles and a usb-hub, I hacked nmdm(4)
: up to actually respect the baud-rate set with stty.
: 
: Would this be considered generally useful ?

Yes.  It would be useful, but only as an option....

Warner
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 0 comments |
headers
Youlin Feng | 3 Jan 2005 23:23

User process starvation in FreeBSD 5.3

Hello all,

We are building a network appliance running FreeBSD 5.3 and under very
heavy network traffic the user processes don't get scheduled for an
unacceptable period of time. Marking the user process/thread real-time
class doesn't help since the real-time user threads priorities are still
lower than the interrupt threads. BTW, in our case, the CPU spends very
long time at PI_NET (16) priority level, instead of at the (PI_SOFT + 4)
soft intr level due to the packet forwarding nature. Either way, the
user processes don't have a chance. In the following discussion I'll use
PI_NET to represent the network interrupt threads priority.

I am experimenting with improving the real-time threads scheduling
performance and would like to hear from you.

Here is what I am doing:

1.	Raise the minimum real-time user threads priority to a value
higher than PI_NET, e.g. 

#define PRI_MIN_REALTIME            0

And use the rtprio() syscall or command to set the priority to be higher
than PI_NET, e.g. "rtprio 10 <my_proc_id>"

This didn't turn out to be enough, since the real-time user process
still uses system services or drivers that run at a lower priority than
PI_NET, e.g. disk, tty, etc.

2.	Change the PI_NET value to be the highest of all interrupt
(Continue reading)

Permalink | Reply | 1 comment |
headers
Stefan Bethke | 4 Jan 2005 02:05
Picon

Re: User process starvation in FreeBSD 5.3

Am 03.01.2005 um 23:23 schrieb Youlin Feng:

> We are building a network appliance running FreeBSD 5.3 and under very
> heavy network traffic the user processes don't get scheduled for an
> unacceptable period of time. Marking the user process/thread real-time
> class doesn't help since the real-time user threads priorities are 
> still
> lower than the interrupt threads.

The effect you're describing very much sounds like 'livelock': the 
system is so overwhelmed with interrupts that it has no time to do 
anything else but servicing them. FreeBSD offers polling(4), which is 
intended to mitigate the overhead of a high interrupt rate on certain 
network controllers; especially in high-throughput scenarios, it can 
improve system load, throughput, and latency quite dramatically.

On the other hand, your hardware might just not be able handle the 
packet rate.

I'd suggest asking this on freebsd-net, where I believe quite a few 
people with experience in high-throughput setups are reading.

Cheers,
Stefan

--

-- 
Stefan Bethke <stb <at> lassitu.de>   Fon +49 170 346 0140

_______________________________________________
freebsd-arch <at> freebsd.org mailing list
(Continue reading)

Permalink | Reply | 0 comments |
headers
Pawel Jakub Dawidek | 4 Jan 2005 23:40
Picon
Favicon

BigDisk project: du(1) 64bit clean.

Hi.

I want you to look at two patches which makes du(1) 64bit clean.
This work is part of the BigDisk project:

	http://www.freebsd.org/projects/bigdisk/

The main problem here is that du(1) uses fts(3) and fts_number field from
one of its structures to store size.
This field is defined as 'long' so it doesn't give us what we want
(on 32bit archs).

So first of all, we need this patch:

	http://people.freebsd.org/~pjd/patches/fts.h.patch

It adds 64bit fts_bignum field, but because it is hiden under union,
it doesn't break ABI. It also doesn't break API, thanks to #defines.

Patch for du(1) is here:

	http://people.freebsd.org/~pjd/patches/du-64bit-clean.patch

We should decide if we want fts.h.patch also in HEAD or only in RELENG_5,
because we can make it much cleaner in HEAD, but we will break ABI
(API should be quite ok) - we need to change size of fts_number field.

--

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd <at> FreeBSD.org                           http://www.FreeBSD.org
(Continue reading)

Permalink | Reply | 15 comments |
headers
Julian Elischer | 5 Jan 2005 00:47

Re: BigDisk project: du(1) 64bit clean.


Pawel Jakub Dawidek wrote:

>Hi.
>
>I want you to look at two patches which makes du(1) 64bit clean.
>This work is part of the BigDisk project:
>
>	http://www.freebsd.org/projects/bigdisk/
>  
>
One thing that needs to be done is an 2ndary storage fsck.
 that doesn't try put everything in RAM.
Basically this will mean extracting all the metadata from filesystems into
files and running sort operations of various kinds on them
to order the data in ways that allows consistencies to be checked.
It will take a bit longer than a RAM fsck but maybe not as much as
one might fear.
We all remember those "sort a mag-tape larger than RAM"
lessons from CS101 don't we?
At least it doesn't have to be "in place" so merge sorts are OK. :-)

why?

A bitmap of 1TB of 512 byte records is 244MB so with a 4BG machine
with 3GB available to the process you can't even fit the bitmaps into
memory for a 12TB Filesystem let alone other metadata.

Going to 2048 byte frags helps but you still run into a limit.
last I tried it, you need about 600MB per TB of fileysstem to check.
(Continue reading)

Permalink | Reply | 6 comments |
headers
Doug White | 5 Jan 2005 03:58

Re: BigDisk project: du(1) 64bit clean.

On Tue, 4 Jan 2005, Pawel Jakub Dawidek wrote:

> I want you to look at two patches which makes du(1) 64bit clean.
> This work is part of the BigDisk project:
>
> 	http://www.freebsd.org/projects/bigdisk/
>
> The main problem here is that du(1) uses fts(3) and fts_number field from
> one of its structures to store size.
> This field is defined as 'long' so it doesn't give us what we want
> (on 32bit archs).

No offense intended, but can we avoid introducing LP64 bugs, please?
Particularly when the goal is "ABI compatibility."*

dwlab3,ttyp1,~,24>uname -m
i386
dwlab3,ttyp1,~,25>./test
sizeof(long) [4] + sizeof(void *) [4] == 8 == sizeof(int64_t) [8]

ok .. but:

dwlab4,ttyp1,~,20>uname -m
amd64
dwlab4,ttyp1,~,21>./test
sizeof(long) [8] + sizeof(void *) [8] == 16 != sizeof(int64_t) [8]

oops! The struct just grew by 8 bytes!

(*) On the same platform, obviously.
(Continue reading)

Permalink | Reply | 3 comments |
headers
Brooks Davis | 5 Jan 2005 04:04

Re: BigDisk project: du(1) 64bit clean.

On Tue, Jan 04, 2005 at 11:40:43PM +0100, Pawel Jakub Dawidek wrote:
> Hi.
> 
> I want you to look at two patches which makes du(1) 64bit clean.
> This work is part of the BigDisk project:
> 
> 	http://www.freebsd.org/projects/bigdisk/
> 
> The main problem here is that du(1) uses fts(3) and fts_number field from
> one of its structures to store size.
> This field is defined as 'long' so it doesn't give us what we want
> (on 32bit archs).
> 
> So first of all, we need this patch:
> 
> 	http://people.freebsd.org/~pjd/patches/fts.h.patch
> 
> It adds 64bit fts_bignum field, but because it is hiden under union,
> it doesn't break ABI. It also doesn't break API, thanks to #defines.
> 
> Patch for du(1) is here:
> 
> 	http://people.freebsd.org/~pjd/patches/du-64bit-clean.patch
> 
> We should decide if we want fts.h.patch also in HEAD or only in RELENG_5,
> because we can make it much cleaner in HEAD, but we will break ABI
> (API should be quite ok) - we need to change size of fts_number field.

I'd be inclined to use the somewhat gross fix in PR 74567 in RELENG_5
and do it right in HEAD.  bde suggested changing fts_num to intmax_t.
(Continue reading)

Permalink | Reply | 2 comments |

Gmane