headers
Dieter Baron | 1 Mar 2007 09:13
Picon

Re: CVS commit: src/sys/dev/usb

hi,

> >   Please note that fs/unicode.h does not handle UTF-16 surrogates
> > correctly.  What's worth, the API does not allow this to be fixed.
> > 
> >   (Unicode defines more characters than fit in a 16 bit int.  In
> > UTF-16, a character with a code above 0xffff is represented as two
> > surrogate values.  In UTF-8, it is encoded as a 5 byte sequence.
> > Encoding/decoding one 16 bit value at a time does not allow for this
> > conversion to be done correctly.)
> 
> Please feel free to suggest ways that this should be fixed. Patches are 
> best!
> 
> We all would like better unicode handling, and AFAIK no one is wedded to 
> the existing interface.

  Okay, here is what I currently use in the HFS+ implementation
(netbsd-soc.cvs.sf.net:/cvsroot/netbsd-soc hfs/hfsp/unicode.[ch]):

#define UNICODE_DECOMPOSE    0x01  /* convert to decomposed NF */
#define UNICODE_PRECOMPOSE   0x02  /* convert to precomposed NF */

size_t utf8_to_utf16(uint16_t *out, size_t outlen,
		     const char *in, size_t inlen,
		     int flags, int *errcountp);

  Converts the UTF-8 string IN to UTF-16 and stores at most OUTLEN
words in OUT.  FLAGS may be one of the above to convert the string to
normal form during conversion.  if ERRCOUNTP is non-NULL, the number
(Continue reading)

Permalink | Reply | 6 comments |
headers
Stephen Borrill | 1 Mar 2007 10:22
Picon
Favicon

Re: Missile launchers vs uhid

On Wed, 28 Feb 2007, Matthias Drochner wrote:
> salo <at> Xtrmntr.org said:
>> iow, "Ten X Missile Launcher is not a HID device."
>
> It claims to be one (albeit a pretty generic one according
> to the descriptor dumps in Stephen's mail), and there is
> no indication that there is some specific bug which suggests
> not treat it as one.

Agreed.

> I guess the OpenBSD people just wanted to keep the uhid
> driver out, to make the Linux stuff work out of the box.

Agreed again, while this will work, it seems lazy. After all, it's HID 
under Windows. OTOH, I like the flexibility of Linux's idea of telling 
uhid to go away and leave us to talk directly. It gives more driver 
compatibility and doesn't require kernel changes or quirks.

> I'd assume that it would just work to send the 8/64 byte
> datablocks defined in the Python script to the uhid device
> through USB_SET_REPORT ioctls.

I think I'm missing something here. I've tried writing the data using 
USB_SET_REPORT (both UHID_OUTPUT_REPORT and UHID_FEATURE_REPORT) by 
putting the data in ucr_data[], but it hangs for a while and then gives an 
I/O error. Similarly, using hid_set_data() gives similar results.

--

-- 
Stephen
(Continue reading)

Permalink | Reply | 0 comments |
headers
Matthew Mondor | 1 Mar 2007 11:22

M:N and blocking ops without SA, AIO

From my understanding, an M:N threading model which uses a pool of
kernel LWPs without using Scheduler Activations would need to be able to
poll asynchroneously in its userland library scheduler for any
potentially blocking syscalls.

Threads which wouldn't invoke calls internally yielding them back to the
user-space scheduler could be mapped to an LWP so that the kernel may
preempt them efficiently, but a form of preemption would also be
required in the user-space scheduler to detect this event and assign
them to an LWP, potentially with minimal kernel help to efficiently
detect this condition.

As for I/O blocking syscalls and locking functions, they could be
remapped to be non-blocking by the library in userland to be handled by
the userland scheduler.  However:

Although I see that kqueue provides good and efficient mechanisms to
handle network I/O polling, I noticed lately that it doesn't provide
similar functionality for disk I/O (or at least the man page doesn't
mention that it might work on other descriptors than sockets, pipes,
FIFOs and vnodes (for modification notification)).

After a short exchange on IRC it appeared that AIO would be a feature
which we're missing to allow I/O asynchroneous operations and polling
for disk I/O.  Without this functionality, any thread blocking for the
disk would also have to be considered candidates for mapping to an LWP
at least temporarily (or to a pool of disk I/O LWP slaves).

It becomes apparent that with all required tools, only non-yielding
threads performing number crunching (and those for which heuristics show
(Continue reading)

Permalink | Reply | 23 comments |
headers
Joachim König-Baltes | 1 Mar 2007 13:58
Picon

Re: M:N and blocking ops without SA, AIO

Matthew Mondor wrote:
> However:
>
> Although I see that kqueue provides good and efficient mechanisms to
> handle network I/O polling, I noticed lately that it doesn't provide
> similar functionality for disk I/O (or at least the man page doesn't
> mention that it might work on other descriptors than sockets, pipes,
> FIFOs and vnodes (for modification notification)).
>
> After a short exchange on IRC it appeared that AIO would be a feature
> which we're missing to allow I/O asynchroneous operations and polling
> for disk I/O.
[...]

> Do others agree that an M:N implementation without SA done with a
> user-space scheduler would greatly benefit from AIO?  If so, were there
> ever plans for AIO to eventually be implemented on NetBSD?  What are the
> known challenges involved that prevented such a feature so far, if any?
> Other than the buffer cache, DMA and interrupts handled by drivers and
> limited part of VFS, UFS layout, the rest of the disk I/O system is
> still currently unknown to me.
>   
If AIO will be implemented, please do not forget to also implement the 
functionality for kevent on
disk I/O too (e.g. prefetch data in the kernel for EVFILT_READ and 
return the number of available bytes
or return the free bytes in a kernel buffer that can be written without 
blocking), taking also NFS
into account.
(Continue reading)

Permalink | Reply | 0 comments |
headers
YAMAMOTO Takashi | 1 Mar 2007 17:08
Picon

Re: sched_changepri

> On Sun, Feb 25, 2007 at 05:21:22PM +0900, YAMAMOTO Takashi wrote:
> 
> > - why doesn't sched_changepri() change l_priority if it was < PUSER? 
> 
> If the LWP has received a priority boost while in the kernel and is waiting
> to run it shouldn't get dropped until it has run.

thanks for explanation.
isn't it better to have a flag to indicate if an lwp is boosted or not,
and kill l_priority?

> Right now the drop in
> priority takes place in userret(). I think that should be moved back into
> remrunqueue() once cpu_switch() is eliminated.

i think userret() is a better place to unboost an lwp than remrunqueue().
if it was in remrunqueue() and we support in-kernel preemption,
an lwp can be preempted immediately after scheduled.

YAMAMOTO Takashi
Permalink | Reply | 14 comments |
headers
Blair Sadewitz | 1 Mar 2007 17:38
Picon

pci_mapreg_map with 64-bit addresses

I've been playing around with radeonfb, and the first (of many, I'm
sure, given I'm using a PCIE card) is that pci_find_mem (called from
pci_mapreg_map) wants a type 0 address (32-bit) but gets a type 4
(64-bit).   I've never done anything like this before, so I'm
wondering what the proper/preferred way is to handle such situations.

I am a bit discouraged by the fact that the PCI vga driver ignores
64-bit BARs. ;)

Thanks,

--Blair

--

-- 
Support WFMU-FM: free-form radio for the masses!
<http://www.wfmu.org/>

"The frivolity and boredom which unsettle the established order, the
vague foreboding of something unknown, these are the heralds of
approaching change.  The gradual crumbling that left unaltered the
face of the whole is cut short by a sunburst which, in one flash,
illuminates the features of the new world."  --G.W.F. Hegel,
_Phenomenology of Spirit_ 5:11
Permalink | Reply | 1 comment |
headers
Martin Husemann | 1 Mar 2007 17:44
Picon

Re: pci_mapreg_map with 64-bit addresses

On Thu, Mar 01, 2007 at 11:38:51AM -0500, Blair Sadewitz wrote:
> I am a bit discouraged by the fact that the PCI vga driver ignores
> 64-bit BARs. ;)

Look at sys/dev/pci/if_ath_pci.c and search for PCI_MAPREG_MEM_TYPE_64BIT.
It is simple to fix, and if you can test it...

Martin
Permalink | Reply | 0 comments |
headers
Tom Spindler | 1 Mar 2007 18:17
Favicon

Re: CVS commit: src/sys/dev/usb

> #define UNICODE_DECOMPOSE    0x01  /* convert to decomposed NF */
> #define UNICODE_PRECOMPOSE   0x02  /* convert to precomposed NF */

To be excessively pedantic: I'd also indicate here whether
you're normalizing to NFD, NFC, NFKD, or NFKC, as defined in
http://www.unicode.org/reports/tr15/ . See my commentary on the
invalid input stuff, too.

> size_t utf8_to_utf16(uint16_t *out, size_t outlen,
> 		     const char *in, size_t inlen,
> 		     int flags, int *errcountp);

Why bother specifying inlen? If you copy out at most outlen chars...
To be extra whingy, I'd suggest you order the args more like strncpy -
e.g. (out, in, inlen, flags, errcountp)

> HFS+ requires file names to be stored in decomposed
> form (u+0065 u+0301), while, IIUC, NTFS requires them to be in
> pre-composed form (u+0E9).  [I have not yet implemnted
> (de)composition.]

From what teh intarweb tells me (references at end), HFS uses
FCD[1] ("Fast C or D") which is a superset[2] of NFD. :-/ Similarly,
When Windows uses NTFS, it happens to use precomposed characters - but
NTFS itself doesn't actually notice or specify which to use.[3] (Worse,
it's only defined to use chars in the BMP (esp for sorting/comparison
purposes.)

>   I'm not sure how to handle invalid imput.
>
(Continue reading)

Permalink | Reply | 5 comments |
headers
Bill Stouder-Studenmund | 1 Mar 2007 18:28
Picon

Re: M:N and blocking ops without SA, AIO

On Thu, Mar 01, 2007 at 05:22:45AM -0500, Matthew Mondor wrote:
> From my understanding, an M:N threading model which uses a pool of
> kernel LWPs without using Scheduler Activations would need to be able to
> poll asynchroneously in its userland library scheduler for any
> potentially blocking syscalls.

I'm not sure. I don't think we've done enough of a post-mortem to tell.

One idea I've been batting around is revive SA, but change the way the 
kernel tells userland to make a new thread. My understanding is that 
scheduler activations were effectively signals, and the kernel would fork 
a new lwp and run the SA upcall on it.

The problem was that the kernel had to be able to allocate a new LWP 
whenever it blocked. That means we had to be able to allocate LWPs even 
when the locking forbid it.

My thought was instead do something more mach-like. Have userland fork
threads. Make a new system call that forks a new thread/lwp and blocks. 
libpthread has already created a thread in userland, and now it just 
passes that into the kernel. The kernel then makes the new lwp and runs 
the thread on it. The call then checks to see if the process is below its 
concurrency level. If not, it blocks until it is. If it is below, it 
immediately returns. Userland then looks for a thread to run and repeats 
the process. If there's no thread to run, the call just passes in a  NULL 
thread, or something to say "nothing more to do, just rearming".

The main point is that we change how a new LWP is created. Rather than 
doign it when we block, we have something sleeping which we wake. It in 
turn does something about it. And we also have a thread sitting around
(Continue reading)

Permalink | Reply | 19 comments |
headers
Manuel Bouyer | 1 Mar 2007 18:53

Re: bus_dmamap_sync() in ti(4) or the lack thereof

On Wed, Feb 28, 2007 at 12:52:21AM +0100, Tobias Nygren wrote:
> Izumi Tsutsui wrote:
> >tnn+nbsd <at> nygren.pp.se wrote:
> >
> >  
> >>ti(4) doesn't work right on sparc64. With DIAGNOSTIC i get tons
> >>of "iommu_dvmamap_load: map still in use".
> >>    
> >
> >This is usually caused by missing bus_dmamap_unload(9).
> >(IMHO other ports should have the similar assersions)
> >
> >  
> >>I took a glance at the ti(4) code and noticed the absence of any
> >>preread/postread dma syncs, despite the fact that the driver relies
> >>heavily on reuse of dma maps. This can't be right, can it?
> >>    
> >
> >It should have proper sync calls otherwise it won't work on
> >non DMA coherent architecture (like sgimips O2) or architectures
> >which require bounce buffer (like some alpha), but many FreeBSD's
> >drivers often lack them since they are not required by "major" ports ;-p
> >
> >  
> >>The fact that the driver appears to reload maps without unloading
> >>them first might be caused by something else, though.
> >>    
> >
> >It seems bus_dmamap_unload(9) is needed before calling ti_newbuf_jumbo()
> >or ti_newbuf_mini() in ti_rxeof().
(Continue reading)

Permalink | Reply | 0 comments |

Gmane