headers
Frantisek Rysanek | 1 Aug 2007 23:54
Picon

Re: serial flow control appears broken

On 28 Jul 2007 at 17:39, Lee Howard wrote:
> >>Curiously, the session at 38400 bps that skipped 858 bytes... coincided, 
> >>not just in sequence but also in precice timing within the session, with 
> >>a small but noticeable disk load that I caused by grepping through a 
> >>hundred session logs.  
>
> I've attached dmesg output.
>
I'd like to summarize a few points that have appeared in this long 
thread.

Someone else has mentioned interrupt latency.
I seem to recall that the RTAI people can measure that. 
http://issaris.org/rtai/list.php
They report that some P4-class Intel chipsets (i845/865 for example) 
that are using message-signaled interrupt passing across the HubLink 
(serialized PCI) from the South Bridge to the North Bridge, exhibit 
interrupt latencies on the order of milliseconds, if the HubLink is 
loaded by some heavy PCI transfers, such as intensive disk IO. 
Reportedly this is because the interrupt messages are competing for 
the "right of the way" with the DMA-driven PCI transactions that 
carry the bulk of the disk IO payload, where a single PCI transaction 
can take that long.

Now: you've quoted a Shuttle HOT-661. That's an older chipset, i440 
class. That one was still using the traditional out-of-band interrupt 
delivery mechanism, using the INTR signal + vector reading over the 
ISA bus. Yet, come to think of that, the native PCI interconnect 
between the MCH and the PIIX4 can still present a bottleneck to IRQ 
processing. The AT PIC is located in the PIIX4 (south bridge). After
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mark Lord | 2 Aug 2007 16:57
Picon
Favicon

Re: serial flow control appears broken

Maciej W. Rozycki wrote:
> On Sat, 28 Jul 2007, Russell King wrote:
> 
>> Essentially, any complex interrupt handler (such as an IDE interrupt
>> doing a multi-sector PIO transfer _in interrupt context_) can cause this
>> kind of starvation.  That's why Linux 1.x had bottom halves - so that
>> the time consuming work could be moved out of the interrupt handler,
>> thereby causing minimal the blockage of other interrupts.
>>
>> Unfortunately, that kind of design has been long since forgotten.
>> Apparantly modern machines are fast enough that it doesn't have to be
>> worried about anymore...  Or are they?
> 
>  I would guess it is not that the machines are fast enough, but that this 
> two-level processing makes things more complicated.  Enough that most 
> people would not bother digging into it unless really forced.  Only 
> occasional latency problems are probably not enough of a force.

I don't believe the speed of the machine has much to do with it,
as IDE PIO is always at pretty much the same speed (or slower)
regardless of the CPU speed.

Best case is about .120 usec per 16-bit word, but that doesn't often pan out
in practice.  More typical is something closer to 1 usec per 16-bit word.

So, for multcount=16 (very common), best case is 16 * 256 * .120 = 491 usec,
plus extra overhead for reading the IDE status register (another usec or so),
and other stuff.  Figure maybe 500usec total per interrupt for multcount=16
in the best case, or 4000usec in the worst case.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Robert Hancock | 2 Aug 2007 18:14
Picon
Favicon

Re: serial flow control appears broken

Mark Lord wrote:
> I don't believe the speed of the machine has much to do with it,
> as IDE PIO is always at pretty much the same speed (or slower)
> regardless of the CPU speed.
> 
> Best case is about .120 usec per 16-bit word, but that doesn't often pan 
> out
> in practice.  More typical is something closer to 1 usec per 16-bit word.
> 
> So, for multcount=16 (very common), best case is 16 * 256 * .120 = 491 
> usec,
> plus extra overhead for reading the IDE status register (another usec or 
> so),
> and other stuff.  Figure maybe 500usec total per interrupt for multcount=16
> in the best case, or 4000usec in the worst case.
> 
> At 115200bps, we get a byte every 86 usec or so.  Assuming the UART FIFO
> is set to interrupt (warn) us at 12/16 full, we have 4*86 = 344 usec to
> respond and de-assert RTS.  Less than that in practice.
> 
> Conclusion:  using IDE multisector PIO is not a good idea with high speed
> serial transfers happening, since we cannot respond quickly enough.
> 
> It might be possible to set the buffer underrun threshold lower in the 
> UART (?).
> 
> All that said, I doubt that his system is using IDE PIO in the first place.
> Dunno how long IDE DMA interrupts take, but it's probably in the 20-50 
> usec range.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mark Lord | 2 Aug 2007 18:29
Picon
Favicon

Re: serial flow control appears broken

Robert Hancock wrote:
> Mark Lord wrote:
>> I don't believe the speed of the machine has much to do with it,
>> as IDE PIO is always at pretty much the same speed (or slower)
>> regardless of the CPU speed.
>>
>> Best case is about .120 usec per 16-bit word, but that doesn't often 
>> pan out
>> in practice.  More typical is something closer to 1 usec per 16-bit word.
>>
>> So, for multcount=16 (very common), best case is 16 * 256 * .120 = 491 
>> usec,
>> plus extra overhead for reading the IDE status register (another usec 
>> or so),
>> and other stuff.  Figure maybe 500usec total per interrupt for 
>> multcount=16
>> in the best case, or 4000usec in the worst case.
>>
>> At 115200bps, we get a byte every 86 usec or so.  Assuming the UART FIFO
>> is set to interrupt (warn) us at 12/16 full, we have 4*86 = 344 usec to
>> respond and de-assert RTS.  Less than that in practice.
>>
>> Conclusion:  using IDE multisector PIO is not a good idea with high speed
>> serial transfers happening, since we cannot respond quickly enough.
>>
>> It might be possible to set the buffer underrun threshold lower in the 
>> UART (?).
>>
>> All that said, I doubt that his system is using IDE PIO in the first 
>> place.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Robert Hancock | 2 Aug 2007 18:40
Picon
Favicon

Re: serial flow control appears broken

Mark Lord wrote:
>> I think that PIO transfers only have to be done with interrupts 
>> disabled on really old, evil controllers (without unmask set). I don't 
>> think libata ever disables interrupts during transfers(?)
> 
> That's what "hdparm -u1" (or -u0) controls.
> 
> But it doesn't matter a whit here.  The problem is that the IDE interrupt
> handling can take a long time, regardless of whether it unmasks IRQs or 
> not.
> And if that IDE interrupt interrupts a serial interrupt, then the serial
> stuff won't get handled until the IDE stuff completes.  Thus the problem.
> 
> The "fix" could be to have the serial IRQ handler never unmask interrupts,
> but that's a bit unsociable to others.  The IDE stuff really needs to not
> do so much during the actual IRQ handler.
> 
> Ingo's RT patches would probably fix all of this.

libata also doesn't do the actual PIO transfer from the interrupt 
handler like old IDE does, either, and it only disables interrupts for 
the transfer if it's transferring to/from high memory..

--

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr <at> nospamshaw.ca
Home Page: http://www.roberthancock.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-serial" in
(Continue reading)

Permalink | Reply | 0 comments |
headers
Alan Cox | 2 Aug 2007 18:57
Picon

Re: serial flow control appears broken

> I think that PIO transfers only have to be done with interrupts disabled 
> on really old, evil controllers (without unmask set). I don't think 
> libata ever disables interrupts during transfers(?)

Currently libata PIO is mostly done in the IRQ path. Albert Lee was doing
some work on that but its actually very hard to fix without doing polled
PIO.

-
To unsubscribe from this list: send the line "unsubscribe linux-serial" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Robert Hancock | 2 Aug 2007 19:02
Picon
Favicon

Re: serial flow control appears broken

Alan Cox wrote:
>> I think that PIO transfers only have to be done with interrupts disabled 
>> on really old, evil controllers (without unmask set). I don't think 
>> libata ever disables interrupts during transfers(?)
> 
> Currently libata PIO is mostly done in the IRQ path. Albert Lee was doing
> some work on that but its actually very hard to fix without doing polled
> PIO.

Ah, right. Misread the code.
-
To unsubscribe from this list: send the line "unsubscribe linux-serial" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Alan Cox | 2 Aug 2007 19:13
Picon

Re: serial flow control appears broken

> That's what "hdparm -u1" (or -u0) controls.

Only some of the time.

> Ingo's RT patches would probably fix all of this.

The worst case IDE times we've seen for executing a single indivisible
un-interruptible I/O cycle with a drive are around 1mS. Thats a hardware
limit.

Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-serial" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Maciej W. Rozycki | 3 Aug 2007 11:32
Favicon

Re: serial flow control appears broken

On Thu, 2 Aug 2007, Alan Cox wrote:

> Currently libata PIO is mostly done in the IRQ path. Albert Lee was doing
> some work on that but its actually very hard to fix without doing polled
> PIO.

 Hmm, when the drive signals it is ready for a PIO data transfer can't 
just the interrupt handler mask the originating interrupt and post a 
softirq to handle the case?  That should be rather straightforward.

  Maciej
-
To unsubscribe from this list: send the line "unsubscribe linux-serial" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Permalink | Reply | 0 comments |
headers
Lee Howard | 4 Aug 2007 20:19

Re: serial flow control appears broken

Maciej W. Rozycki wrote:

>On Fri, 27 Jul 2007, Lee Howard wrote:
>
>  
>
>>Okay, so let's say we've got a loop around a blocking read on the modem file
>>descriptor...
>>
>> for (;;) {
>>     read some data from modem
>>     process data from modem
>>     if (end-of-data detected) break;
>> }
>>
>>Are you suggesting that the application should be using deasserting RTS after
>>the read and asserting it before?
>>    
>>
>
> It certainly could -- you were asking how it would know. ;-)
>

So, to test... I put this in the application before every read:

    int flags;
    ioctl(modemFd, TIOCMGET, &flags);
    flags |= TIOCM_RTS;
    ioctl(modemFd, TIOCMSET, &flags);
(Continue reading)

Permalink | Reply | 0 comments |

Gmane