headers
Laurens Van Houtven | 2 Jul 2011 23:22
Gravatar

replacing "break" with "continue" causes line to not be covered w/ coverage.py

Hey.

I'm experiencing some strange behavior. I have a loop, and when I continue to the next element on some condition, it told me that line wasn't covered. Then, I put a pdb.set_trace() in, and I saw that it was, in fact, being run. I put a raise ValueError() instead of that "continue", and it raised. I replaced "continue" with "break", and now coverage.py thinks the line is covered.

What could be going on? Here's the diff:

=== modified file 'twisted/positioning/nmea.py'
--- twisted/positioning/nmea.py    2011-07-02 18:40:44 +0000
+++ twisted/positioning/nmea.py    2011-07-02 21:14:31 +0000
<at> <at> -552,7 +552,7 <at> <at>
             prn, azimuth, elevation, snr = values
 
             if prn is None or snr is None:
-                break
+                continue
 
             satellite = base.Satellite(prn, azimuth, elevation, snr)
             self._sentenceData['_partialBeaconInformation'].beacons.add(satellite)


The code lives in lp:~lvh/twisted/positioning.

The strange thing is even trial --coverage (which is unrelated to coverage.py...) thinks it's not covered. How can that be?
 
cheers

lvh

_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 8 comments |
headers
Ned Batchelder | 3 Jul 2011 03:28
Favicon
Gravatar

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

Ah, the joys of code optimizers.  The peephole optimizer recognizes your code and optimizes away the continue, essentially turning your if statement into a jump directly to the beginning of the loop.  All of your tests that change the continue into a different statement break that optimization, and "prove" to you that the statement is executed.  Coverage.py can't measure the continue as being executed because Python never reports that it is executed.

When I first discovered this in my own code, I thought, "We've seen this before: C compilers have switches to disable optimizations for precisely this sort of reason."  I wrote a bug against Python asking for a way to disable optimizations in those cases where it's more important to reason about the code than for the code to run quickly.  The bug is here: http://bugs.python.org/issue2506 , it was frustratingly closed as "won't fix".  Raymond Hettinger has argued against complicating the optimizer in other places, mostly, it seems due to a lack of automated testing of the optimizer, and the fear that comes from that realization.  Ironically, the best way to test an optimizer is to provide a way to completely disable it, and then to run code twice, once with optimization and once without, and see that they produce the same results.

Any help you can provide to get people on board with a well-tested optimizer that can be completely disabled, would be appreciated.  Until then, there's nothing to do except add a "#pragma: no cover" comment to that line.

--Ned.

On 7/2/2011 5:22 PM, Laurens Van Houtven wrote:
Hey.

I'm experiencing some strange behavior. I have a loop, and when I continue to the next element on some condition, it told me that line wasn't covered. Then, I put a pdb.set_trace() in, and I saw that it was, in fact, being run. I put a raise ValueError() instead of that "continue", and it raised. I replaced "continue" with "break", and now coverage.py thinks the line is covered.

What could be going on? Here's the diff:

=== modified file 'twisted/positioning/nmea.py'
--- twisted/positioning/nmea.py    2011-07-02 18:40:44 +0000
+++ twisted/positioning/nmea.py    2011-07-02 21:14:31 +0000
<at> <at> -552,7 +552,7 <at> <at>
             prn, azimuth, elevation, snr = values
 
             if prn is None or snr is None:
-                break
+                continue
 
             satellite = base.Satellite(prn, azimuth, elevation, snr)
             self._sentenceData['_partialBeaconInformation'].beacons.add(satellite)


The code lives in lp:~lvh/twisted/positioning.

The strange thing is even trial --coverage (which is unrelated to coverage.py...) thinks it's not covered. How can that be?
 
cheers
lvh

_______________________________________________ testing-in-python mailing list testing-in-python <at> lists.idyll.org http://lists.idyll.org/listinfo/testing-in-python 
_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 7 comments |
headers
Eric Snow | 3 Jul 2011 07:25
Picon
Gravatar

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

On Sat, Jul 2, 2011 at 7:28 PM, Ned Batchelder <ned <at> nedbatchelder.com> wrote:
> Ah, the joys of code optimizers.  The peephole optimizer recognizes your
> code and optimizes away the continue, essentially turning your if statement
> into a jump directly to the beginning of the loop.  All of your tests that
> change the continue into a different statement break that optimization, and
> "prove" to you that the statement is executed.  Coverage.py can't measure
> the continue as being executed because Python never reports that it is
> executed.
>
> When I first discovered this in my own code, I thought, "We've seen this
> before: C compilers have switches to disable optimizations for precisely
> this sort of reason."  I wrote a bug against Python asking for a way to
> disable optimizations in those cases where it's more important to reason
> about the code than for the code to run quickly.  The bug is here:
> http://bugs.python.org/issue2506 , it was frustratingly closed as "won't
> fix".  Raymond Hettinger has argued against complicating the optimizer in
> other places, mostly, it seems due to a lack of automated testing of the
> optimizer, and the fear that comes from that realization.  Ironically, the
> best way to test an optimizer is to provide a way to completely disable it,
> and then to run code twice, once with optimization and once without, and see
> that they produce the same results.
>
> Any help you can provide to get people on board with a well-tested optimizer
> that can be completely disabled, would be appreciated.  Until then, there's
> nothing to do except add a "#pragma: no cover" comment to that line.
>

Incidentally, there is an effort right now to introduce better
optimization at the AST level, but I don't recall the impact that will
have on the peephole optimizer.

-eric

> --Ned.
>
> On 7/2/2011 5:22 PM, Laurens Van Houtven wrote:
>
> Hey.
>
> I'm experiencing some strange behavior. I have a loop, and when I continue
> to the next element on some condition, it told me that line wasn't covered.
> Then, I put a pdb.set_trace() in, and I saw that it was, in fact, being run.
> I put a raise ValueError() instead of that "continue", and it raised. I
> replaced "continue" with "break", and now coverage.py thinks the line is
> covered.
>
> What could be going on? Here's the diff:
>
> === modified file 'twisted/positioning/nmea.py'
> --- twisted/positioning/nmea.py    2011-07-02 18:40:44 +0000
> +++ twisted/positioning/nmea.py    2011-07-02 21:14:31 +0000
>  <at>  <at>  -552,7 +552,7  <at>  <at> 
>              prn, azimuth, elevation, snr = values
>
>              if prn is None or snr is None:
> -                break
> +                continue
>
>              satellite = base.Satellite(prn, azimuth, elevation, snr)
>
> self._sentenceData['_partialBeaconInformation'].beacons.add(satellite)
>
>
> The code lives in lp:~lvh/twisted/positioning.
>
> The strange thing is even trial --coverage (which is unrelated to
> coverage.py...) thinks it's not covered. How can that be?
>
> cheers
> lvh
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python <at> lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python <at> lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>
Permalink | Reply | 0 comments |
headers
Laurens Van Houtven | 3 Jul 2011 11:40
Gravatar

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

Gah. I figured it'd have to be something this stupid.

Thanks, Ned!
 
cheers

lvh

_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 0 comments |
headers
Laurens Van Houtven | 3 Jul 2011 12:41
Gravatar

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

Hang on. This appears to be for *unconditional* jumps. My jump isn't unconditional, it's "if snr is None or prn is None:". How does that get optimized away? Where's the always-true condition? "prn is None or snr is None" is certainly not always true, in fact, that's the exception, not the norm.

_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 4 comments |
headers
Eric Henry | 3 Jul 2011 15:09
Picon
Favicon

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

Right, but the point is that you're not doing anything else if the condition is true.


I can't say for sure, but it would seem that it could easily convert the if ... continue into a "branch on equal" type bytecode. What happens if you have a print statement before the continue?

Eric


On Sun, Jul 3, 2011 at 6:41 AM, Laurens Van Houtven <_ <at> lvh.cc> wrote:
Hang on. This appears to be for *unconditional* jumps. My jump isn't unconditional, it's "if snr is None or prn is None:". How does that get optimized away? Where's the always-true condition? "prn is None or snr is None" is certainly not always true, in fact, that's the exception, not the norm.

_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python


_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 3 comments |
headers
Laurens Van Houtven | 3 Jul 2011 15:18
Gravatar

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

On Sun, Jul 3, 2011 at 3:09 PM, Eric Henry <henryer2 <at> msu.edu> wrote:
Right, but the point is that you're not doing anything else if the condition is true.

Ugh, right. Sorry, eyes glazing over...
 
I can't say for sure, but it would seem that it could easily convert the if ... continue into a "branch on equal" type bytecode. What happens if you have a print statement before the continue?

Same as with pretty much any modification to that body: line is covered.
 

Eric




I'm probably just going to put # pragma: no cover there and be done with it.

cheers
lvh 
_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 0 comments |
headers
Ned Batchelder | 3 Jul 2011 15:42
Favicon
Gravatar

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

(Enormous hand-waving over significant details below)

If your code is this:

    10:    for item in container:
    11:        if item.some_condition() or item.some_other():
    12:            continue
    13:        item.do_real_stuff()

Then the generated code looks very crudely something like:

    10a:    start iterating container
    10b:    get next item
    11a:        if item.some_condition() jump to 12a
    11b:        if item.some_other() jump to 12a
    11c:        jump to 13a
    12a:        jump to 10b
    13a:        item.do_real_stuff()
    13b:        jump to 10b

The peephole optimizer sees that lines 11a and 11b include a jump to 12a, but 12a is an unconditional jump to 10b, so it changes line 11a and 11b:

    11a:        if item.some_condition() jump to 10b
    11b:        if item.some_other() jump to 10b

Line 12 is now never executed.

--Ned.


On 7/3/2011 9:09 AM, Eric Henry wrote:
Right, but the point is that you're not doing anything else if the condition is true.

I can't say for sure, but it would seem that it could easily convert the if ... continue into a "branch on equal" type bytecode. What happens if you have a print statement before the continue?

Eric


On Sun, Jul 3, 2011 at 6:41 AM, Laurens Van Houtven <_ <at> lvh.cc> wrote:
Hang on. This appears to be for *unconditional* jumps. My jump isn't unconditional, it's "if snr is None or prn is None:". How does that get optimized away? Where's the always-true condition? "prn is None or snr is None" is certainly not always true, in fact, that's the exception, not the norm.

_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python


_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 1 comment |
headers
Eric Henry | 3 Jul 2011 15:51
Picon
Favicon

Re: replacing "break" with "continue" causes line to not be covered w/ coverage.py

I knew I should have included some psuedo bytecode...


Conversely, if you have some code that does something between the if and the continue, you'll have some other code on line 12a and line 12b will be the jump, and the optimizer won't want to remove line 12a anymore.

Eric


On Sun, Jul 3, 2011 at 9:42 AM, Ned Batchelder <ned <at> nedbatchelder.com> wrote:
(Enormous hand-waving over significant details below)

If your code is this:

    10:    for item in container:
    11:        if item.some_condition() or item.some_other():
    12:            continue
    13:        item.do_real_stuff()

Then the generated code looks very crudely something like:

    10a:    start iterating container
    10b:    get next item
    11a:        if item.some_condition() jump to 12a
    11b:        if item.some_other() jump to 12a
    11c:        jump to 13a
    12a:        jump to 10b
    13a:        item.do_real_stuff()
    13b:        jump to 10b

The peephole optimizer sees that lines 11a and 11b include a jump to 12a, but 12a is an unconditional jump to 10b, so it changes line 11a and 11b:

    11a:        if item.some_condition() jump to 10b
    11b:        if item.some_other() jump to 10b

Line 12 is now never executed.

--Ned.



On 7/3/2011 9:09 AM, Eric Henry wrote:
Right, but the point is that you're not doing anything else if the condition is true.

I can't say for sure, but it would seem that it could easily convert the if ... continue into a "branch on equal" type bytecode. What happens if you have a print statement before the continue?

Eric


On Sun, Jul 3, 2011 at 6:41 AM, Laurens Van Houtven <_ <at> lvh.cc> wrote:
Hang on. This appears to be for *unconditional* jumps. My jump isn't unconditional, it's "if snr is None or prn is None:". How does that get optimized away? Where's the always-true condition? "prn is None or snr is None" is certainly not always true, in fact, that's the exception, not the norm.

_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python



_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 0 comments |
headers
Eric Henry | 3 Jul 2011 21:26
Picon
Favicon

flexmock

Am I missing something, or does flexmock lack item assignment support?

Eric
_______________________________________________
testing-in-python mailing list
testing-in-python <at> lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python
Permalink | Reply | 3 comments |

Gmane