headers
Larry Hastings | 1 Apr 2012 02:07

Re: Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?


On 03/31/2012 12:47 AM, Victor Stinner wrote:
>> Can you go into more detail about QPC()'s issues?
> Yes, see the PEP:
> http://www.python.org/dev/peps/pep-0418/#windows-queryperformancecounter

FYI, Victor, the PEP is slightly incomplete.  Not that this is your 
fault--you've done your homework.  But I've actually lived through it.  
I was a professional Win32 developer for 15 years, and I attempted to 
write a game on Windows back in the early-mid 2000s.

On Windows XP, QPC /usually/ uses the ACPI timer in my experience, but 
sometimes uses RDTSC.  Both of these had troubles.

With TSC, there's the clock skew between the two cores that they claim 
was fixed in SP2.  (You could also sidestep this problem by setting core 
affinity to the same core for all your threads that were going to 
examine the time.)  But there's another problem: the TSC frequency 
actually *does* change when SpeedStep kicks in.  I know someone who 
complained bitterly about running Half-Life 2 on their shiny new laptop, 
and when it'd overheat SpeedStep would knock down the processor speed 
and the game's logic update rate would drop in half and now Gordon was 
running through molasses.

With the ACPI timer, that's where you saw the 
leap-forwards-by-several-seconds-under-heavy-load problem (the cited 
MSKB article KB274323).  That only happened with a specific south bridge 
chipset, which was Pentium-III-only.  I never heard about anyone 
experiencing that problem--personally I had good experiences with that 
timer.  The downside of the ACPI timer is that it's slow to read; it
(Continue reading)

Permalink | Reply | 1 comment |
headers
Nick Coghlan | 1 Apr 2012 03:41
Picon
Gravatar

Re: Issue 14417: consequences of new dict runtime error


On Apr 1, 2012 8:54 AM, "Benjamin Peterson" <benjamin <at> python.org> wrote:
>
> 2012/3/31 Guido van Rossum <guido <at> python.org>:
> > Try reducing sys.setcheckinterval().
>
> setcheckinterval() is a no-op since the New-GIL. sys.setswitchinterval
> has superseded it

Ah, that's at least one thing wrong with my initial attempt - I was still thinking in terms of "number of bytecodes executed". Old habits die hard :)

--
Sent from my phone, thus the relative brevity :) 

<div>
<p><br>
On Apr 1, 2012 8:54 AM, "Benjamin Peterson" &lt;<a href="mailto:benjamin <at> python.org">benjamin <at> python.org</a>&gt; wrote:<br>
&gt;<br>
&gt; 2012/3/31 Guido van Rossum &lt;<a href="mailto:guido <at> python.org">guido <at> python.org</a>&gt;:<br>
&gt; &gt; Try reducing sys.setcheckinterval().<br>
&gt;<br>
&gt; setcheckinterval() is a no-op since the New-GIL. sys.setswitchinterval<br>
&gt; has superseded it</p>
<p>Ah, that's at least one thing wrong with my initial attempt - I was still thinking in terms of "number of bytecodes executed". Old habits die hard :)</p>
<p>--<br>
Sent from my phone, thus the relative brevity :) </p>
</div>
Permalink | Reply | 0 comments |
headers
Victor Stinner | 1 Apr 2012 03:56
Picon
Gravatar

Re: Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

> FYI, Victor, the PEP is slightly incomplete.

Sure. What should be added to the PEP?

> But there's another problem: the TSC frequency actually *does*
> change when SpeedStep kicks in.  I know someone who complained bitterly
> about running Half-Life 2 on their shiny new laptop, and when it'd overheat
> SpeedStep would knock down the processor speed and the game's logic update
> rate would drop in half and now Gordon was running through molasses.

Yes, I already changed the PEP to not use QPC anymore for
time.monotonic() because it has too many issues.

I didn't mention the CPU frequency change issue in the PEP because I
failed to find recent information about this issue. Is it an old bug
or does it still occur with Windows Vista or Seven? Does Windows Vista
and Seven still use TSC or they prefer other hardware clocks like ACPI
PMT or HPET?

Last info that I found: "Historically, the TSC increased with every
internal processor clock cycle, but now the rate is usually constant
(even if the processor changes frequency) and usually equals the
maximum processor frequency. The instructor RDTSC can be used to read
this counter."

> The documentation warnings about timeBeginPeriod is ancient, like Windows 95 era.

Which warning? The power consumption issue mentioned in the PEP?

> Likewise with calling into winmm--it shipped with every OS 3.3
> supports.  It's just not a big deal and you don't need to mention it in the
> PEP.

I mentioned that the function requires the winmm library because a
static or dynamic link to a library can be an issue. Especially if we
use this function in Python core. clock_gettime(CLOCK_REALTIME) is not
used on Linux for _PyTime_gettimeofday() because it requires to link
Python to the rt (real-time) library, but it is used by time.time() (I
changed it recently).

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 0 comments |
headers
Victor Stinner | 1 Apr 2012 04:37
Picon
Gravatar

Re: Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

> If we provide a way to check if the monotonic clock is monotonic (or
> not), I agree to drop the flag from time.monotonic(fallback=True) and
> always fallback. I was never a fan of the "truly monotonic clock".
>
> time.clock_info('monotonic')['is_monotonic'] is a good candidate to
> store this information.

I updated the PEP to add time.get_clock_info() and to drop the
fallback parameter of time.monotonic() (which now always falls back).

Because "monotonic" word cannot define time.monotonic() anymore, I
suggest to rename the time.monotonic() function to time.steady(). So
we would have:

- time.steady() may or may not be monotonic, but its is as steady as possible.
- time.get_clock_info('steady')['is_monotonic'] which looks less
surprising than time.get_clock_info('monotonic')['is_monotonic']

It doesn't follow the C++ steady_clock definition, but it looks like
the Boost library doesn't follow the C++ definition... (it uses
CLOCK_MONOTONIC on Linux)

By the way, it now prefer to use CLOCK_MONOTONIC instead of
CLOCK_MONOTONIC_RAW on Linux. It is what I need in practice. If the
hardware clock is a little bit too fast or too slow, NTP adjusts its
rate so a delta of two timestamps is really a number of seconds. It's
not yet written explicitly in the PEP, but the unit of
time.monotonic/time.steady and time.highres is a second.

Victor
Permalink | Reply | 1 comment |
headers
Guido van Rossum | 1 Apr 2012 05:46
Favicon

Re: Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?

On Sat, Mar 31, 2012 at 7:37 PM, Victor Stinner
<victor.stinner <at> gmail.com> wrote:
>> If we provide a way to check if the monotonic clock is monotonic (or
>> not), I agree to drop the flag from time.monotonic(fallback=True) and
>> always fallback. I was never a fan of the "truly monotonic clock".
>>
>> time.clock_info('monotonic')['is_monotonic'] is a good candidate to
>> store this information.
>
> I updated the PEP to add time.get_clock_info() and to drop the
> fallback parameter of time.monotonic() (which now always falls back).
>
> Because "monotonic" word cannot define time.monotonic() anymore, I
> suggest to rename the time.monotonic() function to time.steady(). So
> we would have:
>
> - time.steady() may or may not be monotonic, but its is as steady as possible.
> - time.get_clock_info('steady')['is_monotonic'] which looks less
> surprising than time.get_clock_info('monotonic')['is_monotonic']
>
> It doesn't follow the C++ steady_clock definition, but it looks like
> the Boost library doesn't follow the C++ definition... (it uses
> CLOCK_MONOTONIC on Linux)
>
> By the way, it now prefer to use CLOCK_MONOTONIC instead of
> CLOCK_MONOTONIC_RAW on Linux. It is what I need in practice. If the
> hardware clock is a little bit too fast or too slow, NTP adjusts its
> rate so a delta of two timestamps is really a number of seconds. It's
> not yet written explicitly in the PEP, but the unit of
> time.monotonic/time.steady and time.highres is a second.

Hmm... I believe NTP can also slew the clock to deal with leap seconds
(which the POSIX standard requires must be ignored). That is, when a
leap second is inserted, the clock is supposed to stop value for one
second. What actually happens is that for some time around the leap
second (I've heard maybe a day), the clock is slowed down slightly.
I'm guessing that this affects CLOCK_MONOTONIC but not
CLOCK_MONOTONIC_RAW. Personally I'd rather use the latter -- if I want
to be synchronous with wall clock time, I can just use time.time().

--

-- 
--Guido van Rossum (python.org/~guido)
Permalink | Reply | 0 comments |
headers
Lennart Regebro | 1 Apr 2012 15:16
Picon
Gravatar

Re: datetime module and pytz with dateutil

On Sat, Mar 31, 2012 at 21:20, Terry Reedy <tjreedy <at> udel.edu> wrote:
> The Windows installer, by default, installs tcl/tk while Python on other
> systems uses the system install. Why can't we do the same for the Olson
> database?

The problem is that it needs updating.
We could include pytz, but it would be useless on Windows, unless you
also separately installs the Olson database. But including it and
updating it is not Python's job and should not be.

//Lennart
Permalink | Reply | 1 comment |
headers
Lennart Regebro | 1 Apr 2012 15:17
Picon
Gravatar

Re: PEP 418: Add monotonic clock

On Sat, Mar 31, 2012 at 11:50, Nadeem Vawda <nadeem.vawda <at> gmail.com> wrote:
> Out of the big synonym list Guido posted, I rather like time.stopwatch() - it
> makes it more explicit that the purpose of the function is to measure intervals,
> rather identifying absolute points in time.

I guess it's the least bad.

//Lennart
Permalink | Reply | 0 comments |
headers
Etienne Robillard | 1 Apr 2012 17:49
Picon

Re: Issue 14417: consequences of new dict runtime error

On 03/30/2012 03:25 PM, R. David Murray wrote:
> On Fri, 30 Mar 2012 15:13:36 -0400, Etienne Robillard<animelovin <at> gmail.com>  wrote:
>> So far I was only attempting to verify whether this is related to
>> PEP-416 or not. If this is indeed related PEP 416, then I must obviously
>> attest that I must still understand why a immutable dict would prevent
>> this bug or not...
>
> OK, that seems to be the source of your confusion, then.  This has
> nothing to do with PEP-416.
>
> We are talking about issue Issue 14417 (like it says in the subject),
> which in turn is a reaction to the fix for issue 14205.
>
> --David
>

Don't be so naive, David. This issue is more likely related to immutable 
dicts whether you like it or not, otherwise there would be no need to 
patch python 3.3 and include a new dict proxy type without exposing it 
fully.

And secondly this is not only a speculation but my humble understanding 
of the interdependencies which seems to be related to the inclusion of a 
new dict proxy (immutable) mapper affecting invariably code expecting a 
mutable dictionary lookup to succeed whenever the dict structure has 
changed by (ie) overriding __hash__.

Now if you don't mind, I don't mind to remind you how cell phones can
be unsafe on an extended period of times for your health and brain so I 
really don't recommend their uses for cloud-based platforms requiring 
more advanced thread locking mecanism than what is being included in 
traditional CPython using standard (mutable) dicts.

Regards,
Etienne
Permalink | Reply | 0 comments |
headers
Terry Reedy | 1 Apr 2012 23:29
Picon
Favicon

Re: datetime module and pytz with dateutil

On 4/1/2012 9:16 AM, Lennart Regebro wrote:
> On Sat, Mar 31, 2012 at 21:20, Terry Reedy<tjreedy <at> udel.edu>  wrote:
>> The Windows installer, by default, installs tcl/tk while Python on other
>> systems uses the system install. Why can't we do the same for the Olson
>> database?
>
> The problem is that it needs updating.
> We could include pytz, but it would be useless on Windows, unless you
> also separately installs the Olson database. But including it and
> updating it is not Python's job and should not be.

My main point is that I (as a Windows's user) do not think that 
difficulties with Windows should stop inclusion of a useful module. On 
import, pytz should check for database accessibility and raise an 
exception if not, and possibly refer to manual section on how to make it 
accessible.

--

-- 
Terry Jan Reedy
Permalink | Reply | 0 comments |
headers
Brian Curtin | 1 Apr 2012 23:46
Favicon

Re: .{git,bzr}ignore in cpython HG repo

On Fri, Mar 30, 2012 at 10:00, Matěj Cepl <mcepl <at> redhat.com> wrote:
> Why does HG cpython repo contains .{bzr,git}ignore at all?
> IMHO, all .*ignore files should be strictly repository dependent and they
> should not be mixed together.

For what reason? Are the git or bzr files causing issues on HG?
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 16 comments |

Gmane