headers
Ray Schumacher | 1 Jul 2005 21:11
Favicon

Numeric.where vs. ctypes-and-C DLL vs PyInline vs weave.inline

In addition to the previous post, I tried PyInline and weave for executing a C routine over a Numeric array.
PyInline has about the same calling overhead as ctypes, apparently, about 240us for a simple C statement.
It also requires some extra hoops to digest the pointer to short I need (  void b2int(short *f, int N)   ), so I
didn't fully try it.

weave.inline, however, was very good, with different issues. I found I had to patch msvccompiler.py for MS
C 7.1 .NET, but that's online.

        import weave
        inline = weave.inline # for speed
        N = 1000
        code="int i; for(i = 0; i < N; i++){if (dataArray[i]>8191){dataArray[i]-=16383;}}"
        inline(code, ['dataArray', 'N']) # just pass Python objects

This created  sc_019a1cf36209cb2dfc688820080541ef0.pyd in C:\Documents and Settings\rays\Local Settings\Temp\rays\python24_compiled\
Using the above code is slow; ~32000 us, as the compiler checks or runs each time. However, after much
fiddling and dir()s, I copied the long-name.pyd to C:\Python24\DLLs and just did

        import sc_019a1cf36209cb2dfc688820080541ef0
        b2iw = sc_019a1cf36209cb2dfc688820080541ef0.compiled_func   # note the exposed function!
        N = 1
        t1=clock()
        b2iw({'dataArray':dataArray}, {'N':N})    # note the dicts!
        t2 = clock()
        print 'weave', round((t2-t1)*1000000, 1), 'us'

25us! (~300us for N=10000) I think that's as close to C speed as I can expect, although I'm looking at the
compiler options in msvccompiler.py for P4 optimization.
I still get "Missing compiler_cxx fix for MSVCCompiler" on the initial compile, but apparently to no harm.
(Continue reading)

Permalink | Reply | 0 comments |
headers
JUrner | 1 Jul 2005 12:34
Picon

wnd 0.1.12 released


wnd 0.1.12 released

wnd is a python only gui framework for win32 systems,
using low level MS api calls. It lets you setup highly native
guis with low memory footprint and low distibution size.

The module comes equipped with wrapper classes for 
comon controls, common dialogs, classes wrapping gdi
functionality, a bare bones dialog and menu editor and other
modules peeking into the world of the win32 api. A documentation
in chm format comes shipped along with the module.

Hilights in this release are:

    - drag and drop support now implemented
    - added dataobject and clipboard support
    - added two new controls handling files, folders and drives
        and an Explorer like sample gui to demonstrate shell and
        dragdrop capabilities of the framework
    - many other bugfixes and enhancements

    For details see the change log

BTW did I mention that co-writers, patch-suppliers, bug-sniffers
and other people with to much spare time are welcome to the project ?

Anyway, get the new version at http://sourceforge.net/projects/wnd/

-------------------------------------------------------
(Continue reading)

Permalink | Reply | 0 comments |
headers
Andreas Degert | 3 Jul 2005 16:28
Picon
Favicon

Re: subclasses of ctypes types

Thomas Heller <theller <at> python.net> writes:

> Let me, after quite some time, followup on this discussion again.
>
> [Andreas Degert]
>
[...]

Hi Thomas,

I've been rather busy and will be offline for the next 4 weeks (being
in south america). I'll be back on the list after that.

ciao
Andreas

-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
Permalink | Reply | 0 comments |
headers
Mike C. Fletcher | 4 Jul 2005 03:21
Favicon
Gravatar

Proposal: add "mode" flag to dlopen functions

I've just tracked down why GLUT refuses to load on Linux machines. 
Basically the DLL is assuming that the GL and GLU libraries will have
been loaded with the RTLD_GLOBAL flag to allow for resolving the symbols
in libglut.so .  At the moment ctypes always uses RTLD_LOCAL (save on
cygwin, where RTLD_LOCAL isn't defined), so the GL symbols aren't
available to GLUT and the GLUT load fails.

I'd like to propose adding a mode flag to the CDLL init and to the
py_dl_open function such that dlopen can receive a RTLD_GLOBAL flag when
required.

Patches below; note that this has only been tested (and that fairly
minimally) on Linux, testing on cygwin and Win32 (and OSX) will be needed.

Enjoy all,
Mike

Index: ctypes/__init__.py
===================================================================
RCS file: /cvsroot/ctypes/ctypes/ctypes/__init__.py,v
retrieving revision 1.61.2.6
diff -u -3 -p -r1.61.2.6 __init__.py
--- ctypes/__init__.py    15 Apr 2005 18:20:25 -0000    1.61.2.6
+++ ctypes/__init__.py    4 Jul 2005 01:18:18 -0000
 <at>  <at>  -14,6 +14,10  <at>  <at>  from _ctypes import Union, Structure, Ar
 from _ctypes import _Pointer
 from _ctypes import CFuncPtr as _CFuncPtr
 from _ctypes import __version__ as _ctypes_version
+try:
+    from _ctypes import RTLD_LOCAL, RTLD_GLOBAL
(Continue reading)

Permalink | Reply | 2 comments |
headers
Mark McMahon | 5 Jul 2005 18:47
Favicon

RE: Numeric.where vs. ctypes-and-C DLL vs PyInline vs weave.inline

Hi,

Just as a note to your final point about pyco. I have an application
that includes tons of win32com and ctypes calls and I tried using pyco
and found it slowed things downs quite a bit. (I didn't keep details).

Thanks for the interesting thread,
	Mark

-----Original Message-----
From: ctypes-users-admin <at> lists.sourceforge.net
[mailto:ctypes-users-admin <at> lists.sourceforge.net] On Behalf Of Ray
Schumacher
Sent: Friday, July 01, 2005 3:12 PM
To: ctypes-users <at> lists.sourceforge.net
Subject: [ctypes-users] Numeric.where vs. ctypes-and-C DLL vs PyInline
vs weave.inline

In addition to the previous post, I tried PyInline and weave for
executing a C routine over a Numeric array.
PyInline has about the same calling overhead as ctypes, apparently,
about 240us for a simple C statement. It also requires some extra hoops
to digest the pointer to short I need (  void b2int(short *f, int N)
), so I didn't fully try it.

weave.inline, however, was very good, with different issues. I found I
had to patch msvccompiler.py for MS C 7.1 .NET, but that's online.

        import weave
        inline = weave.inline # for speed
(Continue reading)

Permalink | Reply | 0 comments |
headers
Thomas Heller | 5 Jul 2005 20:44
Favicon

Re: Proposal: add "mode" flag to dlopen functions

"Mike C. Fletcher" <mcfletch <at> vrplumber.com> writes:

> I've just tracked down why GLUT refuses to load on Linux machines. 
> Basically the DLL is assuming that the GL and GLU libraries will have
> been loaded with the RTLD_GLOBAL flag to allow for resolving the symbols
> in libglut.so .  At the moment ctypes always uses RTLD_LOCAL (save on
> cygwin, where RTLD_LOCAL isn't defined), so the GL symbols aren't
> available to GLUT and the GLUT load fails.
>
> I'd like to propose adding a mode flag to the CDLL init and to the
> py_dl_open function such that dlopen can receive a RTLD_GLOBAL flag when
> required.
>
> Patches below; note that this has only been tested (and that fairly
> minimally) on Linux, testing on cygwin and Win32 (and OSX) will be needed.

I'll try to add this patch before the next release.
One question though:  Do you have any thoughts about how this can be
intregrated with the decorators, or don't you use them?

(Hopefully this gets through now, I had email problems yesterday...)

Thomas

-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
(Continue reading)

Permalink | Reply | 1 comment |
headers
Thomas Heller | 5 Jul 2005 20:47
Favicon

Re: xml2struct?

"Hihn, Jason" <Jason.Hihn <at> verint.com> writes:

> '''
> If you use xml2py.py, there should be no need to specify the packing
> manually.  gccxml should be able to figure it out, and the codegenerator
> generates _pack_ attributes if needed.
> '''
>
> Nice! But I am still having issues. I am on a bug-bash so my time is
> limited to investigate this week...
>
> One thing I would like to have *please*please* is the ability to
> compensate for endians. I am reading a big endian file on a little
> endian machine. I can override and manually swap for any 4-byte int
> type, but I cannot tell the 4byte-ints from the shorts by the time it is
> in python. This is something I realized after my mailing yesterday.
>
> Pointers? (N.P.I.)

I don't think ctypes can support them.  Too much bitfiddling in already
quite complicated code.

But you may be able to override the codegeneration and let it spit out
structure definitions in a different syntax - there are some other
structure implementations out there which may support different endians
since they are based on the struct module.

One is name sstruct (or was it xstruct?) by Just v. Rossum, the other
that I know of is called ptypes by Bob Ippolito.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mike C. Fletcher | 5 Jul 2005 21:09
Favicon

Re: Re: Proposal: add "mode" flag to dlopen functions

Thomas Heller wrote:

>"Mike C. Fletcher" <mcfletch <at> vrplumber.com> writes:
>  
>
...

>>Patches below; note that this has only been tested (and that fairly
>>minimally) on Linux, testing on cygwin and Win32 (and OSX) will be needed.
>>    
>>
>
>I'll try to add this patch before the next release.
>  
>
Okay, do check on systems that don't have the flag, the __init__.py
patch will need some fiddling to make it work on those I'd guess.

>One question though:  Do you have any thoughts about how this can be
>intregrated with the decorators, or don't you use them?
>  
>
I don't use them because I'm still *trying* to target Python 2.3 for all
the code I develop.  This is AFAICS a bit of a corner case where you can
readily tell a user to write out the CDLL(...) call with the extra
parameter, but then I centralise all of this into a "platform" module
and then forget it, so it doesn't bother me to have it be a little more
complex (i.e. I'm only ever going to write it once).

Have fun,
(Continue reading)

Permalink | Reply | 0 comments |
headers
Thomas Heller | 5 Jul 2005 20:50
Favicon

Re: xml2struct?

Dave Brueck <dave <at> pythonapocrypha.com> writes:

> Thomas Heller wrote:
>> "Hihn, Jason" <Jason.Hihn <at> verint.com> writes:
>>
>>>'''
>>>_pack_ is something you specify in the ctypes structures themselves,
>>>e.g.
>>>
>>>class Foo(Structure):
>>>   _pack_ = 1
>>>   _fields_ = ...
>>>
>>>So you do this in the source code (before you'd ever run the program).
> [snip]
>> If you use xml2py.py, there should be no need to specify the packing
>> manually.  gccxml should be able to figure it out, and the codegenerator
>> generates _pack_ attributes if needed.
>
> Hmm... how does it figure out whether or not packing is needed? Is
> there a reliable way to infer that from just the code? (I always
> thought the answer was 'no' unless the code includes compiler-specific
> directives that explicitly request packing, e.g. Microsoft's #pragma
> pack)

gccxml doesn't write pragmas into the xml file, but it outputs field
sizes and alignments.  The ctypes code generator then simply tries
several possibilities for the _pack_ attribute until it finds one that
fits (or bails out with an error).  So far this works well for MSVC
header files.  With GCC it is possible to specify the alignment
(Continue reading)

Permalink | Reply | 1 comment |
headers
Dave Brueck | 5 Jul 2005 21:29

Re: Re: xml2struct?

Thomas Heller wrote:
> Dave Brueck <dave <at> pythonapocrypha.com> writes:
>>Thomas Heller wrote:
>>>If you use xml2py.py, there should be no need to specify the packing
>>>manually.  gccxml should be able to figure it out, and the codegenerator
>>>generates _pack_ attributes if needed.
>>
>>Hmm... how does it figure out whether or not packing is needed? Is
>>there a reliable way to infer that from just the code? (I always
>>thought the answer was 'no' unless the code includes compiler-specific
>>directives that explicitly request packing, e.g. Microsoft's #pragma
>>pack)
> 
> 
> gccxml doesn't write pragmas into the xml file, but it outputs field
> sizes and alignments.  The ctypes code generator then simply tries
> several possibilities for the _pack_ attribute until it finds one that
> fits (or bails out with an error).  So far this works well for MSVC
> header files.  With GCC it is possible to specify the alignment
> separately for each structure field - it may not work in these cases.

Or if the compiler+hardware for the program that created the data differs 
sufficiently from the compiler+hardware for the program that's reading the data, 
but that's an inherently messy problem anyway, so no surprises there.

Thanks for the extra info about gccxml / ctypes.

-Dave

-------------------------------------------------------
(Continue reading)

Permalink | Reply | 0 comments |

Gmane