Using ctypes with embedded python

Re: What's the proper way of handling error-checking return values?

Thomas Heller <theller <at> python.net> writes:

> "Mike C. Fletcher" <mcfletch <at> rogers.com> writes:
>
>> How about an alternate proposal:
>>
>>     For the cFunction return-type protocol there shall be two entry points:
>>
>>         restype -- a ctypes instance *or* a callable value, required,
>>         default c_int
>>         errcheck -- a callable instance taking { return-value from
>>         restype, the cFunction instance, and the set of argument objects
>>         passed into the call }, and checking for errors, optional
>>
>>     The advantage here is that we don't need a different "type" of c_int
>>     to do error checking for a return value, instead, we assign a
>>     property on the function that wants the error checking.  This should
>>     be completely backward compatible, it simply introduces a new entry
>>     point.  In psuedo-code it looks like this:
>>
>>         result = TraditionalGetResult( ... )
>>         if (function.errcheck):
>>             # errcheck raises errors on failure, or can mutate result
>>             result = function.errcheck( result, function, arguments )
>>         return result
>>
>>     The use of callable values for restype is deprecated, but provided
>>     for backward compatability.  I'm not sure, is there the potential
>>     for the raw copy of memory into result type to get "result" has a
>>     failure case?  Anyway, that would be no different than the current
>>     situation if I'm reading correctly.
>>
>>     Why all the extra arguments to errcheck?  To allow for proper
>>     error-messages and to allow the handlers to be generically coded
>>     (i.e. you get a pointer to the cFunction so that you can decide
>>     which function it is in which you are checking for errors).
>>
>> In short, I'd rather see the checking code stored on the cFunction
>> object.  I actually have this functionality in the "wrapper" module in
>> OpenGL-ctypes, it's just that I don't want to have to use (the
>> comparatively heavy) wrapper module for *every* OpenGL call.
>
> This proposal looks rather good.  It lets me even implement proper error
> handling for COM method calls where the interface id is needed to call
> InterfaceSupportsErrorInfo.  I think I'll go with this.

I just wanted to start implementing Mike's suggestion when I found that
there *is* already an implementation of the protocol that *I* suggested.
From the ChangeLog:

2004-10-29  Thomas Heller  <theller <at> python.net>

	* (Message): Define a new protocol for function's restype.  If it
	is a ctypes' type, it is handled in the usual way.  But if it
	additionally has an _check_retval_ attribute, this is called with
	the result, and the return value of this call is the functions
	return value.

This has not yet been advertized, and it does neither pass the function
nor the arguments to the 'checker', but maybe it allows some experiments.
Anyway, I'll implement the new protocol although I cannot promise how
fast it will go.

Thomas

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf

Re: Using ctypes with embedded python

"Brian Wong" <BrianWong <at> AirgoNetworks.Com> writes:

> Hi,
>
>  
>
> Sorry if this question sounds naive, but I was wondering if it is
> possible to use ctypes with embedded python to make it easier to pass C
> data structures to python?  For example, could I call a python function
> from my C application and pass it the address to my C data structure,
> and in the python function have it use the ctypes from_address() method
> cast that pointer into the equivalent ctypes class for reading?  If this
> does work, this seems like a much easier way than the traditional method
> of converting the C data structure into a python object in the C code.

This is an interesting idea, but I'm not sure it's a good one.  What
comes to mind is the PyStruct_Sequence api, which seems undocumented.

But in the Python core are some modules which use this interface, and it
doesn't look comlicated. IIRC, a Struct_Sequence is a kind of tuple (so
it supports indexing), but you can also access the fields with names.

src/Modules/resource.c and src/Modules/spwdmodule.c are compact modules
using it, src/Modules/timemodule.c is a larger one.

Thomas

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf

Re: What's the proper way of handling error-checking return values?

Thomas Heller <theller <at> python.net> writes:

> Thomas Heller <theller <at> python.net> writes:
>
>> "Mike C. Fletcher" <mcfletch <at> rogers.com> writes:
>>
>>> How about an alternate proposal:
>>>
>>>     For the cFunction return-type protocol there shall be two entry points:
>>>
>>>         restype -- a ctypes instance *or* a callable value, required,
>>>         default c_int
>>>         errcheck -- a callable instance taking { return-value from
>>>         restype, the cFunction instance, and the set of argument objects
>>>         passed into the call }, and checking for errors, optional
>>>
>>>     The advantage here is that we don't need a different "type" of c_int
>>>     to do error checking for a return value, instead, we assign a
>>>     property on the function that wants the error checking.  This should
>>>     be completely backward compatible, it simply introduces a new entry
>>>     point.  In psuedo-code it looks like this:
>>>
>>>         result = TraditionalGetResult( ... )
>>>         if (function.errcheck):
>>>             # errcheck raises errors on failure, or can mutate result
>>>             result = function.errcheck( result, function, arguments )
>>>         return result
>>>
>>>     The use of callable values for restype is deprecated, but provided
>>>     for backward compatability.  I'm not sure, is there the potential
>>>     for the raw copy of memory into result type to get "result" has a
>>>     failure case?  Anyway, that would be no different than the current
>>>     situation if I'm reading correctly.
>>>
>>>     Why all the extra arguments to errcheck?  To allow for proper
>>>     error-messages and to allow the handlers to be generically coded
>>>     (i.e. you get a pointer to the cFunction so that you can decide
>>>     which function it is in which you are checking for errors).
>>>
>>> In short, I'd rather see the checking code stored on the cFunction
>>> object.  I actually have this functionality in the "wrapper" module in
>>> OpenGL-ctypes, it's just that I don't want to have to use (the
>>> comparatively heavy) wrapper module for *every* OpenGL call.
>>
>> This proposal looks rather good.  It lets me even implement proper error
>> handling for COM method calls where the interface id is needed to call
>> InterfaceSupportsErrorInfo.  I think I'll go with this.
>
> I just wanted to start implementing Mike's suggestion when I found that
> there *is* already an implementation of the protocol that *I* suggested.
> From the ChangeLog:
>
> 2004-10-29  Thomas Heller  <theller <at> python.net>
>
> 	* (Message): Define a new protocol for function's restype.  If it
> 	is a ctypes' type, it is handled in the usual way.  But if it
> 	additionally has an _check_retval_ attribute, this is called with
> 	the result, and the return value of this call is the functions
> 	return value.
>
> This has not yet been advertized, and it does neither pass the function
> nor the arguments to the 'checker', but maybe it allows some experiments.
> Anyway, I'll implement the new protocol although I cannot promise how
> fast it will go.

I've repaired the _check_retval_ protocol in CVS (it worked only for
simple restypes, like c_int or c_void_p and subclasses, and leaked
references in error situations), but I would discourage using it.  I
will probably change it to be more compatible with the new 'errcheck'
protocol.

I've also made a first implementation of Mike's 'errcheck' protocol,
although the complete details are not yet fleshed out (not about the
protocol itself, but its further integration in ctypes).

I have made one extension to Mike's proposal.  The errcheck function
normally receives (result, function, arguments) results, but should be
able to accept more arguments.  So, the signature should be

  def errcheck(result, arguments, function, *others):
      ...

Thomas

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf

Re: What's the proper way of handling error-checking return values?

Thomas Heller wrote:
...

>>>>        if (function.errcheck):
>>>>            # errcheck raises errors on failure, or can mutate result
>>>>            result = function.errcheck( result, function, arguments )
>>>>        return result
>>>>        
>>>>
...

>I have made one extension to Mike's proposal.  The errcheck function
>normally receives (result, function, arguments) results, but should be
>able to accept more arguments.  So, the signature should be
>
>  def errcheck(result, arguments, function, *others):
>      ...
>  
>
Looks fine, but I believe you have implemented (as originally proposed, 
minus the extra-arguments):

    def errcheck(result, function, arguments, *others):
       ...

I've altered OpenGL-ctypes to use this protocol throughout.

Thanks!
Mike

--

-- 
________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf

RE: Using ctypes with embedded python

Hi,

 

(sorry if this comes in as HTML mail - writing from Outlook thin client).

 

If I understand the question then I think I have been doing this extensively.

I wanted to extend an application that allowed DLL plugins - but I wanted to extend it in python.

 

So I did the following:

 * Wrote a C wrapper to convert the C++ Plugin Interface to C functions and exposed them (C++ --> C because of Pyrex and I exported those functions from the DLL so I could call them from Python)

 * Wrote quite a simply Pyrex wrapper that exposed the functions that the Plugin could call, these just forwarded calls to a python module that was loaded.

 * Used Ctypes to then convert the recieved data, and call the exposed plugin interface functions.

 

Works well for me Most of the types I am recieve from the application I leave as pointers (which round trip back to the C plugin without needing conversion) but the other items I convert using ctypes.

 

I am not sure if that is what you were looking for or it was more specifically data conversion that you were looking at.

 

Thanks

   Mark

-----Original Message-----
From: ctypes-users-admin <at> lists.sourceforge.net on behalf of Thomas Heller
Sent: Fri 02-Sep-05 14:19
To: Brian Wong
Cc: ctypes-users <at> lists.sourceforge.net
Subject: Re: [ctypes-users] Using ctypes with embedded python

"Brian Wong" <BrianWong <at> AirgoNetworks.Com> writes:

> Hi,
>
> 
>
> Sorry if this question sounds naive, but I was wondering if it is
> possible to use ctypes with embedded python to make it easier to pass C
> data structures to python?  For example, could I call a python function
> from my C application and pass it the address to my C data structure,
> and in the python function have it use the ctypes from_address() method
> cast that pointer into the equivalent ctypes class for reading?  If this
> does work, this seems like a much easier way than the traditional method
> of converting the C data structure into a python object in the C code.

This is an interesting idea, but I'm not sure it's a good one.  What
comes to mind is the PyStruct_Sequence api, which seems undocumented.

But in the Python core are some modules which use this interface, and it
doesn't look comlicated. IIRC, a Struct_Sequence is a kind of tuple (so
it supports indexing), but you can also access the fields with names.

src/Modules/resource.c and src/Modules/spwdmodule.c are compact modules
using it, src/Modules/timemodule.c is a larger one.

Thomas





-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
ctypes-users mailing list
ctypes-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ctypes-users

Callback return values and exceptions

Excuse me if this has been discussed before.

I am wondering what the official ctypes behaviour is for a python callback that 
raises an exception. Testing shows that the exception is propagate back. What has 
me curious though is what happens when the callback has a return value, 
particularly of type c_uint. What value does the callback return to its C caller when 
an exception is raised?

Lenard Lindstrom
<len-l <at> telus.net>

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf

Re: Callback return values and exceptions

len-l <at> telus.net writes:

> Excuse me if this has been discussed before.

No it hasn't, IIRC.

> I am wondering what the official ctypes behaviour is for a python
> callback that raises an exception. Testing shows that the exception is
> propagate back. What has me curious though is what happens when the
> callback has a return value, particularly of type c_uint. What value
> does the callback return to its C caller when an exception is raised?

I *think* that exceptions are not propgated any longer through the
callback function.  That should be true for 0.9.6 (although I'm not 100%
sure about that), but is true for current CVS.  PyErr_Print() is called,
which gives the correct behaviour (imo).  For example, if the callback
raises SystemExit, the script ends.

The return value is 'not specified', whatever is on the stack somewhere.
Maybe it would be better to return 0 or NULL, but if you want the
callback to return a certain value you must catch exceptions in the
callback yourself.

Here is a session that demonstrates the current behaviour:

>>> from ctypes import *
>>>
>>> proto = WINFUNCTYPE(c_uint)
>>>
>>> def callback(*args):
...     print "callback", args
...     raise ValueError
...
>>> func = proto(callback)
>>>
>>> print "result %s" % func()
callback ()
Traceback (most recent call last):
  File "C:\TSS5\components\_Pythonlib\source/callbacks.c", line 233, in 'calling callback function'
    result = PyObject_CallObject(callable, arglist);
  File "<stdin>", line 3, in callback
ValueError
result 11404196
>>>
>>>
>>> def callback(*args):
...     print "callback", args
...     raise SystemExit
...
>>> func = proto(callback)
>>>
>>> print "result %s" % func()
callback ()

C:\sf\ctypes>

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf

Problem on .h definitions

Hi list,
I have a problem with this code. My issue is that into this code (I
report it with the real appear order) the FT_MemoryRec_ structure need
the FT_Alloc_Func, but the FT_Alloc_Func need the FT_MemoryRec_ struct!
So I don't know how to translate it into python code. Can someone help me?
Thanks,
Michele

typedef struct FT_MemoryRec_*  FT_Memory;

typedef void*
   (*FT_Alloc_Func)( FT_Memory  memory,
                     long       size );

struct  FT_MemoryRec_ {
     void*            user;
     FT_Alloc_Func    alloc;
     FT_Free_Func     free;
     FT_Realloc_Func  realloc;
};

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf

COM Python trouble

Hello ctypes-users,

  I have Python 2.4 and install ctypes for it.
  I want start bho_skel.py  example and receive next error from IE6:
    "
    Traceback   (most recent call last):
    File "D:\Python24\Lib\site-packages\ctypes\com\com\server.py"
    DllClassObject
    cls = inproc_find_class(clssid)
    File "D:\Python24\Lib\site-packages\ctypes\com\com\server.py"
    inproc_find_class
    if not pathdir in sys.path:
     UnicodeDecodeError: 'ascii' codec can't decode byte  0xc5 in
     position 26: ordinal not in range  (128)
     "

     Please advice.

--

-- 
Best regards,
 ulik                          mailto:ulik <at> happyagency.com

-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf