headers
Colin J. Williams | 1 Mar 2006 02:00
Picon
Favicon

Re: subclassing ndaray

Travis Oliphant wrote:

>
>> Travis Oliphant wrote:
>>
>>> Stefan van der Walt wrote:
>>>
>>>>> The __init__ and __new__ methods are not called because they may 
>>>>> have arbitrary signatures.  Instead,  the __array_finalize__ 
>>>>> method is always called.  So, you should use that instead of 
>>>>> __init__.
>>>>>   
>>>>
>>>>
>>>>
>>> This is now true in SVN.  Previously, __array_finalize__ was not 
>>> called if the "parent" was NULL.  However, now, it is still called 
>>> with None as the value of the first argument.
>>>
>>> Thus __array_finalize__ will be called whenever 
>>> ndarray.__new__(<some subclass>,...) is called.
>>
>>
>>
>> Why this change in style from the the common Python idom of __new__, 
>> __init__, with the same signature to __new__, __array_finalize__ with 
>> possibly different signatures?
>>
>
> I don't see it as a change in style but adding a capability to the
(Continue reading)

Permalink | Reply | 0 comments |
headers
Paul F. Dubois | 1 Mar 2006 02:19

Re: Numpy and PEP 343

You're reinventing C++ expression templates, although since Python is 
dynamically typed you don't need templates. The crucial feature in C++ 
that lets it all work is that you can override the action for assignment.

a = b*c + d*e

If we could realize we were at the "equals" sign we could evaluate the 
RHS, and assign it to a.

This is not possible in Python; to make is possible would require 
slowing down regular assignment, which is perhaps a definition of bad.

a[...] = RHS

could be overridden but it is ugly and 'naive' users will forget.

a := RHS

could be added to the language with the semantics that it tries to do 
a.__assignment__(RHS) but Guido told me no long ago. (:->. Also, you 
might forget the : in :=.

a.assign(RHS)

would also work but then the original statement would produce a strange 
object with surprising results.

David M. Cooke wrote:
> Tim Hochberg <tim.hochberg <at> cox.net> writes:
>
(Continue reading)

Permalink | Reply | 29 comments |
headers
Sasha | 1 Mar 2006 02:50
Picon

Re: Numpy and PEP 343

Lazy evaluation has been part of many array languages since early days
of APL (which makes this idea almost 50 years old).  I was
entertaining an idea of bringing lazy evaluation to python myself and
concluded that there are two places where it might fit

1. At the level of python optimizer a * x +  y, for example, can be
translated into a call to a call to axpy if a, x and y are known to be
arrays.  This aproach quickly brings you to the  optional static
typing idea. <http://www.artima.com/weblogs/viewpost.jsp?thread=85551>

2. Overload arithmetic operators for ufunc objects.  This will allow
some form of tacit programming
<http://elliscave.com/APL_J/IversonAPL.htm> and you would be able to
write

f = multiply + multiply
f(x, y, z, t)

and have it evaluated without temporaries.  Both of these ideas are
from the pie-in-the-sky variety.

On 2/28/06, Paul F. Dubois <paul <at> pfdubois.com> wrote:
> You're reinventing C++ expression templates, although since Python is
> dynamically typed you don't need templates. The crucial feature in C++
> that lets it all work is that you can override the action for assignment.
>
> a = b*c + d*e
>
> If we could realize we were at the "equals" sign we could evaluate the
> RHS, and assign it to a.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Charles R Harris | 1 Mar 2006 04:40
Picon

Re: Numpy and PEP 343

On 2/28/06, Paul F. Dubois <paul <at> pfdubois.com> wrote:
> You're reinventing C++ expression templates, although since Python is

Yes indeedy, and although they might work well enough they produce the
most godawful looking assembly code I have ever looked at. The boost
ublas template library takes this approach and I regard it more as a
meta-compiler research project written in a template language than as
an array library. I think that there are two main users of arrays:
those who want quick and convenient (optimize programmer time) and
those who want super-fast execution (optimize cpu time). Because a
human can generally do a better job and knows more about the intent
than the average compiler, I think that the best bet is to provide the
tools needed to write efficient code if the programmer so desires, but
otherwise concentrate on convenience. When absolute speed is essential
it is worth budgeting programmer time to achieve it, but generally I
don't think that is the case.

Chuck

-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
Permalink | Reply | 27 comments |
headers
Travis Oliphant | 1 Mar 2006 04:53
Picon

Re: Numpy and PEP 343

Charles R Harris wrote:

>Yes indeedy, and although they might work well enough they produce the
>most godawful looking assembly code I have ever looked at. The boost
>ublas template library takes this approach and I regard it more as a
>meta-compiler research project written in a template language than as
>an array library. I think that there are two main users of arrays:
>those who want quick and convenient (optimize programmer time) and
>those who want super-fast execution (optimize cpu time). Because a
>human can generally do a better job and knows more about the intent
>than the average compiler, I think that the best bet is to provide the
>tools needed to write efficient code if the programmer so desires, but
>otherwise concentrate on convenience. When absolute speed is essential
>it is worth budgeting programmer time to achieve it, but generally I
>don't think that is the case.
>  
>
I think this is ultimately why nothing has been done except to make it 
easier and easier to write compiled code that gets called from Python.

I'm sure most have heard that ctypes will be added to Python 2.5.  This 
will make it very easy to write a C function to do what you want and 
just call it from Python.

Weave can still help with the "auto-compilation" of the specific library 
for your type.  Ultimately such code will be faster than NumPy can every 
be. 

-Travis
(Continue reading)

Permalink | Reply | 26 comments |
headers
eric jones | 1 Mar 2006 06:27

Re: Numpy and PEP 343

Travis Oliphant wrote:

>
> Weave can still help with the "auto-compilation" of the specific 
> library for your type.  Ultimately such code will be faster than NumPy 
> can every be.

Yes.  weave.blitz() can be used to do the equivalent of this lazy 
evaluation for you in many cases without much effort.  For example:

import weave
from scipy import arange

a = arange(1e7)
b = arange(1e7)
c=2.0*a+3.0*b

# or with weave
weave.blitz("c=2.0*a+3.0*b")

As Paul D. mentioned.what Tim outlined is essentially template 
expressions in C++.  blitz++ (http://www.oonumerics.org/blitz/) is a C++ 
template expressions library for array operations, and weave.blitz 
translates a Numeric expression into C++ blitz code.  For the example 
above on large arrays, you get about a factor of 4 speed up on large 
arrays. (Notice, the first time you run the example it will be much 
slower because of compile time.  Use timings from subsequent runs.)

C:\temp>weave_time.py
Expression: c=2.0*a+3.0*b
(Continue reading)

Permalink | Reply | 25 comments |
headers
Pearu Peterson | 1 Mar 2006 06:34
Favicon

Re: setting package path


On Wed, 1 Mar 2006, Stefan van der Walt wrote:

> In numpytest.py, set_package_path is provided for handling path
> changes while doing unit tests.  It reads
>
> def set_package_path(level=1):
>    """ Prepend package directory to sys.path.
>
>    set_package_path should be called from a test_file.py that
>    satisfies the following tree structure:
>
>      <somepath>/<somedir>/test_file.py
>
>    Then the first existing path name from the following list
>
>      <somepath>/build/lib.<platform>-<version>
>      <somepath>/..
>
>    is prepended to sys.path.
> ...
>
> However, the line that supposedly calculates "somepath/.." is
>
> d = os.path.dirname(os.path.dirname(os.path.abspath(testfile)))
>
>
> which calculates "somepath".  Which is wrong: the docstring, the code
> or my interpretation?
(Continue reading)

Permalink | Reply | 1 comment |
headers
Travis Oliphant | 1 Mar 2006 08:15
Picon

Re: Table like array

Michael Sorich wrote:

> Hi,
>
> I am looking for a table like array. Something like a 'data frame' 
> object to those familiar with the statistical languages R and Splus. 
> This is mainly to hold and manipulate 2D spreadsheet like data, which 
> tends to be of relatively small size (compared to what many people 
> seem to use numpy for), heterogenous, have column and row names, and 
> often contains missing data. 

You could subclass the ndarray to produce one of these fairly easily, I 
think.   The missing data item could be handled by a mask stored along 
with the array (or even in the array itself).  Or you could use a masked 
array as your core object (though I'm not sure how it handles the 
arbitrary (i.e. record-like) data-types yet).

Alternatively, and probably the easiest way to get started, you could 
just create your own table-like class and use simple 1-d arrays or 1-d 
masked arrays for each of the columns ---  This has always been a way to 
store record-like tables.

It really depends what you want the data-frames to be able to do and 
what you want them to "look-like."

> A RecArray seems potentially useful, as it allows different fields to 
> have different data types and holds the name of the field. However it 
> doesn't seem easy to manipulate the data. Or perhaps I am simply 
> having difficulty finding documentation on there features.
(Continue reading)

Permalink | Reply | 7 comments |
headers
Zachary Pincus | 1 Mar 2006 10:05
Picon
Favicon

numpy.dot gives wrong results with floats?

Hi folks,

I'm using numpy 0.95 and came across an odd error with numpy.dot and  
degenerate 2D floating-point arrays. See below for an illustration.

In [1]: import numpy
In [2]: numpy.version.version
Out[2]: '0.9.5'

In [3]: numpy.dot([[5]],[[1,1,1]]) # 2D int case OK
Out[3]: array([[5, 5, 5]])

In [4]: numpy.dot([[5.]],[[1,1,1]]) # 2D float case: what???
Out[4]: array([[  5.00000000e+000,   4.46601253e-316,    
5.42111867e-306]])

In [5]: numpy.dot([5.],[1,1,1]) # 1D float case OK
Out[5]: array([ 5.,  5.,  5.])

In [6]: numpy.dot([[[5.]]],[[[1,1,1]]]) # 3D float case OK
Out[6]: array([[[[ 5.,  5.,  5.]]]])

Zach

-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
(Continue reading)

Permalink | Reply | 5 comments |
headers
Niklas Volbers | 1 Mar 2006 12:40
Picon

Re: Re: [SciPy-user] Table like array

[[Oops, I accidentally sent my post to scipy-users, but it was intended for numpy-discussion, so here's
another attempt]]

Hey Michael,

take a look at my attempt of such a Table implementation!

The newest release 0.5.2 of my plotting project

http://developer.berlios.de/projects/sloppyplot

contains a Table class (in Sloppy.Base.dataset) which wraps a heterogeneous numpy array. The class
should be fairly self-documenting, at least I hope so. Don't get confused by the 'undolist' stuff, this is
my private undo implementation which could be easily removed from the code.

If you want a similar implementation using a list of 1-dimensional arrays, then download the previous
release 0.5.1 (which uses Numeric).

The reason I switched over to the heterogeneous approach was that it is easier to provide similar wrappers
for 2-d table like data (using a 2d heterogeneous array) and for 2-d matrix like data (using a 2d
homogeneous array). Using a list of arrays gives you some problems when you would like to access the rows,
because then you are in charge of creating a new 1-d array that represents the row, while with the
heterogeneous array you can access both the columns and the rows quite naturally.

By the way, I had first planned to subclass ndarray, but I did not know how to resize the array and still keep
the array as such persistent. This is why I wrapped the array into a class called 'Dataset' which you can
consider constant.

If you need some more help on this, feel free to ask,
(Continue reading)

Permalink | Reply | 1 comment |

Gmane