Developers of Scientific Python

David Cournapeau | 2 Jul 2007 06:45

[scikits] setuptools, tests and subpackages

Hi,

    I would like to know if anyone knowledgable about setuptools knows 
the best way to tell setuptools how to call unittests in a packages 
containing subpackages ? In a scipy package, having a test function in 
__init__ is enough to get the unittests added to the whole scipy test 
suite, but with setuptools, how do I do that ?

    cheers,

    David

headers

Robert Kern | 2 Jul 2007 18:21

Re: [scikits] setuptools, tests and subpackages

David Cournapeau wrote:
> Hi,
> 
>     I would like to know if anyone knowledgable about setuptools knows 
> the best way to tell setuptools how to call unittests in a packages 
> containing subpackages ? In a scipy package, having a test function in 
> __init__ is enough to get the unittests added to the whole scipy test 
> suite, but with setuptools, how do I do that ?

http://peak.telecommunity.com/DevCenter/setuptools#test-build-package-and-run-a-unittest-suite

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

headers

Niels L. Ellegaard | 2 Jul 2007 21:31

Re: equivalent to online Octave calculator?

dmitrey <openopt <at> ukr.net> writes:
> has numpy/scipy project something equivalent to online Octave calculator?
> http://www.online-utility.org/math/math_calculator.jsp

I found web a webpage that allows you to try of python online, but it
doesn't provide scipy.

http://www.mired.org/home/mwm/try_python/

                   Niels

headers

David Cournapeau | 3 Jul 2007 08:16

Implementing a distance matrix between two sets of vectors concept

Hi,

    for my machine learning toolbox, I need the concept of distance 
matrix, that is for two sets of vectors v and u (N u and M v), of 
dimension d, I want to compute the matrix D such as d(i,j) = 
distance(v_i, u_j). This is easy to do in numpy, but for big datasets, 
this becomes difficult without a significance loss of efficiency or big 
memory consumption.
    So I am thinking about implementing it in C. I think the overall 
concept is useful for other people, so before implementing something, I 
was wondering if other people would need/use it, and what would they need:
    - several distance (Euclidian, Mahalanobis, etc...), which would be 
a separate object to handle different sets of parameters.
    - C Api ?
    - datatypes ? Layout ? Contiguity ?
    - handling Nan ?

     cheers,

    David

headers

Bill Baxter | 3 Jul 2007 11:36

Re: Implementing a distance matrix between two sets of vectors concept

I would use it.

I only need Euclidean distance,
Python API only is ok.
Data-types: float and double would do it for me. Double only if it's too much effort to do both.
Order -- all combos of F and C both would be nice, but not critical
Strides -- with strides better than without, but not critical
nan -- don't need it.

--bb

On 7/3/07, David Cournapeau < david <at> ar.media.kyoto-u.ac.jp> wrote:

Hi,

    for my machine learning toolbox, I need the concept of distance
matrix, that is for two sets of vectors v and u (N u and M v), of
dimension d, I want to compute the matrix D such as d(i,j) =
distance(v_i, u_j). This is easy to do in numpy, but for big datasets,
this becomes difficult without a significance loss of efficiency or big
memory consumption.
    So I am thinking about implementing it in C. I think the overall
concept is useful for other people, so before implementing something, I
was wondering if other people would need/use it, and what would they need:
    - several distance (Euclidian, Mahalanobis, etc...), which would be
a separate object to handle different sets of parameters.
    - C Api ?
    - datatypes ? Layout ? Contiguity ?
    - handling Nan ?

     cheers,

    David
_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

headers

Christopher Hanley | 3 Jul 2007 18:25

ndimage problems

Hi,

We have found two problems with ndimage.  I have filed a ticket #455 on 
the scipy trac page.  The first problem can be seen with this example:

 > import numpy as n
 > from scipy import ndimage as nd
 > a = n.ones((10,5),dtype=n.float32) * 12.3
 > x = nd.rotate(a,90.0)
 > x
Out[17]:
array([[ 12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019,  12.30000019,  12.30000019,
         0.        ,   0.        ],
      [ 12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019],
      [ 12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019],
      [ 12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019],
      [ 12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019,  12.30000019,  12.30000019,
        12.30000019,  12.30000019]], dtype=float32)
}}}

Notice that the last two entries of the first row are now 0.

The second problem has to do with the reversing of byte order if you 
have big-endian data on a little endian machine.  Please see the example 
below:

 >>> a = N.ones ((2,3), dtype=N.float32) * 12.3
 >>> a = a.byteswap()
 >>> a.dtype = a.dtype.newbyteorder (">")
 >>> print a
[[ 12.30000019 12.30000019 12.30000019]
[ 12.30000019 12.30000019 12.30000019]]
 >>> print ndimage.rotate (a, 90.)
[[ 0.00000000e+00 -4.28378144e+08]
[ 0.00000000e+00 -4.28378144e+08]
[ -4.28378144e+08 -4.28378144e+08]]

I have taken a look at the ndimage python code and cannot find any 
explicit calls to byteswap.  I'm guessing something is funny in one of 
the c-api calls.  I haven't been able to track it down yet.

Chris

headers

Peter Skomoroch | 3 Jul 2007 20:10

Re: Implementing a distance matrix between two sets of vectors concept

I've rolled my own in the past. If the vectors are really large and you are holding a collection of them, you probably want to use a sparse matrix data structure in either numpy or C.

On 7/3/07, Bill Baxter <wbaxter <at> gmail.com> wrote:

I would use it.

I only need Euclidean distance,
Python API only is ok.
Data-types: float and double would do it for me. Double only if it's too much effort to do both.
Order -- all combos of F and C both would be nice, but not critical
Strides -- with strides better than without, but not critical
nan -- don't need it.

--bb

On 7/3/07, David Cournapeau < david <at> ar.media.kyoto-u.ac.jp> wrote:
Hi,

    for my machine learning toolbox, I need the concept of distance
matrix, that is for two sets of vectors v and u (N u and M v), of
dimension d, I want to compute the matrix D such as d(i,j) =
distance(v_i, u_j). This is easy to do in numpy, but for big datasets,
this becomes difficult without a significance loss of efficiency or big
memory consumption.
    So I am thinking about implementing it in C. I think the overall
concept is useful for other people, so before implementing something, I
was wondering if other people would need/use it, and what would they need:
    - several distance (Euclidian, Mahalanobis, etc...), which would be
a separate object to handle different sets of parameters.
    - C Api ?
    - datatypes ? Layout ? Contiguity ?
    - handling Nan ?

     cheers,

    David
_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

--
Peter N. Skomoroch
peter.skomoroch <at> gmail.com
http://www.datawrangling.com

_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

headers

Lino Mastrodomenico | 4 Jul 2007 01:22

PEP 368: Standard image protocol and class

[Sorry for the cross-posting, but I think this may be relevant for
both NumPy and ndimage.]

Hello everyone,

I have submitted to the Python core developers a new PEP (Python
Enhancement Proposal):

   http://www.python.org/dev/peps/pep-0368/

It proposes two things:

 * the creation of a standard image protocol/interface that can be
hopefully implemented interoperably by most Python libraries that
manipulate images;

 * the addition to the Python standard library of a basic
implementation of the new protocol.

The new image protocol is heavily inspired by a subset of the NumPy
array interface, with a few image-specific additions and changes (e.g.
the "size" attribute of an image is a tuple (width, height)).

Of course it would be wonderful if these new image objects could
interoperate out-of-the-box with numpy arrays and ndimage functions.
There is another proposal that would be very useful for that, PEP 3118
by Travis Oliphant and Carl Banks:

    http://www.python.org/dev/peps/pep-3118/

The image PEP (368) currently lists only modes based on uint8/16/32
numbers, but the final version will probably also include modes based
on float32 and float16 (converted in software to/from float32/64 when
necessary).

A discussion about it is currently going on in the python-3000 mailing list:

<http://mail.python.org/pipermail/python-3000/2007-July/thread.html#8648>

Any suggestion, comment or criticism from the NumPy/SciPy people would
be very useful, but IMHO keeping the discussion only on the
python-3000 ML may be a good idea, to avoid duplicating answers on
different mailing lists.

Thanks in advance.

--

-- 
Lino Mastrodomenico
E-mail: l.mastrodomenico <at> gmail.com

headers

Bill Baxter | 4 Jul 2007 03:24

Re: PEP 368: Standard image protocol and class

I'm not subscribed to the main Python list, so I'll just ask here.

It looks like the protocol doesn't support any floating point image formats, judging from the big table of formats in the PEP. These are becoming more important these days in computer graphics as a way to pass around high dynamic range images. OpenEXR is the main example of such a format: http://www.openexr.com/. I think a PEP that aims to be a generic image protocol should support at least 32 bit floats if not 64-bit doubles and 16 bit "Half"s used by some GPUs (and supported by the OpenEXR format).

---bb

On 7/4/07, Lino Mastrodomenico <l.mastrodomenico <at> gmail.com> wrote:

[Sorry for the cross-posting, but I think this may be relevant for
both NumPy and ndimage.]

Hello everyone,

I have submitted to the Python core developers a new PEP (Python
Enhancement Proposal):

http://www.python.org/dev/peps/pep-0368/

It proposes two things:

* the creation of a standard image protocol/interface that can be
hopefully implemented interoperably by most Python libraries that
manipulate images;

* the addition to the Python standard library of a basic
implementation of the new protocol.

The new image protocol is heavily inspired by a subset of the NumPy
array interface, with a few image-specific additions and changes ( e.g.
the "size" attribute of an image is a tuple (width, height)).

Of course it would be wonderful if these new image objects could
interoperate out-of-the-box with numpy arrays and ndimage functions.
There is another proposal that would be very useful for that, PEP 3118
by Travis Oliphant and Carl Banks:

http://www.python.org/dev/peps/pep-3118/

The image PEP (368) currently lists only modes based on uint8/16/32
numbers, but the final version will probably also include modes based
on float32 and float16 (converted in software to/from float32/64 when
necessary).

A discussion about it is currently going on in the python-3000 mailing list:

<http://mail.python.org/pipermail/python-3000/2007-July/thread.html#8648 >

Any suggestion, comment or criticism from the NumPy/SciPy people would
be very useful, but IMHO keeping the discussion only on the
python-3000 ML may be a good idea, to avoid duplicating answers on
different mailing lists.

Thanks in advance.

--
Lino Mastrodomenico
E-mail: l.mastrodomenico <at> gmail.com
_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

headers

David Cournapeau | 4 Jul 2007 09:52

Re: Implementing a distance matrix between two sets of vectors concept

Peter Skomoroch wrote:
> I've rolled my own in the past.  If the vectors are really large and 
> you are holding a collection of them, you probably want to use a 
> sparse matrix data structure in either numpy or C.
Mmm, not sure to understand what you mean. The problem is that you have 
{u_1, ... , u_N} and {v_1, ..., v_M} vectors, and you want the distance 
for any possible combination {u_i, v_j}, which is a real (eg the actual 
size of the matrix in memory does not depends on the dimension of the 
data, only on N and M). I don't see how sparsity can help help here ?

David

75	May 2013
197	April 2013
112	March 2013
147	February 2013
106	January 2013
68	December 2012
50	November 2012
56	October 2012
74	September 2012
68	August 2012
137	July 2012
89	June 2012
205	May 2012
88	April 2012
148	March 2012
162	February 2012
196	January 2012
65	December 2011
44	November 2011
85	October 2011
122	September 2011
95	August 2011
33	July 2011
67	June 2011
82	May 2011
61	April 2011
62	March 2011
128	February 2011
99	January 2011
52	December 2010
123	November 2010
62	October 2010
92	September 2010
85	August 2010
335	July 2010
613	June 2010
207	May 2010
194	April 2010
101	March 2010
202	February 2010
113	January 2010
201	December 2009
178	November 2009
301	October 2009
166	September 2009
328	August 2009
162	July 2009
215	June 2009
183	May 2009
177	April 2009
251	March 2009
523	February 2009
227	January 2009
306	December 2008
417	November 2008
174	October 2008
116	September 2008
158	August 2008
130	July 2008
213	June 2008
160	May 2008
333	April 2008
170	March 2008
145	February 2008
200	January 2008
152	December 2007
42	November 2007
116	October 2007
124	September 2007
126	August 2007
192	July 2007
171	June 2007
180	May 2007
197	April 2007
130	March 2007
66	February 2007
116	January 2007
181	December 2006
51	November 2006
151	October 2006
64	September 2006
45	August 2006
71	July 2006
85	June 2006
45	May 2006
194	April 2006
230	March 2006
159	February 2006
606	January 2006
358	December 2005
477	November 2005
536	October 2005
118	September 2005
51	August 2005
45	July 2005
38	June 2005
51	May 2005
26	April 2005
52	March 2005
55	February 2005
66	January 2005
47	December 2004
254	November 2004
143	October 2004
94	September 2004
21	August 2004
12	July 2004
20	June 2004
28	May 2004
50	April 2004
43	March 2004
9	February 2004
16	January 2004
22	December 2003
42	November 2003
77	October 2003
46	September 2003
22	August 2003
14	July 2003
10	June 2003
48	May 2003
23	April 2003
99	March 2003
39	February 2003
34	January 2003
56	December 2002
64	November 2002
89	October 2002
95	September 2002
76	August 2002
16	July 2002
24	June 2002
31	May 2002
1	January 2000

Mon	Tue	Wed	Thu	Fri	Sat	Sun

						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28	29
30	31