Discussion of Numerical Python

Robert Kern | 1 Jun 2011 02:31

Re: what does "in" do with numpy arrays?

On Tue, May 31, 2011 at 11:25, Christopher Barker <Chris.Barker <at> noaa.gov> wrote:
> Hi folks,
>
> I've re-titled this thread, as it's about a new question, now:
>
> What does:
>
> something in a_numpy_array
>
> mean? i.e. how has __contains__ been defined?
>
> A couple of us have played with it, and can't make sense of it:
>
>> In [24]: a
>> Out[24]: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
>>
>> In [25]: 3 in a
>> Out[25]: True
>>
>> So the simple case works just like a list. But what If I look for an array in another array?
>
>> In [26]: b
>> Out[26]: array([3, 6, 4])
>>
>> In [27]: b in a
>> Out[27]: False
>>
>> OK, so the full b array is not in a, and it doesn't "vectorize" it,
>> either. But:
>>

(Continue reading)

headers

Charles R Harris | 1 Jun 2011 03:08

New functions.

Hi All,

I've been contemplating new functions that could be added to numpy and thought I'd run them by folks to see if there is any interest.

1) Modified sort/argsort functions that return the maximum k values.
This is easy to do with heapsort and almost as easy with mergesort.

2) Ufunc fadd (nanadd?) Treats nan as zero in addition. Should make a faster version of nansum possible.

3) Fast medians.

Chuck

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

headers

Robert Kern | 1 Jun 2011 03:15

Re: New functions.

On Tue, May 31, 2011 at 20:08, Charles R Harris
<charlesr.harris <at> gmail.com> wrote:
> Hi All,
>
> I've been contemplating new functions that could be added to numpy and
> thought I'd run them by folks to see if there is any interest.
>
> 1) Modified sort/argsort functions that return the maximum k values.
>     This is easy to do with heapsort and almost as easy with mergesort.
>
> 2) Ufunc fadd (nanadd?) Treats nan as zero in addition. Should make a faster
> version of nansum possible.
>
> 3) Fast medians.

+3

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

headers

Warren Weckesser | 1 Jun 2011 03:18

Re: New functions.

On Tue, May 31, 2011 at 8:08 PM, Charles R Harris <charlesr.harris <at> gmail.com> wrote:

Hi All,

I've been contemplating new functions that could be added to numpy and thought I'd run them by folks to see if there is any interest.

1) Modified sort/argsort functions that return the maximum k values.
This is easy to do with heapsort and almost as easy with mergesort.

While you're at, how about a function that finds both the max and min in one pass? (Mentioned previously in this thread: http://mail.scipy.org/pipermail/numpy-discussion/2010-June/051072.html)

2) Ufunc fadd (nanadd?) Treats nan as zero in addition. Should make a faster version of nansum possible.

3) Fast medians.

Chuck

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

headers

Charles R Harris | 1 Jun 2011 03:26

Re: New functions.

On Tue, May 31, 2011 at 7:18 PM, Warren Weckesser <warren.weckesser <at> enthought.com> wrote:

On Tue, May 31, 2011 at 8:08 PM, Charles R Harris <charlesr.harris <at> gmail.com> wrote:
Hi All,

I've been contemplating new functions that could be added to numpy and thought I'd run them by folks to see if there is any interest.

1) Modified sort/argsort functions that return the maximum k values.
This is easy to do with heapsort and almost as easy with mergesort.

While you're at, how about a function that finds both the max and min in one pass? (Mentioned previously in this thread: http://mail.scipy.org/pipermail/numpy-discussion/2010-June/051072.html)

What should it be called? minmax?

This also brings suggests a function that returns both mean and standard deviation, something that could also be implemented using a more stable algorithm than the current one.

2) Ufunc fadd (nanadd?) Treats nan as zero in addition. Should make a faster version of nansum possible.

3) Fast medians.

Other suggestions are welcome. Most of these are of the low hanging fruit variety and shouldn't be too much work.

Chuck

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

headers

Benjamin Root | 1 Jun 2011 03:31

Re: New functions.

On Tue, May 31, 2011 at 8:18 PM, Warren Weckesser <warren.weckesser <at> enthought.com> wrote:

On Tue, May 31, 2011 at 8:08 PM, Charles R Harris <charlesr.harris <at> gmail.com> wrote:
Hi All,

I've been contemplating new functions that could be added to numpy and thought I'd run them by folks to see if there is any interest.

1) Modified sort/argsort functions that return the maximum k values.
This is easy to do with heapsort and almost as easy with mergesort.

While you're at, how about a function that finds both the max and min in one pass? (Mentioned previously in this thread: http://mail.scipy.org/pipermail/numpy-discussion/2010-June/051072.html)

+1 from myself and probably just about anybody in matplotlib. If both the maxs and mins are searched during the same run through an array, I would imagine that would result in a noticeable speedup with automatic range finding.

Ben Root

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

headers

David | 1 Jun 2011 03:33

Re: New functions.

On 06/01/2011 10:08 AM, Charles R Harris wrote:
> Hi All,
>
> I've been contemplating new functions that could be added to numpy and
> thought I'd run them by folks to see if there is any interest.
>
> 1) Modified sort/argsort functions that return the maximum k values.
>      This is easy to do with heapsort and almost as easy with mergesort.
>
> 2) Ufunc fadd (nanadd?) Treats nan as zero in addition. Should make a
> faster version of nansum possible.
>
> 3) Fast medians.

+1 for fast median as well, and more generally fast "linear" (O(kN)) 
order statistics would be nice.

cheers,

David

headers

Charles R Harris | 1 Jun 2011 03:34

Re: New functions.

On Tue, May 31, 2011 at 7:33 PM, David <david <at> silveregg.co.jp> wrote:

On 06/01/2011 10:08 AM, Charles R Harris wrote:
> Hi All,
>
> I've been contemplating new functions that could be added to numpy and
> thought I'd run them by folks to see if there is any interest.
>
> 1) Modified sort/argsort functions that return the maximum k values.
> This is easy to do with heapsort and almost as easy with mergesort.
>
> 2) Ufunc fadd (nanadd?) Treats nan as zero in addition. Should make a
> faster version of nansum possible.
>
> 3) Fast medians.

+1 for fast median as well, and more generally fast "linear" (O(kN))
order statistics would be nice.

OK, noob question. What are order statistics?

Chuck

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

headers

Skipper Seabold | 1 Jun 2011 03:36

Re: New functions.

On Tue, May 31, 2011 at 9:31 PM, Benjamin Root <ben.root <at> ou.edu> wrote:
>
>
> On Tue, May 31, 2011 at 8:18 PM, Warren Weckesser
> <warren.weckesser <at> enthought.com> wrote:
>>
>>
>> On Tue, May 31, 2011 at 8:08 PM, Charles R Harris
>> <charlesr.harris <at> gmail.com> wrote:
>>>
>>> Hi All,
>>>
>>> I've been contemplating new functions that could be added to numpy and
>>> thought I'd run them by folks to see if there is any interest.
>>>
>>> 1) Modified sort/argsort functions that return the maximum k values.
>>>     This is easy to do with heapsort and almost as easy with mergesort.
>>>
>>
>>
>> While you're at, how about a function that finds both the max and min in
>> one pass?  (Mentioned previously in this thread:
>> http://mail.scipy.org/pipermail/numpy-discussion/2010-June/051072.html)
>>
>>
>
> +1 from myself and probably just about anybody in matplotlib.  If both the
> maxs and mins are searched during the same run through an array, I would
> imagine that would result in a noticeable speedup with automatic range
> finding.
>

I don't know if it's one pass off the top of my head, but I've used
percentile for interpercentile ranges.

[docs]
[1]: X = np.random.random(1000)

[docs]
[2]: np.percentile(X,[0,100])
[2]: [0.00016535235312509222, 0.99961513543316571]

[docs]
[3]: X.min(),X.max()
[3]: (0.00016535235312509222, 0.99961513543316571)

Skipper

headers

David | 1 Jun 2011 03:39

Re: New functions.

On 06/01/2011 10:34 AM, Charles R Harris wrote:
>
>
> On Tue, May 31, 2011 at 7:33 PM, David <david <at> silveregg.co.jp
> <mailto:david <at> silveregg.co.jp>> wrote:
>
>     On 06/01/2011 10:08 AM, Charles R Harris wrote:
>      > Hi All,
>      >
>      > I've been contemplating new functions that could be added to
>     numpy and
>      > thought I'd run them by folks to see if there is any interest.
>      >
>      > 1) Modified sort/argsort functions that return the maximum k values.
>      >      This is easy to do with heapsort and almost as easy with
>     mergesort.
>      >
>      > 2) Ufunc fadd (nanadd?) Treats nan as zero in addition. Should make a
>      > faster version of nansum possible.
>      >
>      > 3) Fast medians.
>
>     +1 for fast median as well, and more generally fast "linear" (O(kN))
>     order statistics would be nice.
>
>
> OK, noob question. What are order statistics?

In statistics, order statistics are statistics based on sorted samples, 
median, min and max being the most common:

http://en.wikipedia.org/wiki/Order_statistic

Concretely here, I meant a fast way to compute any rank of a given data 
set, e.g. with the select algorithm. I wanted to do that for some time, 
but never took the time for it,

David

255	June 2013
318	May 2013
408	April 2013
346	March 2013
332	February 2013
395	January 2013
283	December 2012
311	November 2012
236	October 2012
296	September 2012
223	August 2012
438	July 2012
518	June 2012
565	May 2012
475	April 2012
455	March 2012
983	February 2012
366	January 2012
397	December 2011
259	November 2011
475	October 2011
280	September 2011
413	August 2011
536	July 2011
833	June 2011
369	May 2011
340	April 2011
729	March 2011
328	February 2011
359	January 2011
259	December 2010
463	November 2010
623	October 2010
503	September 2010
488	August 2010
787	July 2010
576	June 2010
440	May 2010
502	April 2010
641	March 2010
848	February 2010
441	January 2010
655	December 2009
735	November 2009
497	October 2009
911	September 2009
608	August 2009
594	July 2009
842	June 2009
590	May 2009
665	April 2009
897	March 2009
681	February 2009
587	January 2009
504	December 2008
532	November 2008
557	October 2008
778	September 2008
758	August 2008
1019	July 2008
620	June 2008
1244	May 2008
1110	April 2008
620	March 2008
551	February 2008
672	January 2008
317	December 2007
398	November 2007
391	October 2007
293	September 2007
319	August 2007
351	July 2007
282	June 2007
504	May 2007
617	April 2007
720	March 2007
435	February 2007
594	January 2007
361	December 2006
500	November 2006
873	October 2006
530	September 2006
636	August 2006
803	July 2006
664	June 2006
436	May 2006
543	April 2006
584	March 2006
673	February 2006
439	January 2006
61	December 2005
118	November 2005
145	October 2005
65	September 2005
56	August 2005
120	July 2005
58	June 2005
78	May 2005
205	April 2005
295	March 2005
318	February 2005
185	January 2005
60	December 2004
55	November 2004
131	October 2004
75	September 2004
84	August 2004
150	July 2004
149	June 2004
72	May 2004
49	April 2004
70	March 2004
75	February 2004
110	January 2004
71	December 2003
83	November 2003
61	October 2003
107	September 2003
24	August 2003
48	July 2003
12	June 2003
76	May 2003
6	April 2003
42	March 2003
72	February 2003
110	January 2003
104	December 2002
23	November 2002
36	October 2002
59	September 2002
34	August 2002
36	July 2002
123	June 2002
34	May 2002
62	April 2002
126	March 2002
37	February 2002
72	January 2002
54	December 2001
81	November 2001
33	October 2001
56	September 2001
92	August 2001
52	July 2001
33	June 2001
50	May 2001
67	April 2001
42	March 2001
54	February 2001
56	January 2001
24	December 2000
32	November 2000
60	October 2000
44	September 2000
16	August 2000
16	July 2000
28	June 2000
37	May 2000
28	April 2000
48	March 2000
49	February 2000
8	January 2000

Mon	Tue	Wed	Thu	Fri	Sat	Sun

		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30