Re: Call for testing: full blas/lapack builds of numpy on windows 64 bits

On Sat, Feb 28, 2009 at 12:57 PM, David Cournapeau <cournape <at> gmail.com> wrote:
> On Sat, Feb 28, 2009 at 7:01 AM, Bruce Southey <bsouthey <at> gmail.com> wrote:
>
>> Hi,
>> After much frustration and disabling my antivirus software (McAfee), I
>> was able to get the 'import numpy' to work and tests to pass as
>> expected. Of course after turning it back on and restarting the IDLE gui
>> a few times, the 'import numpy' crashes. This also happens on the
>> command line.
>
> Hm, let's assume this is a problem of the anti virus software, and not
> from mingw :) One way to test this would be to see if a binary built
> with VS 2008 would cause the same trouble.

Or just a Vista bug.

A possible option could be using wine64
(http://wiki.winehq.org/Wine64) . But is probably more work and if
even if it did work it may not be informative.

>>
>> This is clearly related to the antivirus software because enabling or
>> disabling the 'on-access scan' is usually sufficient to failure import
>> failure or success, respectively. Perhaps some thing to do with MinGW on
>> Vista?
>
> Do you have this problem only with numpy ? I noticed those kind of
> crashes (right at import time) on my server 2008, but I really could
> not understand what was wrong - it crashes half of the time, and when
> it does not, it works perfectly (I could run the test suite 10 times

Re: Call for testing: full blas/lapack builds of numpy on windows 64 bits

On Sun, Mar 1, 2009 at 12:26 PM, Bruce Southey <bsouthey <at> gmail.com> wrote:

> Or just a Vista bug.
>
> A possible option could be using wine64
> (http://wiki.winehq.org/Wine64) . But is probably more work and if
> even if it did work it may not be informative.

Win64 is not yet usable AFAIK - it was only a few weeks / months ago
they were capable of running a hello world I believe. Numpy is notably
more complicated than a hello world :) One problem is gcc - gcc 4.4
will be the first release to properly support win64 (I don't claim to
understand all the details, though).

> I virtually avoid windows for work so I can not answer your question.
> I knew about the issues from another person trying to compile code
> using WinGW on another almost identical 64 bit Vista system. Even the
> related thread seemed to solutions that only worked for some people
> and I do not know if a solution was found in that case.

Which related thread ?

> Anyhow, I do agree that having Python 2.6 support is more important
> than running the anti-virus software.

I am afraid that if it crashes from time to time, people will complain
to us, not to the AV software. Hopefully, once gdb runs reliably on
that platform, we will be able to diagnose the problem a bit better.

David

Help on subclassing numpy.ma: __array_wrap__

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: Help on subclassing numpy.ma: __array_wrap__

On Sun, Mar 1, 2009 at 10:37 AM, Kevin Dunn <kgdunn <at> gmail.com> wrote:
> Hi everyone,
>
> I'm subclassing Numpy's MaskedArray to create a data class that handles
> missing data, but adds some extra info I need to carrry around. However I've
> been having problems keeping this extra info attached to the subclass
> instances after performing operations on them.
>
> The bare-bones script that I've copied here shows the basic issue:
> http://pastebin.com/f69b979b8  There are 2 classes: one where I am able to
> subclass numpy (with help from the great description at
> http://www.scipy.org/Subclasses), and the other where I subclass numpy.ma,
> using the same ideas again.
>
> When stepping through the code in a debugger, lines 76 to 96, I can see that
> the numpy subclass, called DT, calls DT.__array_wrap__() after it completes
> unary and binary operations. But the numpy.ma subclass, called DTMA, does
> not seem to call DTMA.__array_wrap__(), especially line 111.
>
> Just to test this idea, I overrode the __mul__ function in my DTMA subclass
> to call DTMA.__array_wrap__() and it returns my extra attributes, in the
> same way that Numpy did.
>
> My questions are:
>
> (a) Is MA intended to be subclassed?
>
> (b) If so, perhaps I'm missing something to make this work.  Any pointers
> will be appreciated.
>
> So far it seems the only way for me to sub-class numpy.ma is to override all
> numpy.ma functions of interest for my class and add a DTMA.__array_wrap()
> call to the end of them.  Hopefully there is an easier way.
>
> Related to this question, was there are particular outcome from this
> archived discussion (I only joined the list recently):
> http://article.gmane.org/gmane.comp.python.numeric.general/24315  because
> that dictionary object would be exactly what I'm after here.
>
> Thanks,
>
> Kevin
>

 timeseries in the scikits are subclassing MaskedArray and might
provide some examples

http://scipy.org/scipy/scikits/browser/trunk/timeseries/scikits/timeseries/tseries.py#L446

Josef

Re: Bilateral filter

>> Well, the latest cython doesn't help -- both errors still appear as
>> below. (Also, the latest cython can't run the numpy tests either.)  
>> I'm
>> befuddled.
>
> That's pretty weird.  Did you remove the .so that was build as well as
> any source files, before doing build_ext with the latest Cython?  Also
> good to make sure that the latest Cython is, in fact, the one being
> used.

Yeah... and I just tried that again, with the same results. I have no  
idea what could be going wrong. E.g. why would 'cimport numpy as np'  
not add np to the namespace on my machine whereas it does so on yours...

Also, I assume that constructs like:
         cdef int i, dim = data.dimensions[0]
are some special numpy-support syntax that's supposed to be added by  
the cimport numpy line? (Because numpy arrays don't expose a  
'dimensions' attribute to python code...) It's like for some reason on  
my machine, cython isn't building its numpy support correctly. Which  
is understandable in the light of the fact that cython can't pass the  
numpy tests on my machine either. Odd indeed. Maybe I'll try on the  
cython list, since you guys seem to have demonstrated that the problem  
isn't in the bilateral code!

Zach

Re: Bilateral filter

1. "dimensions" is a field in the C struct, that describes the array object.
2. Is there a chance that the header file numpy/arrayobject.h belongs to a 
   different numpy version that you run?

I am not very experienced with cython (I suppose that Stefan has some experience).
As you said, probably the cython list is a  better place to look for an answer. I would be happy to see how this
issue resolved.

  Nadav

-----הודעה מקורית-----
מאת: numpy-discussion-bounces <at> scipy.org בשם Zachary Pincus
נשלח: א 01-מרץ-09 20:59
אל: Discussion of Numerical Python
נושא: Re: [Numpy-discussion] Bilateral filter

>> Well, the latest cython doesn't help -- both errors still appear as
>> below. (Also, the latest cython can't run the numpy tests either.)  
>> I'm
>> befuddled.
>
> That's pretty weird.  Did you remove the .so that was build as well as
> any source files, before doing build_ext with the latest Cython?  Also
> good to make sure that the latest Cython is, in fact, the one being
> used.

Yeah... and I just tried that again, with the same results. I have no  
idea what could be going wrong. E.g. why would 'cimport numpy as np'  
not add np to the namespace on my machine whereas it does so on yours...

Also, I assume that constructs like:
         cdef int i, dim = data.dimensions[0]
are some special numpy-support syntax that's supposed to be added by  
the cimport numpy line? (Because numpy arrays don't expose a  
'dimensions' attribute to python code...) It's like for some reason on  
my machine, cython isn't building its numpy support correctly. Which  
is understandable in the light of the fact that cython can't pass the  
numpy tests on my machine either. Odd indeed. Maybe I'll try on the  
cython list, since you guys seem to have demonstrated that the problem  
isn't in the bilateral code!

Zach
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

loadtxt slow

So I have some data sets of about 160000 floating point numbers stored  
in text files.  I find that loadtxt is rather slow.  Is this to be  
expected?  Would it be faster if it were loading binary data?

-gideon

Re: loadtxt slow

On Sun, Mar 1, 2009 at 15:12, Gideon Simpson <simpson <at> math.toronto.edu> wrote:
> So I have some data sets of about 160000 floating point numbers stored
> in text files.  I find that loadtxt is rather slow.  Is this to be
> expected?

Probably. You don't say exactly what you mean by "slow", so it's
difficult to tell. But it is unlikely that you are running into some
slow corner case or something that no one else has seen.

> Would it be faster if it were loading binary data?

Substantially.

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: Bilateral filter

Hi guys,

Dag, the cython person who seems to deal with the numpy stuff, had  
this to say:
> - cimport and import are different things; you need both.
> - The "dimensions" field is in Cython renamed "shape" to be closer  
> to the Python interface. This is done in Cython/Includes/numpy.pxd

After including both the 'cimport' and 'import' lines, and changing  
'dimensions' to 'shape', things work perfectly. And now the shoe is on  
the other foot -- I wonder why you guys were getting proper results  
with what Dag claims to be buggy code! Odd indeed.

Zach

On Mar 1, 2009, at 2:36 PM, Nadav Horesh wrote:

> 1. "dimensions" is a field in the C struct, that describes the array  
> object.
> 2. Is there a chance that the header file numpy/arrayobject.h  
> belongs to a
>   different numpy version that you run?
>
> I am not very experienced with cython (I suppose that Stefan has  
> some experience).
> As you said, probably the cython list is a  better place to look for  
> an answer. I would be happy to see how this issue resolved.
>
>  Nadav
>
>
>
> -----הודעה מקורית-----
> מאת: numpy-discussion-bounces <at> scipy.org בשם Zachary Pincus
> נשלח: א 01-מרץ-09 20:59
> אל: Discussion of Numerical Python
> נושא: Re: [Numpy-discussion] Bilateral filter
>
>>> Well, the latest cython doesn't help -- both errors still appear as
>>> below. (Also, the latest cython can't run the numpy tests either.)
>>> I'm
>>> befuddled.
>>
>> That's pretty weird.  Did you remove the .so that was build as well  
>> as
>> any source files, before doing build_ext with the latest Cython?   
>> Also
>> good to make sure that the latest Cython is, in fact, the one being
>> used.
>
> Yeah... and I just tried that again, with the same results. I have no
> idea what could be going wrong. E.g. why would 'cimport numpy as np'
> not add np to the namespace on my machine whereas it does so on  
> yours...
>
> Also, I assume that constructs like:
>         cdef int i, dim = data.dimensions[0]
> are some special numpy-support syntax that's supposed to be added by
> the cimport numpy line? (Because numpy arrays don't expose a
> 'dimensions' attribute to python code...) It's like for some reason on
> my machine, cython isn't building its numpy support correctly. Which
> is understandable in the light of the fact that cython can't pass the
> numpy tests on my machine either. Odd indeed. Maybe I'll try on the
> cython list, since you guys seem to have demonstrated that the problem
> isn't in the bilateral code!
>
> Zach
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion <at> scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
> <winmail.dat>_______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion <at> scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: Bilateral filter

Zach,

I put the source my Cython generated here:

http://mentat.za.net/refer/bilateral_base.c

Can you try to compile that?

Cheers
Stéfan

2009/3/1 Zachary Pincus <zachary.pincus <at> yale.edu>:
>>> Well, the latest cython doesn't help -- both errors still appear as
>>> below. (Also, the latest cython can't run the numpy tests either.)
>>> I'm
>>> befuddled.
>>
>> That's pretty weird.  Did you remove the .so that was build as well as
>> any source files, before doing build_ext with the latest Cython?  Also
>> good to make sure that the latest Cython is, in fact, the one being
>> used.
>
> Yeah... and I just tried that again, with the same results. I have no
> idea what could be going wrong. E.g. why would 'cimport numpy as np'
> not add np to the namespace on my machine whereas it does so on yours...