headers
Pearu Peterson | 5 Jan 2005 11:29
Picon
Picon
Favicon

Re: scipy testing changes revisited


On 22 Dec 2004, Todd Miller wrote:

> A week or so ago we were discussing the set of changes to scipy's
> testing framework needed to add support for numarray.   The changes I'm
> talking about here should be made to the main trunk of scipy CVS.
> 
> After further review,  I now think the array versions of the assert
> family of functions (e.g. assert_array_equal) are correct with respect
> to truth value testing.  So, the only change I think is needed is the
> addition of "delegation code" so that calls to assert_equal, etc. defer
> to assert_array_equal, etc. when passed array parameters.   Here's the
> patch:

I have applied the patch with an addition of calling array versions
of assert_* functions whenever either of arguments is an array.

> These changes are necessary for adding numarray support to scipy because
> without them there are 12 testing failures (in scipy_base.test()).  The
> failures are all related to numarray's "outlawed" __nonzero__() and
> testers apparently calling the wrong assertion function.  With these
> changes,  7 of the numarray failures go away and 5 identical failures
> remain for both numarray and Numeric.  I'm arguing that the 5 remaining
> failures are either real problems or testing bugs which were masked by
> testers calling the wrong assert functions and by Numeric's sometrue()
> definition of __nonzero__(). 

I agree. These 5 failures are now fixed in the main trunk.

> In each case you can see that assert_array_equal has been called from
(Continue reading)

Permalink | Reply | 3 comments |
headers
Perry Greenfield | 7 Jan 2005 00:16

Status page for numarray integration

Now that there will soon be some visible sign of numarray integration 
efforts, it seems to me that having some sort of page indicating the 
current state of integration would be helpful. That is, it would 
outline the general plan for the effort, indicate what's been done and 
is currently available for use with numarray, what being worked on and 
the general priorities. Users can use to see if the functionality they 
wish to use with numarray is already available (i.e., which libraries) 
or where the unfinished work sits in the queue. It can be used for 
developers to indicate where help would be useful if others want to 
contribute. Any objection to having such a page?

Perry
Permalink | Reply | 2 comments |
headers
Todd Miller | 7 Jan 2005 22:48

Re: CVS scipy.test() segfaulting on me on a Fedora2 box

On Wed, 2004-12-22 at 13:59, Pearu Peterson wrote:
> On Wed, 22 Dec 2004, Fernando Perez wrote:
> 
> > Charles Harris wrote:
> >> Ok,
> >> 
> >> I've got the reorganized version from cvs and still don't get a segfault
> >> using the distro BLAS and gcc 3.4.2.
> >
> > I suspect the problem lies with ATLAS.  I am using 
> > atlas3.6.0_Linux_P4SSE2_2HT.tgz from the scipy website, which from reading 
> > the included makefile, was built as follows:
> >
> >   F77 = /home/pearu/bin/g77
> >   F77FLAGS = -fomit-frame-pointer -O -fno-second-underscore
> >
> > I think the problem is ATLAS, b/c the top of the backtrace I included 
> > yesterday had this:
> >
> > #0  0x55cd67c2 in ATL_cdotc_xp0yp0aXbX ()
> >    from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so
> >
> > That first symbol is, I imagine, an ATLAS call.  But Pearu seems to have 
> > built these binaries with pretty conservative flags (-O), and when you use 
> > the distro BLAS (no ATLAS) you don't get any segfault.  So perhaps it's a bug 
> > in ATLAS and/or g77 which appears even with basic optimizations.
> 
> Actually the Fortran lapack library that was used to complete ATLAS 
> lapack, may have been compiled with -O3, I don't remember exactly anymore.
> If somebody could confirm that compiling lapack library with -O2 will
(Continue reading)

Permalink | Reply | 4 comments |
headers
Pearu Peterson | 7 Jan 2005 23:58
Favicon

Re: CVS scipy.test() segfaulting on me on a Fedora2 box


On Fri, 7 Jan 2005, Todd Miller wrote:

> Can anyone confirm that using -O2 for lapack fixes this?
>  I thought I'd worked around this (crash in ATL_cdotc_xp0yp0aXbX ()) 
> using Pearu's advice but I'm seeing it again.
> Was the scipy ATLAS ever  re-built?

No. Rebuilding all scipy ATLAS libraries is a rather time consuming task
and I was hoping that someone could confirm that the proposed fix works 
before I'd spend time on it. Otherwise its just a waste my time if it 
turns out that using -O2 does not fix the issue.

Pearu
Permalink | Reply | 3 comments |
headers
Todd Miller | 8 Jan 2005 03:28

Re: CVS scipy.test() segfaulting on me on a Fedora2 box

On Fri, 2005-01-07 at 16:58 -0600, Pearu Peterson wrote:
> 
> On Fri, 7 Jan 2005, Todd Miller wrote:
> 
> > Can anyone confirm that using -O2 for lapack fixes this?
> >  I thought I'd worked around this (crash in ATL_cdotc_xp0yp0aXbX ()) 
> > using Pearu's advice but I'm seeing it again.
> > Was the scipy ATLAS ever  re-built?
> 
> No. Rebuilding all scipy ATLAS libraries is a rather time consuming task
> and I was hoping that someone could confirm that the proposed fix works 
> before I'd spend time on it. Otherwise its just a waste my time if it 
> turns out that using -O2 does not fix the issue.

Sorry Pearu.  I think I panicked here and then saw that this was still
an open issue while I was looking for hints.  SciPy's head built fine
for me today and all the tests passed.  If I recall correctly,  there is
some important update to the head that's missing from the numarray
branch so when I built the numarray branch, it failed.  I'll resolve it
Monday morning and report back but I think this was just confusion on my
part.

Regards,
Todd
Permalink | Reply | 2 comments |
headers
Charles Harris | 8 Jan 2005 07:52
Favicon

Types

Looks like Guido is considering adding static typing to python.
How would this match up with traits? Here are some links:

http://www.artima.com/weblogs/viewpost.jsp?thread=87182
http://www.artima.com/weblogs/viewpost.jsp?thread=85551
http://www.artima.com/weblogs/viewpost.jsp?thread=86641

Chuck
Permalink | Reply | 0 comments |
headers
Todd Miller | 10 Jan 2005 18:00

Re: CVS scipy.test() segfaulting on me on a Fedora2 box

On Fri, 2005-01-07 at 21:28, Todd Miller wrote:
> On Fri, 2005-01-07 at 16:58 -0600, Pearu Peterson wrote:
> > 
> > On Fri, 7 Jan 2005, Todd Miller wrote:
> > 
> > > Can anyone confirm that using -O2 for lapack fixes this?
> > >  I thought I'd worked around this (crash in ATL_cdotc_xp0yp0aXbX ()) 
> > > using Pearu's advice but I'm seeing it again.
> > > Was the scipy ATLAS ever  re-built?
> > 
> > No. Rebuilding all scipy ATLAS libraries is a rather time consuming task
> > and I was hoping that someone could confirm that the proposed fix works 
> > before I'd spend time on it. Otherwise its just a waste my time if it 
> > turns out that using -O2 does not fix the issue.
> 
> Sorry Pearu.  I think I panicked here and then saw that this was still
> an open issue while I was looking for hints.  SciPy's head built fine
> for me today and all the tests passed.  If I recall correctly,  there is
> some important update to the head that's missing from the numarray
> branch so when I built the numarray branch, it failed.  I'll resolve it
> Monday morning and report back but I think this was just confusion on my
> part.
> 
> Regards,
> Todd

I was able to resolve this (again).  The root problem was incorrect
detection of ATLAS so other people seeing the ATL_cdotc_xp0yp0aXbX()
crash should check their build output for correct ATLAS info. In my case
the ATLAS detection failure was brought on by using a debug Python build
(Continue reading)

Permalink | Reply | 1 comment |
headers
Perry Greenfield | 10 Jan 2005 21:43

questions about adding new projects

So if we want to start a new "topical" library project hosted on 
scipy.org, are there any guidelines about where in CVS it is to be put? 
Under "topical"? (can we think up a better name than topical?). Under 
the name we want to give it? (By the way, when is a switch to 
subversion planned?).

Thanks, Perry
Permalink | Reply | 0 comments |
headers
Todd Miller | 10 Jan 2005 21:49

Merged numarray branch

As part of an effort to make scipy work with numarray,  I merged the
numarray branch of CVS onto the main trunk about an hour ago.  In
theory,  this has no effect on the operation of scipy using Numeric and
will eventually enable operation based on numarray.  As of today,  the
"numarray merge" mostly affects scipy_base so there are many updates
that remain to be done to scipy as a whole before numarray is equally
supported.

Operation with Numeric is the default.  To run using numarray,  you
"setenv NUMERIX numarray" and then start python and import scipy as
usual.  There is currently output on the console which indicates which
array package you're getting at import time.  

There are still errors in the self-tests using numarray so this is
mostly a curiosity announcement and head's up to be on the lookout for
other problems.  If you're curious, get numarray from CVS on source
forge first.

Regards,
Todd
Permalink | Reply | 12 comments |
headers
Joe Harrington | 10 Jan 2005 22:35
Picon

Re: questions about adding new projects

Others have suggested a different name from "topical" (for the listing
page), and I agree.  My initial intent with "topical" was to get
discipline-specific software cataloged, since it can be harder to
find than general packages.  But, of course there is stuff that is
cross-discipline that is not in the main package, like matplotlib.
Some ideas:

application	Long.  Is everything an application?  No.
addon		probably the most general and accurate
extras		shades of Fedora
contrib		has a "not one of us, but along for the ride" connotation

I like "addon", but better might still be out there.

--jh--
Permalink | Reply | 0 comments |

Gmane