headers
Yakov Keselman | 1 Nov 01:13
Picon

Re: help to speed up the python code

My understanding of the root of the problem is that you end up with
doing many evaluations of sinc. If this is so, one suggestion is to go
with pre-computed filters. For example, if you are resampling from 9
points to 10, essentially you're trying to go from a function that is
defined on points 0, 1, 2..., 8 to a function defined on points 0,
0.9, 1.8, 2.7, ... 0.8. If you take point 2.7, it falls in between 2
and 3, and the value for your new function can be approximately
computed as the weighted average: f(2.7) = 0.7*f(3) + 0.3*f(2). This
is equivalent to convolving with a filter [0.3, 0.7]. The only problem
is that the filter will vary from point to point. So, perhaps, you can
pre-compute several filters for each type of new point (there will be
10 point types for this specific case). Your filters would also be
larger in length, so that your interpolation is more precise. Hope
this helps.

On 10/31/08, Charles R Harris <charlesr.harris <at> gmail.com> wrote:
> On Thu, Oct 30, 2008 at 11:44 PM, frank wang <f.yw <at> hotmail.com> wrote:
>
>>  Hi, Bob,
>>
>> The problem is that I want to resample my data with another sampling rate.
>> the two rates is very close. I use the formula:
>>
>> s(t)=sum(a_k*sinc(t-kTs)).
>>
>> the new sampling rate is Ts', so I have
>> s(nTs')=sum(a_k*sinc(nTs'-kTs)). The sum index k is over the (-P, P),
>> Centered at n. The n is start from zero. THe code is using two for loops
>> and
>> it is slow. The length of s(nTs) is very long, so it takes quite long time
(Continue reading)

Permalink | Reply | 0 comments |
headers
David Cournapeau | 1 Nov 06:10
Picon

Re: Complete LAPACK needed (Frank Lagor)

On Sat, Nov 1, 2008 at 4:42 AM, Frank Lagor <dfranci <at> seas.upenn.edu> wrote:

> ImportError: liblapack.so: cannot open shared object file: No such file or
> directory

Where did you install atlas ? What does ldd says about lapack_lite.so module ?

More likely, your problem is that you install atlas into a directory
not known to the OS. You should either install atlas in a directory
known to it, or use things like LD_LIBRARY_PATH. For example, if you
installed atlas in $HOME/local/lib:

LD_LIBRARY_PATH=$HOME/local/lib python -c "import numpy"

David
Permalink | Reply | 0 comments |
headers
Robert Kern | 1 Nov 09:07
Picon
Gravatar

Re: Simplifying compiler optimization flags logic (fortran compilers)

On Fri, Oct 31, 2008 at 05:25, David Cournapeau
<david <at> ar.media.kyoto-u.ac.jp> wrote:
> Hi,
>
>    I was wondering whether it was really worth having a lot of magic
> going on in fcompilers for flags like -msse2 and co (everything done in
> get_flags_arch, for example). It is quite fragile (we had several
> problems wrt buggy compilers, buggy CPU detection), and I am not sure it
> buys us much anyway. Did some people notice a difference between
> gfortran -O3 -msse2 and gfortran -O3 ?

You're probably right.

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
Permalink | Reply | 3 comments |
headers
Frank Lagor | 1 Nov 15:20
Favicon

Re: Complete LAPACK needed (Frank Lagor)

Problem solved-- This posting is just to complete the thread to document it if others have similar issues.

The previous error:
ImportError: liblapack.so: cannot open shared object file: No such file or directory

was solved simply by checking that the environment variables were set properly.  I set BLAS, LAPACK, and LD_LIBRARY_PATH variables (I'm not sure if the LD one was needed, but this worked)

The next issue I ran into was that the error message changed to "undefined symbol _dgesdd",
which I googled and discovered was a routine for singular value decomposition that was missing-- meaning that my liblapack.so file was incomplete.  I checked this by doing
$ strings liblapack.so | grep dgesdd
and noticing that the output was blank.  The output was not blank however in the lapack _LINUX.a install from the netlib lapack, so that meant that my ATLAS install was incorrect.  I changed the way that I specified the configure for ATLAS.  I had to use the --with-netlib-lapack option (as stated in one of the installation files) instead of the "-Ss flapack pathname" that is specified in the command line help.  Finally, the install worked.

-Frank


_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Permalink | Reply | 1 comment |
headers
David Cournapeau | 1 Nov 15:10
Picon
Picon

Re: Complete LAPACK needed (Frank Lagor)

Frank Lagor wrote:
> Problem solved-- This posting is just to complete the thread to
> document it if others have similar issues.

Glad you could solve it.

>
> The previous error:
> ImportError: liblapack.so: cannot open shared object file: No such
> file or directory
>
> was solved simply by checking that the environment variables were set
> properly.  I set BLAS, LAPACK, and LD_LIBRARY_PATH variables (I'm not
> sure if the LD one was needed, but this worked)

Yes, the LD_ is necessary (LD stands for loader).

>
> The next issue I ran into was that the error message changed to
> "undefined symbol _dgesdd",
> which I googled and discovered was a routine for singular value
> decomposition that was missing-- meaning that my liblapack.so file was
> incomplete.  I checked this by doing
> $ strings liblapack.so | grep dgesdd

Note that strings is not really the best way to check that: nm is.
strings only look for strings in the binary, and does not look for
symbols (e.g. you could find a string which does not correspond to a
function the loader and/or the linker can find).

> and noticing that the output was blank.  The output was not blank
> however in the lapack _LINUX.a install from the netlib lapack, so that
> meant that my ATLAS install was incorrect.  I changed the way that I
> specified the configure for ATLAS.  I had to use the
> --with-netlib-lapack option (as stated in one of the installation
> files) instead of the "-Ss flapack pathname" 

Ah yes, I could never manage to make this option work either. I always
use --with-netlib-lapack when I need to build a custom ATLAS.

cheers,

David
Permalink | Reply | 0 comments |
headers
Jarrod Millman | 1 Nov 18:15
Picon
Favicon
Gravatar

Re: Simplifying compiler optimization flags logic (fortran compilers)

On Sat, Nov 1, 2008 at 1:07 AM, Robert Kern <robert.kern <at> gmail.com> wrote:
> On Fri, Oct 31, 2008 at 05:25, David Cournapeau
> <david <at> ar.media.kyoto-u.ac.jp> wrote:
>>    I was wondering whether it was really worth having a lot of magic
>> going on in fcompilers for flags like -msse2 and co (everything done in
>> get_flags_arch, for example). It is quite fragile (we had several
>> problems wrt buggy compilers, buggy CPU detection), and I am not sure it
>> buys us much anyway. Did some people notice a difference between
>> gfortran -O3 -msse2 and gfortran -O3 ?
>
> You're probably right.

I think it is probably best to take out some of the magic in fcompilers as well.

--

-- 
Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014
http://cirl.berkeley.edu/
Permalink | Reply | 2 comments |
headers
Frank Lagor | 1 Nov 18:30
Favicon

Re: Complete LAPACK needed (Frank Lagor)

Thanks so much for your help, David.  I'm sorry I did not receive your posts previously -- I have the digest mode on and there is a bit of a delay.  I'll try to change my options next time I post a request.

Thanks so much again,
Frank

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Permalink | Reply | 501 comments |
headers
Michael Abshoff | 1 Nov 20:09

Re: Simplifying compiler optimization flags logic (fortran compilers)

Jarrod Millman wrote:
> On Sat, Nov 1, 2008 at 1:07 AM, Robert Kern <robert.kern <at> gmail.com> wrote:

Hi,

>> On Fri, Oct 31, 2008 at 05:25, David Cournapeau
>> <david <at> ar.media.kyoto-u.ac.jp> wrote:
>>>    I was wondering whether it was really worth having a lot of magic
>>> going on in fcompilers for flags like -msse2 and co (everything done in
>>> get_flags_arch, for example). It is quite fragile (we had several
>>> problems wrt buggy compilers, buggy CPU detection), and I am not sure it
>>> buys us much anyway. Did some people notice a difference between
>>> gfortran -O3 -msse2 and gfortran -O3 ?
>> You're probably right.

we removed setting the various SSE flags in Sage's numpy install because 
they caused segfaults when using gfortran. I don't think that there is a 
significant performance difference with SSE for that code because we use 
Lapack and ATLAS build with SSE when it is available.

> I think it is probably best to take out some of the magic in fcompilers as well.
> 

Cheers,

Michael
Permalink | Reply | 1 comment |
headers
Abhimanyu Lad | 2 Nov 19:24
Picon

numpy function to compute sample ranks

Hi,

Is there a direct or indirect way in numpy to compute the sample ranks of a
given array, i.e. the equivalent of rank() in R.

I am looking for:
rank(array([6,8,4,1,9])) -> array([2,3,1,0,4])

Is there some clever use of argsort() that I am missing?

Thanks,
Abhi
Permalink | Reply | 2 comments |
headers
Keith Goodman | 2 Nov 19:35
Picon
Gravatar

Re: numpy function to compute sample ranks

On Sun, Nov 2, 2008 at 10:24 AM, Abhimanyu Lad <abhimanyulad <at> gmail.com> wrote:
> Is there a direct or indirect way in numpy to compute the sample ranks of a
> given array, i.e. the equivalent of rank() in R.
>
> I am looking for:
> rank(array([6,8,4,1,9])) -> array([2,3,1,0,4])
>
> Is there some clever use of argsort() that I am missing?

If there are no NaNs and no ties, then:

>> x = np.array([6,8,4,1,9])
>> x.argsort().argsort()
   array([2, 3, 1, 0, 4])

If you have ties, then scipy has a ranking function.
Permalink | Reply | 1 comment |

Gmane