josef.pktd | 9 Feb 21:22
Picon

miso_filter

In case anyone is interested: how to do a multi-input single-output
linear filter in 2 lines of code and 50 lines of tests

    inp = signal.correlate(x, ma[::-1,:])[:, (x.shape[1]+1)//2]
    signal.lfilter([1], ar, inp), inp

(I needed the tests to clear up a misunderstanding between me and convolve.)

details at
http://bazaar.launchpad.net/~josef-pktd/statsmodels/statsmodels-josef-experimental/annotate/head%3A/scikits/statsmodels/sandbox/regression/mle.py#L363

with application to simulating GARCH models (those are not verified yet)

Josef
mdekauwe | 9 Feb 19:06
Picon

[SciPy-user] Compiling scipy on Solaris


Hi,

I am trying to install the latest scipy libraries and whilst everything (i
think) builds without issues I have encountered some relocation issues
finding libraries.

A number of the basic functionalities work find e.g.

import scipy

a = scipy.arange(10)

However 

if I tried to import something from the optimise library, for example 

from scipy.optimize import leastsq

ImportError: ld.so.1: python: fatal: relocation error: file
/users/eow/mgdk/sun4u/lib/python/scipy/optimize/_lbfgsb.so: symbol etime_:
referenced symbol not found

Now I am not sure to be honest what I am not doing. If it helps...
ldd -d /users/eow/mgdk/sun4u/lib/python/scipy/optimize/_lbfgsb.so

        libfsu.so.1 =>   /nerc/packages/studio/12.0/SUNWspro/lib/libfsu.so.1
        libsunmath.so.1 =>      
/nerc/packages/studio/12.0/SUNWspro/lib/libsunmath.so.1
        libf77compat.so.1 =>    
(Continue reading)

Jose Gomez-Dans | 9 Feb 17:12
Picon

Efficiently applying a function over a large array

Hi!
I have a function that does some fairly involved linear algebra using some vectors that are defined over 2D arrays. You can think of it as the 3rd dimension of a 3D array. For each "pixel" in the 2D array, I want to appy my function to [i,j,:]. Now, that's all very easy to do in theory using a loop, eg
for i in ny:
  for j in nx:
     Out[i,j] = MyFunc ( arr1[i,j,:], arr2[i,j,:], arr3[i,j,:] )

but the array size is quite large (>1000x1000 elements), so I would like to know what the most efficient way of doing this would be. I came accross numpy.apply_along_axis, but also about the fact that is probably quite slow for these large arrays?

Thanks!
Jose

_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
John Salvatier | 9 Feb 06:12
Gravatar

What optimization function should I use?

Hello,

I am writing an adaptive MCMC sampler for PyMC and I have found that providing a good starting point improves its performance dramatically, so I have used scipy.optimize.fmin_cfg to find a mode of the posterior distribution to use as the starting point. However, I don't know much about different optimization routines, so I don't know if this is a good good choice or not.

Does anyone have advice on which of the optimization routines I should use? I don't care too much about performance because I only have to run the optimization routine a few times at the start of sampling. I should also mention that I have access to the analytical gradient. My algorithm is intended to be for general use, so I can't say too much about the functions on which it will be used, other than they will likely be approximately normal (so the function will be approximately multi-quadradic) near the mode. I don't care if it does not work well for multiple optima.

John

_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Nils Wagner | 8 Feb 21:14
Picon
Favicon

Roots of a signal

Hi all,

How can I compute the roots of a signal ?

The following didn't work due to a memory error.

from scipy.interpolate import splrep, splev, sproot
from numpy import exp, sin, linspace, pi
from pylab import plot, show, legend
t = linspace(0,4,20)
y = exp(-0.1*t)*sin(2*pi*t)
plot(t,y,label='Signal')
tck = splrep(t,y)
t_new = linspace(0,10,40)
y2 = splev(t_new,tck)
plot(t_new,y2,label='Spline')
roots = sproot(tck,10)
legend(loc=0)
show()

Any pointer would be appreciated.

Thanks in advance

                                  Nils
Nancy Lee | 8 Feb 21:09
Picon

scikits.timeseries 'Combine TimeSeries objects'

 
Hi everyone,
 
I have one question about the scikits.timeseries TimeSeries objects.
Is there any function that can combine multiple timeseries objects into one in the correct chronological order?
For example,  I have A timeseries object with '2009-02-03' data, and B timeseries object with '2009-02-04' data, and C with '2009-02-05' data, how can I get one timeseries from 2009-02-03 to 2009-02-05 ?
Any suggestion? thanks a lot!
 
Nancy

_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Skipper Seabold | 7 Feb 22:17
Picon

help with f2py directives for lapack function wrapper

I abandoned the ctypes route to exposing dtgsen and am trying the f2py route.  I'm hoping someone might be able to answer my questions.  The f2py documentation is pretty good, and I am beginning to feel like I have some idea of what I am doing, but I am running into problems with a few f2py directives.  For the record, I am only interested in the function for the case when IJOB=0. 

I have produced a file ordqz.so that I can import and look at the docstring, but when I try to use the function I get  

ValueError: failed to create intent(cache|hide)|optional array-- must have defined dimensions but got (-1,)                                          

Which I assume comes from the LWORK argument as stated in the docs.

I have defined M, PL, PR, WORK, LWORK, IWORK, and LIWORK as follows in ordqz.pyf based on the docs <http://linux.die.net/man/l/dtgsen> (in-line below).  These should be defined based on the value of IJOB.  Is there some other way to go about this, or can someone just point me to a relevant function that's already in the numpy/scipy source as a reference for a similar situation?

!xxx - made up dimensions for pl and pr, and dependency of m
           integer depend(pl),intent(hide):: m = shape(pl,0)
           double precision intent(out),dimension(n):: pl
           double precision intent(out),dimension(n):: pr
           double precision dimension(2),intent(out):: dif
           double precision intent(hide,cache),dimension(lwork),depend(lwork):: work
!Made up a value for lwork (its lower bound) 4*n+16
           integer intent(hide),depend(n):: lwork=4*n+16
           integer intent(hide,cache),dimension(liwork),depend(liwork):: iwork
!Made up a value for liwork (its lower bound) n+6
           integer intent(hide):: liwork=n+6


Relevant part of docs
-------
M (output) INTEGER
   The dimension of the specified pair of left and right eigen- spaces (deflating subspaces). 0 <= M <= N.

   PL, PR (output) DOUBLE PRECISION If IJOB = 1, 4 or 5, PL, PR are lower bounds on the reciprocal of the norm of "projections" onto left and right eigenspaces with respect to the selected cluster. 0 < PL, PR <= 1. If M = 0 or M = N, PL = PR = 1. If IJOB = 0, 2 or 3, PL and PR are not referenced.
DIF (output) DOUBLE PRECISION array, dimension (2).
   If IJOB >= 2, DIF(1:2) store the estimates of Difu and Difl.
   If IJOB = 2 or 4, DIF(1:2) are F-norm-based upper bounds on
   Difu and Difl. If IJOB = 3 or 5, DIF(1:2) are 1-norm-based estimates of Difu and Difl. If M = 0 or N, DIF(1:2) = F-norm([A, B]). If IJOB = 0 or 1, DIF is not referenced.
WORK (workspace/output) DOUBLE PRECISION array, dimension (LWORK)
   IF IJOB = 0, WORK is not referenced. Otherwise, on exit, if INFO = 0, WORK(1) returns the optimal LWORK.
LWORK (input) INTEGER
   The dimension of the array WORK. LWORK >= 4*N+16. If IJOB = 1, 2 or 4, LWORK >= MAX(4*N+16, 2*M*(N-M)). If IJOB = 3 or 5, LWORK >= MAX(4*N+16, 4*M*(N-M)).

   If LWORK = -1, then a workspace query is assumed; the routine only calculates the optimal size of the WORK array, returns this value as the first entry of the WORK array, and no error message related to LWORK is issued by XERBLA.
IWORK (workspace/output) INTEGER array, dimension (LIWORK)
   IF IJOB = 0, IWORK is not referenced. Otherwise, on exit, if INFO = 0, IWORK(1) returns the optimal LIWORK.
LIWORK (input) INTEGER
   The dimension of the array IWORK. LIWORK >= 1. If IJOB = 1, 2 or 4, LIWORK >= N+6. If IJOB = 3 or 5, LIWORK >= MAX(2*M*(N-M), N+6).

   If LIWORK = -1, then a workspace query is assumed; the routine only calculates the optimal size of the IWORK array, returns this value as the first entry of the IWORK array, and no error message related to LIWORK is issued by XERBLA.

Thanks for any help,

Skipper

_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Nancy Lee | 7 Feb 03:23
Picon

Install scikits.timeseries error: Unable to find vcvarsall.bat'

 
Hi,
I am very new with Python and sikits.timeseries package. I tired to install scikits.timeseries-0.91.3.tar.gz, I unzip it and
use 'python setup.py install ' command. However, it shows error msg as
 
'No module named msvccompiler in numpy.distutils; trying from distutils
customize MSVCCompiler
customize MSVCCompiler using build_ext
building 'scikits.timeseries.cseries' extension
compiling C sources
error: Unable to find vcvarsall.bat'
 
My OS is win 32 and the scikits.timeseries-0.91.3.win32-py2.6.exe file works fine for me , but for some specifit reason, I need to use the setup.py to install the package.
Could anyone pls help me? Thanks!
 
Nancy

_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Skipper Seabold | 6 Feb 21:10
Picon

Wrapping a lapack function (attempt results seg fault)

All,

I'm trying to wrap the lapack function dtgsen for the ordered generalized Schur decomposition given real valued arrays <http://linux.die.net/man/l/dtgsen>.  However, I'm getting a segmentation fault, and I have no idea why.  I was hoping someone might look at my work and see where I've gone wrong, as this is my first attempt at trying to do something like this.  One specific thing I wasn't sure about is when the lapack function calls for a LOGICAL type can this just be

from ctypes import c_bool
logical_true = c_bool(True)

?

I've attached my functions and arrays that make a self-contained example.  Just put the two files in a directory and run.  When I do a gdb backtrace I receive the following output.

(gdb) run ordqztest.py                                                                              
Starting program: /usr/bin/python ordqztest.py                                                      
[Thread debugging using libthread_db enabled]                                                      

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff5015d7d in dtgsen_ () from /usr/lib/liblapack.so
(gdb) backtrace                                            
#0  0x00007ffff5015d7d in dtgsen_ () from /usr/lib/liblapack.so
#1  0x00007ffff6da8bec in ffi_call_unix64 ()                  
   at /build/buildd/python2.6-2.6.4/Modules/_ctypes/libffi/src/x86/unix64.S:75
#2  0x00007ffff6da82d3 in ffi_call (cif=0x7fffffffd810, fn=<value optimized out>,
   rvalue=<value optimized out>, avalue=<value optimized out>)                  
   at /build/buildd/python2.6-2.6.4/Modules/_ctypes/libffi/src/x86/ffi64.c:430  
#3  0x00007ffff6da2d64 in _call_function_pointer (pProc=<value optimized out>,    
   argtuple=<value optimized out>, flags=<value optimized out>, argtypes=<value optimized out>,
   restype=<value optimized out>, checker=<value optimized out>)                                
   at /build/buildd/python2.6-2.6.4/Modules/_ctypes/callproc.c:816                              
#4  _CallProc (pProc=<value optimized out>, argtuple=<value optimized out>,                      
   flags=<value optimized out>, argtypes=<value optimized out>, restype=<value optimized out>,  
   checker=<value optimized out>) at /build/buildd/python2.6-2.6.4/Modules/_ctypes/callproc.c:1163
#5  0x00007ffff6d9abf7 in CFuncPtr_call (self=0xc2c2c0, inargs=0x18, kwds=<value optimized out>)  
   at /build/buildd/python2.6-2.6.4/Modules/_ctypes/_ctypes.c:3860                                
#6  0x000000000041d6e7 in PyObject_Call (func=0xc2c2c0, arg=0xcb8360, kw=0xc63520)                
   at ../Objects/abstract.c:2492                                                                  
#7  0x00000000004a199c in do_call (f=0xcb98c0, throwflag=<value optimized out>)                    
   at ../Python/ceval.c:3924                                                                      
#8  call_function (f=0xcb98c0, throwflag=<value optimized out>) at ../Python/ceval.c:3729          
#9  PyEval_EvalFrameEx (f=0xcb98c0, throwflag=<value optimized out>) at ../Python/ceval.c:2389    
#10 0x00000000004a40e0 in PyEval_EvalCodeEx (co=0x7ffff7f01f30, globals=<value optimized out>,    
   locals=<value optimized out>, args=0x9, argcount=<value optimized out>, kws=<value optimized out>,
   kwcount=2, defs=0x7ffff7f0a7f8, defcount=4, closure=0x0) at ../Python/ceval.c:2968                
#11 0x00000000004a245f in fast_function (f=0xcb9660, throwflag=<value optimized out>)                  
   at ../Python/ceval.c:3802                                                                          
#12 call_function (f=0xcb9660, throwflag=<value optimized out>) at ../Python/ceval.c:3727              
---Type <return> to continue, or q <return> to quit---                                                
#13 PyEval_EvalFrameEx (f=0xcb9660, throwflag=<value optimized out>) at ../Python/ceval.c:2389
#14 0x00000000004a40e0 in PyEval_EvalCodeEx (co=0x7ffff7f026c0, globals=<value optimized out>,
   locals=<value optimized out>, args=0x8, argcount=<value optimized out>, kws=<value optimized out>,
   kwcount=2, defs=0x7ffff3ea9e78, defcount=3, closure=0x0) at ../Python/ceval.c:2968
#15 0x00000000004a245f in fast_function (f=0x928e00, throwflag=<value optimized out>)
   at ../Python/ceval.c:3802
#16 call_function (f=0x928e00, throwflag=<value optimized out>) at ../Python/ceval.c:3727
#17 PyEval_EvalFrameEx (f=0x928e00, throwflag=<value optimized out>) at ../Python/ceval.c:2389
#18 0x00000000004a40e0 in PyEval_EvalCodeEx (co=0x7ffff7f02378, globals=<value optimized out>,
   locals=<value optimized out>, args=0x0, argcount=<value optimized out>, kws=<value optimized out>,
   kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:2968
#19 0x00000000004a41b2 in PyEval_EvalCode (co=0x0, globals=0xcb8360, locals=0xc63520)
   at ../Python/ceval.c:522
#20 0x00000000004c33a0 in run_mod (fp=0x919900, filename=<value optimized out>,
   start=<value optimized out>, globals=<value optimized out>, locals=0x8b9270, closeit=1,
   flags=0x7fffffffe100) at ../Python/pythonrun.c:1335
#21 PyRun_FileExFlags (fp=0x919900, filename=<value optimized out>, start=<value optimized out>,
   globals=<value optimized out>, locals=0x8b9270, closeit=1, flags=0x7fffffffe100)
   at ../Python/pythonrun.c:1321
#22 0x00000000004c3564 in PyRun_SimpleFileExFlags (fp=<value optimized out>,
   filename=0x7fffffffe518 "ordqztest.py", closeit=1, flags=0x7fffffffe100)
   at ../Python/pythonrun.c:931
#23 0x0000000000418ab7 in Py_Main (argc=-135384960, argv=<value optimized out>) at ../Modules/main.c:599
#24 0x00007ffff6fd0abd in __libc_start_main (main=<value optimized out>, argc=<value optimized out>,
   ubp_av=<value optimized out>, init=<value optimized out>, fini=<value optimized out>,
   rtld_fini=<value optimized out>, stack_end=0x7fffffffe218) at libc-start.c:220
#25 0x0000000000417ca9 in _start () at ../sysdeps/x86_64/elf/start.S:113

Any help or pointers would be appreciated, as I am a bit out of my depth.

-Skipper

Attachment (ordqztest.py): text/x-python, 4544 bytes
Attachment (testvals.npz): application/octet-stream, 6942 bytes
_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Warren Weckesser | 6 Feb 19:55

[Fwd: Re: weired results with ode solver]

I replied to Oz, but didn't "Reply to all", so for completeness, I'm 
forwarding this to the list.

From: Warren Weckesser <warren.weckesser <at> enthought.com>
Subject: Re: weired results with ode solver
Date: 2010-02-06 18:51:38 GMT
Oz Nahum wrote:
> Hi Warren,
> Thanks for your answer,
>
> here is my updated code:
<snip>
> from pylab import *
> plot(timerange/86400, c[:,0], 'bo')
> plot(timerange/86400, c[:,1], 'go')
> plot(timerange/86400, c[:,2], 'ko')
> show()
>
>
> I seems that it runs much faster than matlab (though I have not really
> checked). It's syntacticly clearer to read. Much more fun.
> There is one issue that bothers me still. I don't want to show the
> concentrations every second. So I divide by 86400 sec perday. However,
> this results in smeared concentrations, where every day has  a few of
> them. Why is that ? how can I remove it ?
>   

This is a consequence Python's integer division.  'timerange' is an 
array of integers, and you are dividing by 84600, an integer, so the 
result will be an array of integers.  Change those three plot() function 
calls to:

plot(timerange/86400.0, c[:,0], 'bo')
plot(timerange/86400.0, c[:,1], 'go')
plot(timerange/86400.0, c[:,2], 'ko')

Warren

_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Oz Nahum | 6 Feb 18:32
Picon

weired results with ode solver

Hi Everyone,

I'm trying to convert the following matlab code into python, the code
simulates reduction of organic matter by bacteria in a batch system.
It has to files in the matlab version, and 1 in the python version.
I'm not sure I did the conversion correctly, but I'm sure the result
I'm getting from python is wrong - probably because I did not really
understand how to use the solver. Any insights would be appreciated.

Here are the codes:
% This file is a matlab script to simulate a batch problem as described

% in problem 1a.

% This file calls batch1_rate.m

%clear all;

clc;

%Define variables

Coi=3;              % initial concentration of oxigen [mg/l]

Csi=10;             % initial concentration of substrat [mg/l]

Cbi=0.2;            % initial concentration of biomass [mg/l]

%define vector of concentrations

c1=[Coi;Csi;Cbi];

ko=0.1;             % Monod coefficients Oxigen [mg/l]

ks=0.1;             % Monod coefficients Substract [mg/l]

umax=0.125/86400;   % Maximun growth rate [1/d]

Yo=0.125;           % Oxigen Yield [mg X / mg O]

Ys=0.25;            % Substract Yield [mg X / mg S]

Kdec=0.01/86400;    % Decay rate

kb=1;              % Biomass Inhibition constant [mg/L]

%Solve ODE

timerange=[0 43*86400];

[t,c]=ode15s(@batch1_rate,timerange,c1,[],ko,ks,Ys,Yo,Kdec,umax,kb);

%Plot

hf=figure(1);

clf;

%plot(,c(:,1:3),'linewidth',2);

days=t/86400;

plot(days,c(:,1),'b-',days,c(:,2),'k--','linewidth',2);

hold on

plot(days,c(:,3),'-','Color',[0 0.498 0],'linewidth',1);

xlabel('T [days]','fontsize',14);

ylabel('C [mg/L]','fontsize',14);

ylim([0 10]);

xlim([0 22])

#end of 1st matlab file.
% This file is an aid function to the script batch1.m which describes

% a batch system relations of substrate, oxygen and one microbial specie

% over time.

% Oz Nahum, Anibal Perez, Georgia Kastanioti, Januar 2010.

function f=batch1_rate(t,c1,ko,ks,Ys,Yo,Kdec,umax,kb)

co=c1(1); %concentration of Oxygen

cs=c1(2); %concentration of Substrate

X=c1(3); %concentration of bacteria

kgr=umax*(co/(ko+co))*(cs/(ks+cs)); % specific uptake rate [mol/m3/s]

Ib=1+X/kb;  				 		% inhibiton caused by biomass

foxi=((-kgr/Yo/Ib)*X);  %dc/dt of Oxygen

fsub=((-kgr/Ys/Ib)*X);  %dc/dt of Substrate

fbio=((kgr/Ib-Kdec)*X); %inhibition for growth only

f=[foxi;fsub;fbio];

end

#end of second matlab file

#python script:
from scipy.integrate import odeint
from numpy import *
#%clear all;
#clc;

#%Define variables
Coi=3              # initial concentration of oxigen [mg/l]
Csi=10             # initial concentration of substrat [mg/l]
Cbi=0.2            # initial concentration of biomass [mg/l]

#define vector of concentrations
#c1 = array((Coi,Csi,Cbi))
c1=(Coi,Csi,Cbi)
y0=c1
ko=0.1             #% Monod coefficients Oxigen [mg/l]
ks=0.1             #% Monod coefficients Substract [mg/l]

umax=0.125/86400 #  % Maximun growth rate [1/d]

Yo=0.125#;           % Oxigen Yield [mg X / mg O]
Ys=0.25#;            % Substract Yield [mg X / mg S]

Kdec=0.01/86400#;    % Decay rate
kb=1#;              % Biomass Inhibition constant [mg/L]

def rate_func(c,time):
	ko=0.1             #% Monod coefficients Oxigen [mg/l]
	ks=0.1             #% Monod coefficients Substract [mg/l]
	umax=0.125/86400 #  % Maximun growth rate [1/d]
	Yo=0.125#;           % Oxigen Yield [mg X / mg O]
	Ys=0.25#;            % Substract Yield [mg X / mg S]
	Kdec=0.01/86400#;    % Decay rate
	kb=1#;              % Biomass Inhibition constant [mg/L]
	co=c1[0]#; %concentration of Oxygen
	cs=c1[1]#; %concentration of Substrate
	X=c1[2]#; %concentration of bacteria
	kgr=umax*(co/(ko+co))*(cs/(ks+cs))#; % specific uptake rate [mol/m3/s]
	Ib=1+X/kb#;  				 		% inhibiton caused by biomass
	foxi=((-kgr/Yo/Ib)*X)#;  %dc/dt of Oxygen
	fsub=((-kgr/Ys/Ib)*X)#;  %dc/dt of Substrate
	fbio=((kgr/Ib-Kdec)*X)#; %inhibition for growth only
	f=array((foxi,fsub,fbio))
	return  f

#Solve ODE
timerange=arange(1,43*86400,60)
parms=(c1,ko,ks,Ys,Yo,Kdec,umax,kb)
#[t,c]=ode15s(@batch1_rate,timerange,c1,[],ko,ks,Ys,Yo,Kdec,umax,kb);
#c=odeint(rate_func,parms,timerange)
c=odeint(rate_func,c1,timerange)
#sprint c

from pylab import *
plot(timerange, c[:,0], 'b-')
plot(timerange, c[:,1], 'g-')
plot(timerange, c[:,2], 'k-')
show()

Many thanks in advance.

Oz Nahum
Graduate Student
Zentrum für Angewandte Geologie
Universität Tübingen

---

Imagine there's no countries
it isn't hard to do
Nothing to kill or die for
And no religion too
Imagine all the people
Living life in peace
_______________________________________________
SciPy-User mailing list
SciPy-User <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user

Gmane