Travis Oliphant | 1 Jun 01:18
Favicon

Re: Severe installation problems

Pearu Peterson wrote:

>This was caused by Changeset 3845 in numpy/distutils/misc_util.py.
>Travis, what problems did you have
>with data-files in top-level package directory?
>  
>
I'm not sure if you've fixed this or not,  but the problem I had was 
that I could not add data_files that were in the top-level package 
name-space no matter what I tried.

Look specifically in numpy/setup.py to see the addition of 
COMPATIBILITY, scipy_compatibility, and site.cfg.example.

These files were not being added to the distribution despite the 
presence of an add_data_file line in the setup.py code until I made the 
change.

Granted, I don't follow all of what is going on with numpy.distutils.  
If you were able to fix the problem a different way,  then great.

-Travis
Angus McMorland | 1 Jun 05:33
Picon

scipy compilation error on RHEL4

Hi list,

I'm trying to install scipyfrom svn source on our dual Intel Xeon
x86_64 server, which due to administrative reasons beyond my control
runs RHEL4, and for which I don't have root access, so I'm doing a
home dir install. I'm following the instructions in INSTALL.txt, and
have, I think successfully compiled LAPACK and ATLAS. In the scipy
compilation step, I get the following error:

/usr/bin/ld: /home/raid/amcmorl/lib/atlas/libptf77blas.a(dscal.o):
relocation R_X86_64_PC32 against `atl_f77wrap_dscal__' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
/usr/bin/ld: /home/raid/amcmorl/lib/atlas/libptf77blas.a(dscal.o):
relocation R_X86_64_PC32 against `atl_f77wrap_dscal__' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
error: Command "/usr/bin/g77 -g -Wall -shared
build/temp.linux-x86_64-2.3/Lib/integrate/_odepackmodule.o
-L/home/raid/amcmorl/lib/atlas -Lbuild/temp.linux-x86_64-2.3 -lodepack
-llinpack_lite -lmach -lptf77blas -lptcblas -latlas -lg2c -o
build/lib.linux-x86_64-2.3/scipy/integrate/_odepack.so" failed with
exit status 1

I'm not sure what precisely it is that needs to be recompiled with the
-fPIC flag. Any advice on how to proceed would be appreciated.

Thanks,
(Continue reading)

Robert Kern | 1 Jun 05:35
Picon
Gravatar

Re: scipy compilation error on RHEL4

Angus McMorland wrote:
> Hi list,
> 
> I'm trying to install scipyfrom svn source on our dual Intel Xeon
> x86_64 server, which due to administrative reasons beyond my control
> runs RHEL4, and for which I don't have root access, so I'm doing a
> home dir install. I'm following the instructions in INSTALL.txt, and
> have, I think successfully compiled LAPACK and ATLAS. In the scipy
> compilation step, I get the following error:
> 
> /usr/bin/ld: /home/raid/amcmorl/lib/atlas/libptf77blas.a(dscal.o):
> relocation R_X86_64_PC32 against `atl_f77wrap_dscal__' can not be used
> when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: ld returned 1 exit status
> /usr/bin/ld: /home/raid/amcmorl/lib/atlas/libptf77blas.a(dscal.o):
> relocation R_X86_64_PC32 against `atl_f77wrap_dscal__' can not be used
> when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: ld returned 1 exit status
> error: Command "/usr/bin/g77 -g -Wall -shared
> build/temp.linux-x86_64-2.3/Lib/integrate/_odepackmodule.o
> -L/home/raid/amcmorl/lib/atlas -Lbuild/temp.linux-x86_64-2.3 -lodepack
> -llinpack_lite -lmach -lptf77blas -lptcblas -latlas -lg2c -o
> build/lib.linux-x86_64-2.3/scipy/integrate/_odepack.so" failed with
> exit status 1
> 
> I'm not sure what precisely it is that needs to be recompiled with the
> -fPIC flag. Any advice on how to proceed would be appreciated.

(Continue reading)

Fred Jendrzejewski | 1 Jun 11:41
Picon

Problems with compilling a bjam under ubuntu

I know, that this could be a lil bit offtopic, but maybe not.

I have to compile c++-Library under ubuntu with boost.python.
Did anyone worked like this too? There always appears a warning, that no
Jamfile can be found.

If this is senseless noise in the mailing-list i am really sry, but i
didn't find any solution until now.

Kind Regards,
Fred Jendrzejewski
Pearu Peterson | 1 Jun 11:26
Picon
Picon
Favicon

Re: Severe installation problems

On Fri, June 1, 2007 2:18 am, Travis Oliphant wrote:
> Pearu Peterson wrote:
>
>>This was caused by Changeset 3845 in numpy/distutils/misc_util.py.
>>Travis, what problems did you have
>>with data-files in top-level package directory?
>>
>>
> I'm not sure if you've fixed this or not,  but the problem I had was
> that I could not add data_files that were in the top-level package
> name-space no matter what I tried.
>
> Look specifically in numpy/setup.py to see the addition of
> COMPATIBILITY, scipy_compatibility, and site.cfg.example.

Yes, I noticed that and have fixed in changeset 3848. This problem
was introduced when someone cleaned up the code..

I don't know if you are using mc or not for viewing the content
of sdist generated tar-ball, but I have noticed that sometimes, when
there is no changes in tar-ball file name, the mc uses old
cache to show the content of tar-ball. Copying the tar-ball
to another directory and viewing from there, shows the correct
content of the tar-ball.

Pearu
Erin Sheldon | 1 Jun 14:40
Picon
Gravatar

Re: Binary i/o package

The overwhelming silence tells me that either no one here thinks this
is relevant or no one bothered reading the email.  I feel like the
functionality I have written into this package is so basic it belongs
in scipy io if not in numpy itself. Please give me some feedback one
way or another.

If it just seems irrelevant then I may just look into making it a
scikits package.

Erin

On 5/30/07, Erin Sheldon <erin.sheldon <at> gmail.com> wrote:
> Hi all -
>
> The tofile() and fromfile() methods are excellent for reading and
> writing arrays to disk, and there are good tools in scipy for ascii
> input/output.
>
> I often find myself writing huge binary files to disk and then wanting
> to extract particular rows and columns from that file.  It is natural
> to associate the fields of a numpy array with the fields in the file,
> which may be inhomogeneous.  The ability to extract this information
> is straightforward to code in C/C++ and brings the file close to a
> database in functionality without all the overhead of working with a
> full database or pytables.
>
> I have written a simple C++ numpy extension for reading binary data
> into a numpy array, with the ability to select rows and fields
> (columns).  One enters the dtype that describes each row in
> list-of-tuples form and the code creates a numpy array (with perhaps a
(Continue reading)

dmitrey | 1 Jun 22:28
Favicon

small benchmark of LP solvers [from GSoC project]

hi,
for all who are interested:
take a look at the puny benchmark of LP solvers available in my GSoC openopt module (provided CVXOPT with glpk and mosek is installed, as well as lp_solve).
I shall try to connect the one to scikits during the next week (and switch for some weeks to other work, assigned to my GSoC milestones).
1st number is time elapsed (seconds), 2nd - cputime (opeopt bindings take less than 1-2% of the time)
LP name nVars nConstr density cvxopt_lp cvxopt_glpk cvxopt_mosek LPSolve




LGPL GPL2 non-free LGPL








trig 500 500 1 10.1/9.7 1.1/1.1 N/A(2) 0.4/0.4
trig 500 1000 1 16/15.9 3.5/3.4 N/A(2) 0.76/0.72
trig 1000 1000 1 99/98 6.8/6.7 N/A(2) 1.67/1.64
trig 1500 3000 1 479/470 177(1)/83 N/A(2) 7.9/7.7








trig_sparse 500 500 0.1 0.39/0.38 0.39/0.38 N/A(2) 0.16/0.17
trig_sparse 500 1000 0.1 4.83/4.64 0.83/0.82 N/A(2) 0.34/0.3
trig_sparse 1000 1000 0.1 12.7/12.5 1.88/1.86 N/A(2) 0.64/0.62
trig_sparse 1500 3000 0.1 131/128 17.5(1)/12.3 N/A(2) 3.91/3.81

(1): swap encountered (I have 1 Gb)
(2): internal mosek error (maybe problems with my AMD Athlon 64 3800+ X2)

you should also take into account lb bounds used, it gives additional nVars constraints.
some benchmarks of glpk are also available at http://plato.asu.edu/ftp/lpfree.html
mosek is available at http://plato.asu.edu/ftp/lpcom.html

all the solvers are available via single interface, let me paste data from the LP() doc (excuse my English):
    LP: constructor for Linear Problem assignment
    valid calls are:
    p = LP(f, <params>)
    p = LP(f=objFunVector, <params>)
    p = LP(f, A=A, Aeq=Aeq, Awhole=Awhole, b=b, beq=beq, bwhole=bwhole, dwhole=dwhole, lb=lb, ub=ub)
   
    NB! Constraints can be separated in many ways,
    either AX <= b, Aeq X = beq (MATLAB-, CVXOPT- style),
    or  Awhole X {< | = | >} bwhole  (glpk-, lp_solve- and many other software style),
    or any mix of them
   
    INPUTS:
    f: size n x 1 vector
    A: size m1 x n matrix, subjected to A * x <= b
    Aeq: size m2 x n matrix, subjected to Aeq * x = beq
    Awhole: size m3 x n matrix, subjected to Awhole * x { < | = | > } bwhole
    b, beq, bwhole: corresponding vectors with lengthes m1, m2, m3
    dwhole: vector of length m3 from {-1,0,1}, descriptor, sign of what (Awhole*x_opt - bwhole) should be equal to
    (this will simplify translating from other languages to Python and reduce the amount of mistakes
    as well as amount of additional code lines)
   
    OUTPUT: OpenOpt LP class instance
   
    Solving of LPs is performed via
    r = p.solve(string_name_of_solver)
    r.xf - desired solution (NaNs if a problem occured)
    r.ff - objFun value (<f,x_opt>) (NaN if a problem occured)
    (see also other fields, such as CPUTimeElapsed, TimeElapsed, etc)
    Currently string_name_of_solver can be:
    LPSolve (LGPL) - requires lpsolve + Python bindings installations (all mentioned is available in http://sourceforge.net/projects/lpsolve)
    cvxopt_lp (GPL) - requires CVXOPT (http://abel.ee.ucla.edu/cvxopt)
    cvxopt_glpk(GPL2) - requires CVXOPT(http://abel.ee.ucla.edu/cvxopt) & glpk (www.gnu.org/software/glpk)
    cvxopt_mosek(commercial) - requires CVXOPT(http://abel.ee.ucla.edu/cvxopt) & mosek (www.mosek.com)

    Example:
    Let's concider the problem
    15x1 + 8x2 + 80x3 -> min        (1)
    subjected to
    x1 + 2x2 + 3x3 <= 15              (2)     
    8x1 +  15x2 +  80x3 <= 80      (3)
    8x1  + 80x2 + 15x3 <=150      (4)
    100x1 +  10x2 + x3 >= 800     (5)
    80x1 + 8x2 + 15x3 = 750         (6)
    x1 + 10x2 + 100x3 = 80           (7)
   
    Let's pass (2), (3) to A X <= b, (6) to Aeq X = beq
    and rest of constraints will be handled via Awhole, bwhole, dwhole
   
    array, list, matrix and real number are accepted:
    f = array([15,8,80])
    A = mat('1 2 3; 8 15 80')
    b = [15, 80]
    Aeq = mat('80 8 15')
    beq = 750
    Awhole = mat('8 80 15; 1 10 100; 100 10 1')
    bwhole = array([150, 80, 800])
    dwhole = [-1, 0, 1]
    p = LP(f, A=A, Aeq=Aeq, Awhole=Awhole, b=b, beq=beq, bwhole=bwhole, dwhole=dwhole)
    r = p.solve('LPSolve') #lp_solve must be installed
    print 'objFunValue:', r.ff # should print 204.350288002
    print 'x_opt:', r.xf

There are also NLP, NSP classes available (currently unconstrained solvers ralg and ShorEllipsoid are supplied).
LPSolve and glpk also provide MILP solvers, but cvxopt connection to glpk (that one is required) can't handle the integer indexes, so in my MILP class in nearest future  will be only one solver (LPSolve)
I know that there is a software for connecting commercial LP/MILP/QP solver CPLEX to Python, but (at least currently)  I have no time for the one.

I shall gladly take into account all your suggestions.
Regards, Dmitrey.
 

_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev
Angus McMorland | 2 Jun 01:09
Picon

Re: scipy compilation error on RHEL4

On 01/06/07, Robert Kern <robert.kern <at> gmail.com> wrote:
> Angus McMorland wrote:
> > Hi list,
> >
> > I'm trying to install scipy from svn source on our dual Intel Xeon
> > x86_64 server, which due to administrative reasons beyond my control
> > runs RHEL4, and for which I don't have root access, so I'm doing a
> > home dir install. I'm following the instructions in INSTALL.txt, and
> > have, I think successfully compiled LAPACK and ATLAS. In the scipy
> > compilation step, I get the following error:
> >
> > /usr/bin/ld: /home/raid/amcmorl/lib/atlas/libptf77blas.a(dscal.o):
> > relocation R_X86_64_PC32 against `atl_f77wrap_dscal__' can not be used
> > when making a shared object; recompile with -fPIC
> > /usr/bin/ld: final link failed: Bad value
> > collect2: ld returned 1 exit status
> > /usr/bin/ld: /home/raid/amcmorl/lib/atlas/libptf77blas.a(dscal.o):
> > relocation R_X86_64_PC32 against `atl_f77wrap_dscal__' can not be used
> > when making a shared object; recompile with -fPIC
> > /usr/bin/ld: final link failed: Bad value
> > collect2: ld returned 1 exit status
> > error: Command "/usr/bin/g77 -g -Wall -shared
> > build/temp.linux-x86_64-2.3/Lib/integrate/_odepackmodule.o
> > -L/home/raid/amcmorl/lib/atlas -Lbuild/temp.linux-x86_64-2.3 -lodepack
> > -llinpack_lite -lmach -lptf77blas -lptcblas -latlas -lg2c -o
> > build/lib.linux-x86_64-2.3/scipy/integrate/_odepack.so" failed with
> > exit status 1
> >
> > I'm not sure what precisely it is that needs to be recompiled with the
> > -fPIC flag. Any advice on how to proceed would be appreciated.
>
> ATLAS needs to be recompiled with -fPIC.

Okay, then I don't know how to do that correctly. I tried, in the
ATLAS configuration script, adding -fPIC to the c-compiler flags,
which resulted, in the makefile, in:

CCFLAG0 = -fomit-frame-pointer -O3 -funroll-all-loops -fPIC

and

MMFLAGS = -fomit-frame-pointer -O -fPIC

and recompiled atlas, but still got the same error when compiling
scipy. What's the correct approach?

Thanks again,

Angus.
--

-- 
AJC McMorland, PhD Student
Physiology, University of Auckland
Anne Archibald | 3 Jun 20:48
Picon

Re: Binary i/o package

On 01/06/07, Erin Sheldon <erin.sheldon <at> gmail.com> wrote:
> The overwhelming silence tells me that either no one here thinks this
> is relevant or no one bothered reading the email.  I feel like the
> functionality I have written into this package is so basic it belongs
> in scipy io if not in numpy itself. Please give me some feedback one
> way or another.
>
> If it just seems irrelevant then I may just look into making it a
> scikits package.

I'm not trying to knock your work, but it's not clear to me that
there's enough room between readarray/writearray/tofile/fromfile and
pytables to accommodate another package. Maybe I don't see what your
package does, but why wouldn't I just install pytables instead? What
are its advantages and disadvantages compared to pytables?

Anne
Joachim Dahl | 3 Jun 22:28
Picon

Re: small benchmark of LP solvers [from GSoC project]

If you use the native Python LP solver in CVXOPT for sparse problems, make sure you
specify the matrices as sparse,  as this conversion is not done automatically in the solvers.
I couldn't tell from your post what comparisons you're doing, but just in case...

Joachim

On 6/1/07, dmitrey <openopt <at> ukr.net> wrote:
hi,
for all who are interested:
take a look at the puny benchmark of LP solvers available in my GSoC openopt module (provided CVXOPT with glpk and mosek is installed, as well as lp_solve).
I shall try to connect the one to scikits during the next week (and switch for some weeks to other work, assigned to my GSoC milestones).
1st number is time elapsed (seconds), 2nd - cputime (opeopt bindings take less than 1-2% of the time)

LP name
nVars
nConstr
density
cvxopt_lp
cvxopt_glpk
cvxopt_mosek
LPSolve









LGPL
GPL2
non-free
LGPL


















trig
500
500
1
10.1/9.7
1.1/1.1
N/A(2)
0.4/0.4

trig
500
1000
1
16/15.9
3.5/3.4
N/A(2)
0.76/0.72

trig
1000
1000
1
99/98
6.8/6.7
N/A(2)
1.67/1.64

trig
1500
3000
1
479/470
177(1)/83
N/A(2)
7.9/7.7


















trig_sparse
500
500
0.1
0.39/0.38
0.39/0.38
N/A(2)
0.16/0.17

trig_sparse
500
1000
0.1
4.83/4.64
0.83/0.82
N/A(2)
0.34/0.3

trig_sparse
1000
1000
0.1
12.7/12.5
1.88/1.86
N/A(2)
0.64/0.62

trig_sparse
1500
3000
0.1
131/128
17.5(1)/12.3
N/A(2)
3.91/3.81



(1): swap encountered (I have 1 Gb)
(2): internal mosek error (maybe problems with my AMD Athlon 64 3800+ X2)

you should also take into account lb bounds used, it gives additional nVars constraints.
some benchmarks of glpk are also available at http://plato.asu.edu/ftp/lpfree.html
mosek is available at http://plato.asu.edu/ftp/lpcom.html

all the solvers are available via single interface, let me paste data from the LP() doc (excuse my English):
    LP: constructor for Linear Problem assignment
    valid calls are:
    p = LP(f, <params>)
    p = LP(f=objFunVector, <params>)
    p = LP(f, A=A, Aeq=Aeq, Awhole=Awhole, b=b, beq=beq, bwhole=bwhole, dwhole=dwhole, lb=lb, ub=ub)
   
    NB! Constraints can be separated in many ways,
    either AX <= b, Aeq X = beq (MATLAB-, CVXOPT- style),
    or  Awhole X {< | = | >} bwhole  (glpk-, lp_solve- and many other software style),
    or any mix of them
   
    INPUTS:
    f: size n x 1 vector
    A: size m1 x n matrix, subjected to A * x <= b
    Aeq: size m2 x n matrix, subjected to Aeq * x = beq
    Awhole: size m3 x n matrix, subjected to Awhole * x { < | = | > } bwhole
    b, beq, bwhole: corresponding vectors with lengthes m1, m2, m3
    dwhole: vector of length m3 from {-1,0,1}, descriptor, sign of what (Awhole*x_opt - bwhole) should be equal to
    (this will simplify translating from other languages to Python and reduce the amount of mistakes
    as well as amount of additional code lines)
   
    OUTPUT: OpenOpt LP class instance
   
    Solving of LPs is performed via
    r = p.solve(string_name_of_solver)
    r.xf - desired solution (NaNs if a problem occured)
    r.ff - objFun value (<f,x_opt>) (NaN if a problem occured)
    (see also other fields, such as CPUTimeElapsed, TimeElapsed, etc)
    Currently string_name_of_solver can be:
    LPSolve (LGPL) - requires lpsolve + Python bindings installations (all mentioned is available in http://sourceforge.net/projects/lpsolve)
    cvxopt_lp (GPL) - requires CVXOPT (http://abel.ee.ucla.edu/cvxopt)
    cvxopt_glpk(GPL2) - requires CVXOPT(http://abel.ee.ucla.edu/cvxopt) & glpk (www.gnu.org/software/glpk)
    cvxopt_mosek(commercial) - requires CVXOPT(http://abel.ee.ucla.edu/cvxopt) & mosek (www.mosek.com)

    Example:
    Let's concider the problem
    15x1 + 8x2 + 80x3 -> min        (1)
    subjected to
    x1 + 2x2 + 3x3 <= 15              (2)     
    8x1 +  15x2 +  80x3 <= 80      (3)
    8x1  + 80x2 + 15x3 <=150      (4)
    100x1 +  10x2 + x3 >= 800     (5)
    80x1 + 8x2 + 15x3 = 750         (6)
    x1 + 10x2 + 100x3 = 80           (7)
   
    Let's pass (2), (3) to A X <= b, (6) to Aeq X = beq
    and rest of constraints will be handled via Awhole, bwhole, dwhole
   
    array, list, matrix and real number are accepted:
    f = array([15,8,80])
    A = mat('1 2 3; 8 15 80')
    b = [15, 80]
    Aeq = mat('80 8 15')
    beq = 750
    Awhole = mat('8 80 15; 1 10 100; 100 10 1')
    bwhole = array([150, 80, 800])
    dwhole = [-1, 0, 1]
    p = LP(f, A=A, Aeq=Aeq, Awhole=Awhole, b=b, beq=beq, bwhole=bwhole, dwhole=dwhole)
    r = p.solve('LPSolve') #lp_solve must be installed
    print 'objFunValue:', r.ff # should print 204.350288002
    print 'x_opt:', r.xf

There are also NLP, NSP classes available (currently unconstrained solvers ralg and ShorEllipsoid are supplied).
LPSolve and glpk also provide MILP solvers, but cvxopt connection to glpk (that one is required) can't handle the integer indexes, so in my MILP class in nearest future  will be only one solver (LPSolve)
I know that there is a software for connecting commercial LP/MILP/QP solver CPLEX to Python, but (at least currently)  I have no time for the one.

I shall gladly take into account all your suggestions.
Regards, Dmitrey.
 


_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev


_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-dev

Gmane