Subject: Re: what the heck is an egg

Joe Harrington wrote:
> A lot of the add-on software for numpy/scipy is distributed using novel
> Python install processes (eggs, setup.py) 
setup.py and using distutils is not novel: this is the standard way to 
distribute python packages to be built for years (distutils was included 
in python in 2000, 2001 ?). It offers a common set of rules to build 
python packages, including compiled extensions, on a wide range of 
platforms. Eggs is a different matter.

Think setup.py as the configure script for python packages
> rather than tarballs or the
> preferred OS-native installers (dpkg, rpm, etc.).  
Building binary packages is totally different than distributing source 
tarballs. Work on that is always welcome, but packaging is not a funny 
thing to do, the rewards mostly consisting in getting complaints when it 
does not work :) Several people develop rpm and deb (Andrew Straw has 
debian packages, for example, I started developing rpm for FC and 
openSUSE using the open suse build system), but ideally, this should be 
included upstream in the different distributions (this is the case for 
debian packages at least, I think: it is in debian and ubuntu).
> I'm sure they are
> described, perhaps even well, in other places, but, since scipy.org is
> our portal, I think it would be good to have a few-line description of
> each method on the Download page and a link to more detailed
> descriptions elsewhere (or on subsidiary pages).  An example of
> installing a package many will want, like mayavi2, would be great.
>
> In particular, many sysadmins (who might be considering a user's request
> for an install and know nothing about python) get nervous when package
> managers other than the native one for the OS start mucking around in

Subject: Re: what the heck is an egg

> setup.py and using distutils is not novel: this is the standard way to 
> distribute python packages to be built for years (distutils was included 
> in python in 2000, 2001 ?).

Well, it's a matter of perspective, and the perspective I take is that
of a new user to Python, or even a non-Python-using system manager who
is installing numpy and friends for others to use.  To them, *anything*
that is not a tar.gz, RPM, or Deb is novel, and most would not dare to
use even an un-novel tar.gz in their OS directories.  Then we say, here,
execute this setup.py as root, it's code for an interpreted language and
you have no idea what it will do.  Well, that's pretty terrifying,
especially to the security-conscious.

I know almost nothing about eggs.  I see them being used for all the
Enthought code, which provides the de facto standard 3D environment,
mayavi2.  What's a numerical package without a 3D environment?  While
that's not on scipy.org, it's darn close, and necessary for an
environment that competes with IDL or Matlab.

I agree that the correct path is to push everything into binary
installs, even the experimental stuff.  I love the OS installers, and I
thank the packagers from the bottom of my heart!  If only there were
more of them, and if only they could handle more of these packages.  The
OS installers may not deal with multiple package version on Linux, but I
have never wanted more than one version.  Someone who does is probably a
developer and can handle the tar installs, eggs, or whatever, and direct
Python to find the results.  I believe that we would double our
community's size if all our major packages were available in binary
installs for all major platforms.

Subject: Re: what the heck is an egg

Joe Harrington wrote:
>> setup.py and using distutils is not novel: this is the standard way to 
>> distribute python packages to be built for years (distutils was included 
>> in python in 2000, 2001 ?).
>>     
>
> Well, it's a matter of perspective, and the perspective I take is that
> of a new user to Python, or even a non-Python-using system manager who
> is installing numpy and friends for others to use.  To them, *anything*
> that is not a tar.gz, RPM, or Deb is novel, and most would not dare to
> use even an un-novel tar.gz in their OS directories.  Then we say, here,
> execute this setup.py as root, it's code for an interpreted language and
> you have no idea what it will do.  Well, that's pretty terrifying,
> especially to the security-conscious.
>   
If you have any strong concern on security, you don't use random 
tarballs sources downloaded from the internet. You can also install as a 
non root (I do this all the time, I don't have root access on my 
workstation at my lab, for example). I don't see how this is different 
than any other source package (using autoconf, for example).

If you have a better idea, please speak up, but you will have a hard 
time to sell something which works as well as distutils, on as many 
platforms, and which is less novel. As an aside, I would say that 
someone who is not capable of installing a basic python package after 10 
minutes is not worth being called an administrator: there is a README, 
you can find the info in one minute in google. Now, numpy + scipy is 
certainly more difficult to install, because of dependencies and so on, 
but then this is no different than pure C tarballs.
> I know almost nothing about eggs.  I see them being used for all the

Subject: Re: what the heck is an egg

Joe Harrington wrote:
>> setup.py and using distutils is not novel: this is the standard way to 
>> distribute python packages to be built for years (distutils was included 
>> in python in 2000, 2001 ?).
> 
> Well, it's a matter of perspective, and the perspective I take is that
> of a new user to Python, or even a non-Python-using system manager who
> is installing numpy and friends for others to use.  To them, *anything*
> that is not a tar.gz, RPM, or Deb is novel, and most would not dare to
> use even an un-novel tar.gz in their OS directories.  Then we say, here,
> execute this setup.py as root, it's code for an interpreted language and
> you have no idea what it will do.  Well, that's pretty terrifying,
> especially to the security-conscious.
[...]
> But, a plethora of packagers is not our situation.  It would help the
> inexperienced (including the aforementioned system manager, who will
> never be experienced in Python) to have some plain talk about what these
> Python-specific installers do, and how to use them to install,
> uninstall, and automatically update.  It can probably be knocked off in
> about a page per installer, but it has to be done by someone who knows
> them well.

Here is what I do, maybe it will help.

I'll skip the blah about eggs and keep this focussed.  The nice thing 
about easy_install is that it will find packages for you, deal with 
dependencies, optionally build them if you need to and install packages 
for you.  How best you use them depends on what you want to do.

I'll cover the case for an ubuntu gutsy user who does not want a system 

Subject: Re: what the heck is an egg

Joe Harrington wrote:
>> setup.py and using distutils is not novel: this is the standard way to 
>> distribute python packages to be built for years (distutils was included 
>> in python in 2000, 2001 ?).
> 
> Well, it's a matter of perspective, and the perspective I take is that
> of a new user to Python, or even a non-Python-using system manager who
> is installing numpy and friends for others to use.  To them, *anything*
> that is not a tar.gz, RPM, or Deb is novel, and most would not dare to
> use even an un-novel tar.gz in their OS directories.  Then we say, here,
> execute this setup.py as root, it's code for an interpreted language and
> you have no idea what it will do.  Well, that's pretty terrifying,
> especially to the security-conscious.

It takes enough time to write a package, write the build scripts, 
document everything nicely, polish the documentation, write tests to 
cover an adequate number of cases, test the code, and maintain the code 
while not breaking the tests. Meanwhile, many of us need to get some 
science done in the middle of all of it. The fact that some people 
haven't gotten around to debbing or RPM'ing their packages is 
understandable. It's all a matter of whether the author has time or 
interest in doing packaging. If someone is reluctant to use a package 
because the author has yet to get it packaged up, then how is it the 
problem of the author or community? This does not mean we don't want to 
have everything nicely packaged up in RPM or dpkg format.

> I know almost nothing about eggs.  I see them being used for all the
> Enthought code, which provides the de facto standard 3D environment,
> mayavi2.  What's a numerical package without a 3D environment?  While
> that's not on scipy.org, it's darn close, and necessary for an

scipy 0.6.0 tests kill python interpreter

Hi,

if I install scipy 0.6.0 on Debian and run tests, it kills the
interpreter with Illegal instruction.

More details here:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=452991

Is anyone able to reproduce it on their system, or is it just related
to Debian? Could you please send me
the correct run of tests, so that I can discover where exactly it differs?

Thanks a lot,
Ondrej

Re: scipy 0.6.0 tests kill python interpreter

Ondrej Certik wrote:
> Hi,
> 
> if I install scipy 0.6.0 on Debian and run tests, it kills the
> interpreter with Illegal instruction.
> 
> More details here:
> 
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=452991
> 
> Is anyone able to reproduce it on their system, or is it just related
> to Debian? Could you please send me
> the correct run of tests, so that I can discover where exactly it differs?

Try scipy.test(verbosity=1). This will print the name of the test before it runs
the test. Also, try running Python under gdb so we can get a backtrace.

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

Re: scipy 0.6.0 tests kill python interpreter

On Dec 1, 2007 10:44 PM, Robert Kern <robert.kern <at> gmail.com> wrote:
>
> Ondrej Certik wrote:
> > Hi,
> >
> > if I install scipy 0.6.0 on Debian and run tests, it kills the
> > interpreter with Illegal instruction.
> >
> > More details here:
> >
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=452991
> >
> > Is anyone able to reproduce it on their system, or is it just related
> > to Debian? Could you please send me
> > the correct run of tests, so that I can discover where exactly it differs?
>
> Try scipy.test(verbosity=1). This will print the name of the test before it runs
> the test. Also, try running Python under gdb so we can get a backtrace.

It doesn't seem to help much:

>>> import scipy
s>>> scipy.test(verbosity=1)
  Found 9 tests for scipy.cluster.vq
  Found 18 tests for scipy.fftpack.basic
  Found 4 tests for scipy.fftpack.helper
  Found 20 tests for scipy.fftpack.pseudo_diffs
  Found 1 tests for scipy.integrate
  Found 10 tests for scipy.integrate.quadpack
  Found 3 tests for scipy.integrate.quadrature

Re: scipy 0.6.0 tests kill python interpreter

Ondrej Certik wrote:
> On Dec 1, 2007 10:44 PM, Robert Kern <robert.kern <at> gmail.com> wrote:
>> Ondrej Certik wrote:
>>> Hi,
>>>
>>> if I install scipy 0.6.0 on Debian and run tests, it kills the
>>> interpreter with Illegal instruction.
>>>
>>> More details here:
>>>
>>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=452991
>>>
>>> Is anyone able to reproduce it on their system, or is it just related
>>> to Debian? Could you please send me
>>> the correct run of tests, so that I can discover where exactly it differs?
>> Try scipy.test(verbosity=1). This will print the name of the test before it runs
>> the test. Also, try running Python under gdb so we can get a backtrace.
> 
> It doesn't seem to help much:
> 
>>>> import scipy
> s>>> scipy.test(verbosity=1)

My apologies.

  scipy.test(verbosity=2)

> But when running through gdb, it says:
> 
>   warnings.warn('Mode "reflect" may yield incorrect results on '

Re: scipy 0.6.0 tests kill python interpreter

On Dec 1, 2007 11:06 PM, Robert Kern <robert.kern <at> gmail.com> wrote:
> Ondrej Certik wrote:
> > On Dec 1, 2007 10:44 PM, Robert Kern <robert.kern <at> gmail.com> wrote:
> >> Ondrej Certik wrote:
> >>> Hi,
> >>>
> >>> if I install scipy 0.6.0 on Debian and run tests, it kills the
> >>> interpreter with Illegal instruction.
> >>>
> >>> More details here:
> >>>
> >>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=452991
> >>>
> >>> Is anyone able to reproduce it on their system, or is it just related
> >>> to Debian? Could you please send me
> >>> the correct run of tests, so that I can discover where exactly it differs?
> >> Try scipy.test(verbosity=1). This will print the name of the test before it runs
> >> the test. Also, try running Python under gdb so we can get a backtrace.
> >
> > It doesn't seem to help much:
> >
> >>>> import scipy
> > s>>> scipy.test(verbosity=1)
>
> My apologies.
>
>   scipy.test(verbosity=2)

boundary modes/usr/lib/python2.4/site-packages/scipy/ndimage/interpolation.py:41:
UserWarning: Mode "reflect" may yield incorrect results on boundaries.