headers
Andrew Dalke | 1 Jul 01:32
Gravatar

"import numpy" is slow

(Trying again now that I'm subscribed.  BTW, there's no link to the  
subscription page from numpy.scipy.org .)

The initial 'import numpy' loads a huge number of modules, even when  
I don't need them.

Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import sys
 >>> len(sys.modules)
28
 >>> import numpy
 >>> len(sys.modules)
256
 >>> len([s for s in sorted(sys.modules) if 'numpy' in s])
127
 >>> numpy.__version__
'1.1.0'
 >>>

As a result, I assume that's the reason my program's startup cost is  
quite high.

[josiah:~/src/fp] dalke% time python -c 'a=4'
0.014u 0.038s 0:00.05 80.0%     0+0k 0+1io 0pf+0w
[josiah:~/src/fp] dalke% time python -c 'import numpy'
0.161u 0.279s 0:00.44 97.7%     0+0k 0+9io 0pf+0w

My total runtime is something like 1.4 seconds, and the only thing
(Continue reading)

Permalink | Reply | 83 comments |
headers
Robert Kern | 1 Jul 02:22
Picon
Gravatar

Re: "import numpy" is slow

On Mon, Jun 30, 2008 at 18:32, Andrew Dalke <dalke <at> dalkescientific.com> wrote:
> (Trying again now that I'm subscribed.  BTW, there's no link to the
> subscription page from numpy.scipy.org .)
>
>
> The initial 'import numpy' loads a huge number of modules, even when
> I don't need them.
>
> Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
> [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import sys
>  >>> len(sys.modules)
> 28
>  >>> import numpy
>  >>> len(sys.modules)
> 256
>  >>> len([s for s in sorted(sys.modules) if 'numpy' in s])
> 127
>  >>> numpy.__version__
> '1.1.0'
>  >>>
>
> As a result, I assume that's the reason my program's startup cost is
> quite high.
>
> [josiah:~/src/fp] dalke% time python -c 'a=4'
> 0.014u 0.038s 0:00.05 80.0%     0+0k 0+1io 0pf+0w
> [josiah:~/src/fp] dalke% time python -c 'import numpy'
> 0.161u 0.279s 0:00.44 97.7%     0+0k 0+9io 0pf+0w
(Continue reading)

Permalink | Reply | 5 comments |
headers
Andrew Dalke | 1 Jul 03:00
Gravatar

Re: "import numpy" is slow

On Jul 1, 2008, at 2:22 AM, Robert Kern wrote:
> Your use case isn't so typical and so suffers on the import time  
> end of the
> balance.

I'm working on my presentation for EuroSciPy.  "Isn't so typical"  
seems to be a good summary of my first slide.  :)

>> Any chance of cutting down on the number, in order
>> to improve startup costs?
>
> Not at this point in time, no. That would break too much code.

Understood.

Thanks for the response,

				Andrew
				dalke <at> dalkescientific.com
Permalink | Reply | 4 comments |
headers
Hanni Ali | 1 Jul 12:13
Picon

Re: "import numpy" is slow

Would it not be possible to import just the necessary module of numpy to meet the necessary functionality of your application.

i.e.

import numpy.core

or whatever you're using

you could even do:

import numpy.core as numpy

I think, to simplify your code, I'm no expert though.

Hanni


2008/7/1 Andrew Dalke <dalke <at> dalkescientific.com>:
On Jul 1, 2008, at 2:22 AM, Robert Kern wrote:
> Your use case isn't so typical and so suffers on the import time
> end of the
> balance.

I'm working on my presentation for EuroSciPy.  "Isn't so typical"
seems to be a good summary of my first slide.  :)

>> Any chance of cutting down on the number, in order
>> to improve startup costs?
>
> Not at this point in time, no. That would break too much code.

Understood.

Thanks for the response,

                               Andrew
                               dalke <at> dalkescientific.com


_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Permalink | Reply | 3 comments |
headers
Matthieu Brucher | 1 Jul 12:24
Picon

Re: "import numpy" is slow

Hi,

IIRC, il you do import numpy.core as numpy, it starts by importing
numpy, so it will be even slower.

Matthieu

2008/7/1 Hanni Ali <hanni.ali <at> gmail.com>:
> Would it not be possible to import just the necessary module of numpy to
> meet the necessary functionality of your application.
>
> i.e.
>
> import numpy.core
>
> or whatever you're using
>
> you could even do:
>
> import numpy.core as numpy
>
> I think, to simplify your code, I'm no expert though.
>
> Hanni
>
>
> 2008/7/1 Andrew Dalke <dalke <at> dalkescientific.com>:
>>
>> On Jul 1, 2008, at 2:22 AM, Robert Kern wrote:
>> > Your use case isn't so typical and so suffers on the import time
>> > end of the
>> > balance.
>>
>> I'm working on my presentation for EuroSciPy.  "Isn't so typical"
>> seems to be a good summary of my first slide.  :)
>>
>> >> Any chance of cutting down on the number, in order
>> >> to improve startup costs?
>> >
>> > Not at this point in time, no. That would break too much code.
>>
>> Understood.
>>
>> Thanks for the response,
>>
>>                                Andrew
>>                                dalke <at> dalkescientific.com
>>
>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion <at> scipy.org
>> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion <at> scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>

--

-- 
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
Permalink | Reply | 2 comments |
headers
Hanni Ali | 1 Jul 12:44
Picon

Re: "import numpy" is slow

You are correct, it appears to take slightly longer to import numpy.core and longer again to import numpy.core as numpy

I should obviously check first in future.

Hanni

2008/7/1 Matthieu Brucher <matthieu.brucher <at> gmail.com>:
Hi,

IIRC, il you do import numpy.core as numpy, it starts by importing
numpy, so it will be even slower.

Matthieu

2008/7/1 Hanni Ali <hanni.ali <at> gmail.com>:
> Would it not be possible to import just the necessary module of numpy to
> meet the necessary functionality of your application.
>
> i.e.
>
> import numpy.core
>
> or whatever you're using
>
> you could even do:
>
> import numpy.core as numpy
>
> I think, to simplify your code, I'm no expert though.
>
> Hanni
>
>
> 2008/7/1 Andrew Dalke <dalke <at> dalkescientific.com>:
>>
>> On Jul 1, 2008, at 2:22 AM, Robert Kern wrote:
>> > Your use case isn't so typical and so suffers on the import time
>> > end of the
>> > balance.
>>
>> I'm working on my presentation for EuroSciPy.  "Isn't so typical"
>> seems to be a good summary of my first slide.  :)
>>
>> >> Any chance of cutting down on the number, in order
>> >> to improve startup costs?
>> >
>> > Not at this point in time, no. That would break too much code.
>>
>> Understood.
>>
>> Thanks for the response,
>>
>>                                Andrew
>>                                dalke <at> dalkescientific.com
>>
>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion <at> scipy.org
>> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion <at> scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>



--
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Permalink | Reply | 0 comments |
headers
Andrew Dalke | 1 Jul 12:53
Gravatar

Re: "import numpy" is slow

2008/7/1 Hanni Ali <hanni.ali <at> gmail.com>:
> Would it not be possible to import just the necessary module of  
> numpy to
> meet the necessary functionality of your application.

Matthieu Brucher responded:
> IIRC, il you do import numpy.core as numpy, it starts by importing
> numpy, so it will be even slower.

which you can see if you start python with the "-v" option to display  
imports.

 >>> import numpy.core
import numpy # directory /Library/Frameworks/Python.framework/ 
Versions/2.5/lib/python2.5/site-packages/numpy
# /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/ 
site-packages/numpy/__init__.pyc matches /Library/Frameworks/ 
Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ 
__init__.py
import numpy # precompiled from /Library/Frameworks/Python.framework/ 
Versions/2.5/lib/python2.5/site-packages/numpy/__init__.pyc
# /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/ 
site-packages/numpy/__config__.pyc matches /Library/Frameworks/ 
Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ 
__config__.py
import numpy.__config__ # precompiled from /Library/Frameworks/ 
Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ 
__config__.pyc

   ...
and many more

				Andrew
				dalke <at> dalkescientific.com
Permalink | Reply | 0 comments |
headers
Robert Cimrman | 1 Jul 16:13
Picon

ANN: SfePy 00.46.02


I am pleased announce the release of SfePy 00.46.02.

SfePy is a finite element analysis software in Python, based primarily
on Numpy and SciPy.

Mailing lists, issue tracking, mercurial repository: http://sfepy.org
Home page: http://sfepy.kme.zcu.cz

Major improvements:
- alternative short syntax for specifying essential boundary conditions, 
variables and regions
- manufactured solutions tests:
     - SymPy support
- site configuration now via script/config.py + site_cfg.py
- new solvers
- new terms

For more information on this release, see
http://sfepy.googlecode.com/svn/web/releases/004602_RELEASE_NOTES.txt

If you happen to come to Leipzig for EuroSciPy 2008, see you there!

Best regards,
Robert Cimrman & SfePy developers
Permalink | Reply | 0 comments |
headers
Alan McIntyre | 1 Jul 17:26
Picon

Re: More pending test framework changes (please give feedback)

On Mon, Jun 30, 2008 at 1:54 PM, Alan McIntyre <alan.mcintyre <at> gmail.com> wrote:
> 1. All doctests in NumPy will have the numpy module available in their
> execution context as "np".
>
> 2. Turn on the normalized whitespace option for all doctests.  Having
> a doctest fail just because there's a space after your result seems
> like an unnecessary hassle for documenters.
>
> 3. Output will be ignored for each doctest expected output line that
> contains "#random". I figured this can serve both as an ignore flag
> and indication to the reader that the listed output may differ from
> what they see if they execute the associated command. So you would be
> able to do:
>>>> random.random()
> 0.1234567890                           #random: output may differ on your system
>
> And have the example executed but not cause a failure.  You could also
> use this to ignore the <SomeObject at 0x1234ABCD> output from plot
> methods as well.

Since I didn't see any objections, these changes are now committed.
I'll be updating some doctests to take advantage of them later today.

Alan
Permalink | Reply | 503 comments |
headers
Jarrod Millman | 1 Jul 19:41
Picon
Favicon
Gravatar

Re: More pending test framework changes (please give feedback)

On Tue, Jul 1, 2008 at 8:26 AM, Alan McIntyre <alan.mcintyre <at> gmail.com> wrote:
> Since I didn't see any objections, these changes are now committed.
> I'll be updating some doctests to take advantage of them later today.

Excellent.  Thanks,

--

-- 
Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014
http://cirl.berkeley.edu/
Permalink | Reply | 0 comments |

Gmane