James Turner | 1 Jun 02:11
Favicon

Re: GPU implementation?

Hi Martin,

 > I was wondering if anyone has thought about accelerating NumPy with a
 > GPU. For example nVidia's CUDA SDK provides a feasible way to offload
 > vector math onto the very fast SIMD processors available on the GPU.
 > Currently GPUs primarily support single precision floats and are not
 > IEEE compliant, but still could be useful for some applications.

I wasn't actually there, but I noticed that last year's SciPy
conference page includes a talk entitled "GpuPy: Using GPUs to
Accelerate NumPy", by Benjamin Eitzen (I think I also found his Web
page via Google):

   http://www.scipy.org/SciPy2006/Schedule

I also wondered whether Benjamin or anyone else who is interested had
come across the Open Graphics Project (hadn't got around to asking)?

   http://wiki.duskglow.com/tiki-index.php?page=open-graphics

This would be quite a specialized combination, but I'm sure it could
be useful to some people with high performance requirements or maybe
building some kind of special appliances.

Cheers,

James.
Andrew Corrigan | 1 Jun 02:49
Picon
Favicon

Re: GPU implementation?

Martin Ünsal <martinunsal <at> gmail.com> writes:

> 
> I was wondering if anyone has thought about accelerating NumPy with a
> GPU. For example nVidia's CUDA SDK provides a feasible way to offload
> vector math onto the very fast SIMD processors available on the GPU.
> Currently GPUs primarily support single precision floats and are not
> IEEE compliant, but still could be useful for some applications.
> 
> If there turns out to be a significant speedup over using the CPU, this
> could be a very accessible way to do scientific and numerical
> computation using GPUs, much easier than coding directly to the GPU APIs.
> 
> Martin
> 

I've thought about this too and think that it's a great idea. The existing
library Brook, which has a similar programming model to NumPy, proves that it's
feasible.  And Brook was originally done with OpenGL & DirectX as backends to
access the hardware.  Needless to say, that's a lot harder than using CUDA. 
Since it hasn't already been pointed out, CUDA includes the cuBLAS and cuFFT
libraries.  I don't what the status of a LAPACK built on top of the cuBLAS is,
but I'd be surprised if someone isn't already working on it.  Also, NVIDIA has
stated that double-precision hardware will be available later this year, in case
that's an issue for anyone.

I agree very much that it would make the GPUs more accessible, although CUDA has
done an amazing job at that already.  I think the most helpful thing about this
would be if it allowed us to code using the existing array interface from NumPy
in a way that the code automatically runs on the GPU in an optimized way - using
(Continue reading)

David Cournapeau | 1 Jun 02:52
Picon
Picon

Re: ATLAS,LAPACK compilation - help!

Robert Kern wrote:
> sittner <at> lkb.ens.fr wrote:
>> Hello there,
>> I'm new here, so excuse me if the solution is trivial:
>> i have installed ATLAS and LAPACK on my ubuntu 7 dual core intel machine.
>> now, when i try to install numpy, it tells me it doesn't find these
>> libraries:
>>
>> "
>> $ python setup.py install
>> Running from numpy source directory.
>> F2PY Version 2_3816
>> blas_opt_info:
>> blas_mkl_info:
>>   libraries mkl,vml,guide not found in /usr/local/lib
>>   libraries mkl,vml,guide not found in /usr/lib
>>   NOT AVAILABLE
>>
>> atlas_blas_threads_info:
>> Setting PTATLAS=ATLAS
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS/src/
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS
>>   libraries lapack,blas not found in /usr/local/lib
>>   libraries lapack,blas not found in /usr/lib
>>   NOT AVAILABLE
>>
>> atlas_blas_info:
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS/src/
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS
>>   libraries lapack,blas not found in /usr/local/lib
(Continue reading)

Brian Granger | 1 Jun 04:36
Picon

Re: GPU implementation?

This is very much worth pursuing.  I have been working on things
related to this on and off at my day job.  I can't say specifically
what I have been doing, but I can make some general comments:

* It is very easy to wrap the different parts of cude using ctypes and
call it from/numpy.

* Compared to a recent fast Intel CPU, the speedups we see are
consistent with what the NVIDIA literature reports:   10-30x is common
and in some cases we have seen up to 170x.

* Certain parts of numpy will be very easy to accelerate:  things
covered by blas, ffts, and ufuncs, random variates -  but each of
these will have very different speedups.

* LAPACK will be tough, extremely tough in some cases.  The main issue
is that various algorithms in LAPACK rely on different levels of BLAS
(1,2, or 3).  The algorithms in LAPACK that primarily use level 1 BLAS
functions (vector operations), like LU-decomp, are probably not worth
porting to the GPU - at least not using the BLAS that NVIDIA provides.
 On the other hand, the algorithms that use more of the level 2 and 3
BLAS functions are probably worth looking at.

* NVIDIA made a design decision in its implementation of cuBLAS and
cuFFT that is somewhat detrimental for certain algorithms.  In their
implementation, the BLAS and FFT routines can _only_ be called from
the CPU, not from code running on the GPU.  Thus if you have an
algorithm that makes many calls to cuBLAS/cuFFT, you pay a large
overhead in having to keep the main flow of the algorithm on the CPU.
It is not uncommon for this overhead to completely erode any speedup
(Continue reading)

Charles R Harris | 1 Jun 04:38
Picon

Re: ATLAS,LAPACK compilation - help!



On 5/31/07, David Cournapeau <david <at> ar.media.kyoto-u.ac.jp> wrote:
Robert Kern wrote:
> sittner <at> lkb.ens.fr wrote:
>> Hello there,
>> I'm new here, so excuse me if the solution is trivial:
>> i have installed ATLAS and LAPACK on my ubuntu 7 dual core intel machine.
>> now, when i try to install numpy, it tells me it doesn't find these
>> libraries:
>>
>> "
>> $ python setup.py install
>> Running from numpy source directory.
>> F2PY Version 2_3816
>> blas_opt_info:
>> blas_mkl_info:
>>   libraries mkl,vml,guide not found in /usr/local/lib
>>   libraries mkl,vml,guide not found in /usr/lib
>>   NOT AVAILABLE
>>
>> atlas_blas_threads_info:
>> Setting PTATLAS=ATLAS
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS/src/
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS
>>   libraries lapack,blas not found in /usr/local/lib
>>   libraries lapack,blas not found in /usr/lib
>>   NOT AVAILABLE
>>
>> atlas_blas_info:
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS/src/
>>   libraries lapack,blas not found in /usr/local/lib/ATLAS
>>   libraries lapack,blas not found in /usr/local/lib
>>   libraries lapack,blas not found in /usr/lib
>>   NOT AVAILABLE
>> ......"
>> I have installed ATLAS and lapack with no errors.
>> ATLAS is in usr/local/lib/ATLAS/:
>> $ ls /usr/local/lib/ATLAS
>> bin       doc          interfaces  Make.Linux_UNKNOWNSSE2_2   README  tune
>> CONFIG    include      lib         makes                     src
>> config.c  INSTALL.txt  Makefile    Make.top                  tst.o
>>
>> so, what seems to be the problem?
>
> You haven't actually installed ATLAS. You've just built it. Don't put the source
> in /usr/local/lib/ATLAS/. Put that somewhere else, like ~/src/ATLAS/. Follow the
> installation instructions in INSTALL.txt. Note this section:
>
> """
> There are two mandatory steps to ATLAS installation (config & build), as
> well as three optional steps (test, time, install) and these steps are
> described in detail below.  For the impatient, here is the basic outline:
> **************************************************
>    mkdir my_build_dir ; cd my_build_dir
>    /path/to/ATLAS/configure [flags]
>    make              ! tune and compile library
>    make check        ! perform sanity tests
>    make ptcheck      ! checks of threaded code for multiprocessor systems
>    make time         ! provide performance summary as % of clock rate
>    make install      ! Copy library and include files to other directories
> **************************************************
> """
>
Alternatively, if you are not familiar with compiling softwares (and
Atlas can be tricky to compile/install), just install the packages
provided by ubuntu: sudo apt-get install atlas3-sse2-dev
atlas3-base-dev, and it should be fine.

Maybe, maybe not. On 64bit Intel machines running 64bit linux the fedora package raises an illegal instruction error. Since the fedora package is based on the debian package this might be a problem on Ubuntu also. For recent hardware you are probably better off compiling your own from the latest ATLAS version out there.

Chuck


_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
David Cournapeau | 1 Jun 05:14
Picon
Picon

numpy.distutils, system_info and dll/exp/lib

Hi,

    In one of my package, I use numpy.distutils.system_info to detect a 
C library. This works well on linux and Mac OS X, but on windows, the 
library is not detected if only the dll is present. If the .lib is 
there, then it is detected; the dll itself works OK (I can use it from 
ctypes normally). Am I doing something wrong in my configuration ?

"""
class SndfileNotFoundError(NotFoundError):
    """ sndfile (http://www.mega-nerd.com/libsndfile/) library not found.
    Directories to search for the libraries can be specified in the
    site.cfg file (section [sndfile])."""

class sndfile_info(system_info):
    #variables to override
    section         = 'sndfile'
    notfounderror   = SndfileNotFoundError
    libname         = 'sndfile'
    header          = 'sndfile.h'

    def __init__(self):
        system_info.__init__(self)

    def calc_info(self):
        """ Compute the informations of the library """
        prefix  = 'lib'

        # Look for the shared library
        sndfile_libs    = self.get_libs('sndfile_libs', self.libname)
        lib_dirs        = self.get_lib_dirs()
        for i in lib_dirs:
            tmp = self.check_libs(i, sndfile_libs)
            if tmp is not None:
                info    = tmp
                break
        #else:
        #    raise self.notfounderror()

        # Look for the header file
        include_dirs    = self.get_include_dirs()
        inc_dir         = None
        for d in include_dirs:
            p = self.combine_paths(d,self.header)
            if p:
                inc_dir     = os.path.dirname(p[0])
                headername  = os.path.abspath(p[0])
                break
        if inc_dir is not None and tmp is not None:
            if sys.platform == 'win32':
                # win32 case
                fullname    = prefix + tmp['libraries'][0] + \
                        '.dll'
            elif sys.platform == 'darwin':
                # Mac Os X case
                fullname    = prefix + tmp['libraries'][0] + '.' + \
                        str(SNDFILE_MAJ_VERSION) + '.dylib'
            else:
                # All others (Linux for sure; what about solaris) ?
                fullname    = prefix + tmp['libraries'][0] + '.so' + \
                        '.' + str(SNDFILE_MAJ_VERSION)
            fullname    = os.path.join(info['library_dirs'][0], fullname)
            dict_append(info, include_dirs=[inc_dir],
                    fullheadloc = headername,
                    fulllibloc  = fullname)
        else:
            return

        #print self
        self.set_info(**info)
        return
"""

    The problem is, the library I am using is distributed only as a dll, 
not with the .lib. Right now, users of my package need to generate the 
.lib by themselves from the dll, using visual studio compiler, which is 
really too much (specially since the library itself is fine; using the 
compiler is necessary only for distutils to find the library !).

    cheers,

    David
David Cournapeau | 1 Jun 05:16
Picon
Picon

Re: ATLAS,LAPACK compilation - help!

Charles R Harris wrote:
>
>
> Maybe, maybe not. On 64bit Intel machines running 64bit linux the 
> fedora package raises an illegal instruction error. Since the fedora 
> package is based on the debian package this might be a problem on 
> Ubuntu also. For recent hardware you are probably better off compiling 
> your own from the latest ATLAS version out there.
>
Well, testing the deb packages would take a couple of minutes, whereas 
compiling and installing atlas would take easily much more, specially if 
you have never done it before. So I think trying the debian packages 
first is the least effort path :)

David
Matthieu Brucher | 1 Jun 08:22
Picon

Re: ATLAS,LAPACK compilation - help!

Maybe, maybe not. On 64bit Intel machines running 64bit linux the fedora package raises an illegal instruction error. Since the fedora package is based on the debian package this might be a problem on Ubuntu also. For recent hardware you are probably better off compiling your own from the latest ATLAS version out there.

Red Hat uses Debian packages ? That sounds odd... FC uses RPM, Debian uses deb packages. The problem with RPM is, as stated by David some time ago, that a lot of info is missing in RPM that is present in deb. What is more, it is known that using a lot of packages repositories leads to incompatibilities and instabilities.

Matthieu
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
David Cournapeau | 1 Jun 08:26
Picon
Picon

Re: ATLAS,LAPACK compilation - help!

Matthieu Brucher wrote:
>
>     Maybe, maybe not. On 64bit Intel machines running 64bit linux the
>     fedora package raises an illegal instruction error. Since the
>     fedora package is based on the debian package this might be a
>     problem on Ubuntu also. For recent hardware you are probably
>     better off compiling your own from the latest ATLAS version out
>     there.
>
>
> Red Hat uses Debian packages ? That sounds odd... FC uses RPM, Debian 
> uses deb packages. The problem with RPM is, as stated by David some 
> time ago, that a lot of info is missing in RPM that is present in deb.
I don't think I stated that :)
> What is more, it is known that using a lot of packages repositories 
> leads to incompatibilities and instabilities.
I think what Harris meant is that the rpm package is based on the work 
done by the debian packagers. There are official rpm for atlas, I think 
(not sure, as I do not use Fedora on a regular basis).

David
Matthieu Brucher | 1 Jun 08:44
Picon

Re: ATLAS,LAPACK compilation - help!

> Red Hat uses Debian packages ? That sounds odd... FC uses RPM, Debian
> uses deb packages. The problem with RPM is, as stated by David some
> time ago, that a lot of info is missing in RPM that is present in deb.
I don't think I stated that :)


Well you said, IIRC, that you had troubles making rpm, no ? If that's not the case, my apologies, I must be confusing with another discussion somewhere else :|
 

> What is more, it is known that using a lot of packages repositories
> leads to incompatibilities and instabilities.
I think what Harris meant is that the rpm package is based on the work
done by the debian packagers. There are official rpm for atlas, I think
(not sure, as I do not use Fedora on a regular basis).

Oh... OK

Matthieu
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion <at> scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Gmane