Picon

Calling C code that assumes SIMD aligned data.

Hi!

I've written a little code of numpy code that does a neural network feedforward calculation:

    def feedforward(self,x):
        for activation, w, b in zip( self.activations, self.weights, self.biases ):
            x = activation( np.dot(w, x) + b)

This works fine when my activation functions are in Python, however I've wrapped the activation functions from a C implementation that requires the array to be memory aligned. (due to simd instructions in the C implementation.) So I need the operation np.dot( w, x) + b to return a ndarray where the data pointer is aligned. How can I do that? Is it possible at all?

(BTW: the function works  correctly about 20% of the time I run it, and else it segfaults on the simd instruction in the the C function)

Thanks,
-Øystein
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Honi Sanders | 3 May 17:43 2016
Picon

Cross-correlation PR stuck in limbo

Hello all,
I have completed a pull request to add a “maxlag” functionality to numpy.correlate.  See here: https://github.com/numpy/numpy/pull/5978.  This pull request has passed all tests and has been ready to be merged for around six months.  Several people have commented requesting for it to be included on stackoverflow, the listserve, and github.  Can someone please let me know what needs to be done or can it be merged?

Here is some background:

What was troubling me is that numpy.correlate does not have a maxlag feature. This means that even if I only want to see correlations between two time series with lags between -100 and +100 ms, for example, it will still calculate the correlation for every lag between -20000 and +20000 ms (which is the length of the time series). This (theoretically) gives a 200x performance hit! 

I have introduced this question as a numpy issue, a scipy issue and on the scipy-dev list. It seems the best place to start is with numpy.correlate, so that is what I am requesting. 

Previous discussion of this functionality can be found at another discussion on numpy correlate (and convolution). Other issues related to correlate functions include ENH: Fold fftconvolve into convolve/correlate functions as a parameter #2651Use FFT in np.correlate/convolve? (Trac #1260) #1858, and normalized cross-correlation (Trac #1714) #2310.



The new implementation allows new types of the “mode” argument, to include an int value, which defines the maximum lag for which cross-correlation should be calculated, or a tuple, which defines the minlag, maxlag, and lagstep to be used in the same format as the arguments to numpy.arange.


Please let me know what should be done to move this pull request forward.

Honi



_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
David Morris | 2 May 11:51 2016
Picon

Unable to pickle numpy array on iOS

I have an application running on iOS where I pickle a numpy array in order to save it for later use.  However, I receive the following error:

pickle.dumps(arr)
...
_pickle.PicklingError: Can't pickle <built-in function _reconstruct>: import of module 'multiarray' failed

On a desktop system (OSX), there is no problem dumping the array.

I am using NumPy v1.9.3

Any ideas on why this might be happening?

Thank you,
David

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Henrique Almeida | 29 Apr 18:31 2016
Picon

Re: Help with bit arrays

 Paul, yes, imread() worked for reading the black and white TIFF. The
situation improved, but now, there seems to be some problem with the
color map. Example code:

#!/usr/bin/env python3
import numpy
from matplotlib import pyplot, cm

img = pyplot.imread('oi-00.tiff')
pyplot.imshow(img)
pyplot.colorbar()
pyplot.show()

 The code can open both 1-bit and 8-bit images, but only with 8 bits
the image is shown with the colormap colors. The 1 bit image is shown
as black and white.

 The questions:
 1) Should Image.open() behave like pyplot.imread() ? Is this a bug in PIL ?
 2) Why isn't the colormap working with black and white images ?

2016-04-29 13:06 GMT-03:00 Paul Hobson <pmhobson <at> gmail.com>:
> Does using pyplot.imgread work?
>
> On Fri, Apr 29, 2016 at 8:27 AM, Henrique Almeida <hdante.lnls <at> gmail.com>
> wrote:
>>
>>  Any help with this problem ?
>>
>> 2016-04-27 11:35 GMT-03:00 Henrique Almeida <hdante.lnls <at> gmail.com>:
>> >  Hello, what's the current status on numpy for loading bit-arrays ?
>> >
>> > I'm currently unable to correctly load black and white (1-bit) TIFF
>> > images. Code example follows:
>> >
>> > from PIL import Image
>> > import numpy
>> > from matplotlib import pyplot
>> >
>> > img = Image.open('oi-00.tiff')
>> > a = numpy.array(img)
>> >
>> > ^ does not work for 1-bit TIFF images
>> >
>> > PIL source shows that it incorrectly uses typestr == '|b1'. I tried to
>> > change this to '|t1', but I get :
>> >
>> > TypeError: data type "|t1" not understood
>> >
>> > My goal is to make the above code to work for black and white TIFF
>> > images the same way it works for grayscale images. Any help ?
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion <at> scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion <at> scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
Elliot Hallmark | 28 Apr 01:48 2016
Picon

Is this a known error in Ipython with numpy?

Hello,

I haven't worked hard yet to create a minimal runnable (reproduce-able) example but I wanted to check if this sounds familiar to anyone.

I have a pretty involved program that resizes arrays in place with arr.resize.  When I run it with python it completes and gives the expected result.  When I run it in Ipython, I get the following error:

```
    ---> 43         self._buffer.resize((count,)+self._dim)
         44
         45

    ValueError: cannot resize an array that references or is referenced
    by another array in this way.  Use the resize function
```

It was consistently doing this on the same array, after resizing two others before hand, even after rebooting.  But trying to track it down it goes away even if I undo everything I did to try and track it down.

Does this sound familiar?
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Henrique Almeida | 27 Apr 16:35 2016
Picon

Help with bit arrays

 Hello, what's the current status on numpy for loading bit-arrays ?

I'm currently unable to correctly load black and white (1-bit) TIFF
images. Code example follows:

from PIL import Image
import numpy
from matplotlib import pyplot

img = Image.open('oi-00.tiff')
a = numpy.array(img)

^ does not work for 1-bit TIFF images

PIL source shows that it incorrectly uses typestr == '|b1'. I tried to
change this to '|t1', but I get :

TypeError: data type "|t1" not understood

My goal is to make the above code to work for black and white TIFF
images the same way it works for grayscale images. Any help ?
Saumyajit Dey | 26 Apr 11:35 2016
Picon

(no subject)

Hi,

This is Saumyajit Dey and I am looking forward to start contributing to NumPy. 

I have never contributed to any open source projects before so I would want to know some tips and guidelines to start contributing.

Regards,
Saumyajit

Saumyajit Dey
Junior Undergraduate Student:
Department of Computer Science and Engineering
National Institute of Technology
Warangal (NITW), India
Cell: +91-8885847028
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Christian Aichinger | 25 Apr 23:25 2016

Adding the Linux Wheels for old releases breaks builds

Hi!
The addition of the Linux Wheels broke the build process of several of our Debian packages, which rely on
NumPy installed inside virtualenvs. The problem stems from the pre-compiled shared libraries included
in the Wheels, details are in <https://github.com/numpy/numpy/issues/7570>.

I'm bringing this up here because these changes have implications that may not have been fully realized before.

The Wheel packages are great for end users, they make NumPy much more easily installable for average
people. Unfortunately, they are precisely the wrong thing for anyone re-packaging NumPy (e.g. shipping
it in a virtualenv inside RPM or Debian packages). For that use-case, you typically want to build NumPy
yourself.[1] You could rely on this happening before, now a `--no-binary` argument for pip is needed to
get that behavior. Put another way, the addition of the Wheels silently invalidated the assumption that a
`pip install numpy` would locally compile the package.

In the perfect world, anyone re-packaging NumPy would specify `--no-binary` if they want to enforce local
building. However, currently, --no-binary is not in widespread use because it was never necessary
before. 

I fully agree that the Wheels have great value, but adding them for old releases (back to 1.6.0 from 2011)
suddenly changes the NumPy distribution for people who explicitly pinned an older version to avoid
surprises. It invites downstream build failures (as happened to us) and adds externally-built shared
objects in a way that people won't expect.

I would propose to only add Wheels for new releases and to explicitly mention this issue in the release
notes, so people are not blind-sided by it. I realize that this would be a painfully slow process, but
silently breaking previously working use-cases for old releases seems worse to me (though it is
difficult to estimate how many people are negatively affected by this).

Regards,
Chris

[1]: You want to build locally for many reasons, e.g. to link against your system's libraries which you get
security upgrades for; to have the confidence that you can actually build the package from source if need
be; to be sure that the binaries really correspond to the source code;
...

------------------------------------------------------------------------------------------------------------------- 
UBIMET GmbH - weather matters 
Christian Aichinger • IT 

A-1220 Wien • Donau-City-Straße 11 • Tel +43 1 263 11 22 • Fax +43 1 263 11 22 219 
caichinger <at> ubimet.com • www.ubimet.com
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Steve Mitchell | 20 Apr 21:22 2016

Do getitem/setitem already have GIL?

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Chris Barber | 19 Apr 20:12 2016
Picon
Gravatar

nan version of einsum

Is there any interest in a nan-ignoring version of einsum a la nansum, nanprod, etc?  Any idea how difficult it would be to implement?

- Chris
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Charles R Harris | 16 Apr 23:34 2016
Picon

Preparing for 1.12 branch

Hi All,

This is just a request that numpy reviewers tag PRs that they think merit inclusion in 1.12 with `1.12.0 release`. The tag doesn't mean that the PR need be in 1.12, but it will help prioritize the review process.

Chuck
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Gmane