Matti Picus | 4 Jun 22:58 2016
Picon

PyArray_Scalar should not use memcpy

Hi. This is a heads up and RFC about a pull request I am preparing for 
PyArray_Scalar, within the framework of getting NumPy working properly 
on PyPy. For those who don't know, the numpy HEAD builds and runs on 
PyPy2.7 HEAD (otherwise known as nightly default). However there are a 
number of test failures, some are caused by (ab)use of memcpy on c-level 
pointers obtained from Py*_FromString().

I am currently rewriting PyArray_Scalar to not use memcpy, and wondering 
how deep of a refactoring would be acceptable by the maintainers in a 
single pull request? Should I just stick to small changes to eliminate 
the two calls to memcpy, or clean up and restructure the entire function 
around a more switch(type_num) programming style?

Thanks,
Matti
Hearne, Mike | 2 Jun 23:30 2016
Picon

SciPy 2016

I am one of the co-chairs of the Birds of a Feather (BOF) committee at
SciPy 2016, taking place this year from July 11-17.

We are actively seeking moderators to propose BOF sessions on any
number of topics.  If someone on this list (a numpy dev, perhaps?) is
interested in leading a numpy-focused session at SciPy this year, the
submission form can be found here:

http://scipy2016.scipy.org/ehome/146062/332970/

The BOF can take the form of a panel or open discussion.  We have one
submission already for a BOF focused on future plans for matplotlib
development, so a parallel session on numpy future development is sure
to be of general interest.'

Thanks, and we'll see some of you in Austin next month!

--Mike Hearne
Nathaniel Smith | 2 Jun 04:42 2016
Picon

Re: Changing FFT cache to a bounded LRU cache

On Jun 1, 2016 4:47 PM, "David Cournapeau" <cournape <at> gmail.com> wrote:
>
>
>
> On Tue, May 31, 2016 at 10:36 PM, Sturla Molden <sturla.molden <at> gmail.com> wrote:
>>
>> Joseph Martinot-Lagarde <contrebasse <at> gmail.com> wrote:
>>
>> > The problem with FFTW is that its license is more restrictive (GPL), and
>> > because of this may not be suitable everywhere numpy.fft is.
>>
>> A lot of us use NumPy linked with MKL or Accelerate, both of which have
>> some really nifty FFTs. And the license issue is hardly any worse than
>> linking with them for BLAS and LAPACK, which we do anyway. We could extend
>> numpy.fft to use MKL or Accelerate when they are available.
>
>
> That's what we used to do in scipy, but it was a PITA to maintain. Contrary to blas/lapack, fft does not have a standard API, hence exposing a consistent API in python, including data layout involved quite a bit of work.
>
> It is better to expose those through 3rd party APIs.

Fwiw Intel's new python distribution thing has numpy patched to use mkl for fft, and they're interested in pushing the relevant changes upstream.

I have no idea how maintainable their patches are, since I haven't seen them -- this is just from taking to people here at pycon.

-n

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Sebastian Berg | 31 May 17:37 2016
Picon

Developer Meeting at EuroScipy?

Hi all,

since we had decided to do a regular developers meeting last year, how
would EuroScipy (Aug. 23.-27., Erlangen, Germany) look as a possible
place and time to have one?
I believe EuroScipy would include a few people who were not able to
come to SciPy last year, and it seems SciPy itself already sees some of
the devs quite busy anyway.

Not having checked back for room availability at the conference or
anything, one option may be:

August 24. (parallel to the second/last day of Tutorials)

Does this seem like a good plan or do you have alternative suggestions
or scheduling difficulties with this?

Regards,

Sebastian
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Pierre de Buyl | 31 May 15:05 2016
Picon

EuroSciPy 2016

Dear NumPy and SciPy communities,

No annoucement was made here for EuroSciPy 2016 I believe. The call for
contributions (talks, posters, sprints) is still open for a few days.

EuroSciPy 2016 takes place in Erlangen, Germany, from the 23 to the 27 of August
and consists of two days of tutorials (beginner and advanced tracks) and two
days of conference representing many fields of science, with a focus on Python
tools for science. A day of sprints follows (sprints TBA).

The keynote speakers are Gaƫl Varoquaux and Abby Cabunoc Mayes and we can expect
a rich tutorial and scientific program! Videos from previous years are available
at https://www.youtube.com/playlist?list=PLYx7XA2nY5GeQCCugyvtnHMVLdhYlrRxH and
https://www.youtube.com/playlist?list=PLYx7XA2nY5Gcpabmu61kKcToLz0FapmHu

Visit us, register and submit an abstract on our website!
https://www.euroscipy.org/2016/

SciPythonic regards,

The EuroSciPy 2016 team
Scott Sievert | 29 May 05:53 2016
Picon
Gravatar

ENH: compute many inner products quickly

I recently ran into an application where I had to compute many inner products quickly (roughy 50k inner products in less than a second). I wanted a vector of inner products over the 50k vectors, or `[x1.T <at> A <at> x1, …, xn.T <at> A <at> xn]` with A.shape = (1k, 1k).

My first instinct was to look for a NumPy function to quickly compute this, such as np.inner. However, it looks like np.inner has some other behavior and I couldn’t get tensordot/einsum to work for me.

Then a labmate pointed out that I can just do some slick matrix multiplication to compute the same quantity, `(X.T * A <at> X.T).sum(axis=0)`. I opened [a PR] with this, and proposed that we define a new function called `inner_prods` for this.

However, in the PR, <at> shoyer pointed out 

> The main challenge is to figure out how to transition the behavior of all these operations, while preserving backwards compatibility. Quite likely, we need to pick new names for these functions, though we should try to pick something that doesn't suggest that they are second class alternatives.

Do we choose new function names? Do we add a keyword arg that changes what np.inner returns?




_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Stephan Hoyer | 29 May 04:35 2016
Picon
Gravatar

NumPy 1.11 docs

These still are missing from the SciPy.org page, several months after the release.

What do we need to do to keep these updated? Is there someone at Enthought we should ping? Or do we really just need to transition to different infrastructure?
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Lion Krischer | 27 May 22:51 2016
Picon

Changing FFT cache to a bounded LRU cache

Hi all,

I was told to take this to the mailing list. Relevant pull request:
https://github.com/numpy/numpy/pull/7686

NumPy's FFT implementation caches some form of execution plan for each
encountered input data length. This is currently implemented as a simple
dictionary which can grow without bounds. Calculating lots of different
FFTs thus cause a memory leak from the users' perspective. We
encountered a real world situation where this is an issue.

The PR linked above proposes to replace the simple dictionary with an
LRU (least recently used) cache. It will remove the least recently used
pieces of data if it grows beyond a specified size (currently an
arbitrary limit of 100 MB per cache). Thus almost all users will still
benefit from the caches but their total memory size is now limited.

Things to consider:

* This adds quite some additional complexity but I could not find a
simple way to achieve the same result.
* What is a good limit on cache size? I used 100 MB because it works for
my uses cases.

Cheers!

Lion
Leon Woo | 26 May 14:02 2016

AUTO: Leon Woo is out of the office (returning 06/06/2016)

I am out of the office until 06/06/2016.

For standard requests within the scope of EMG PWM Berlin, please write to EMG PWM Berlin <at> DBEMEA.
For non-standard requests, please cc Hien Pham-Thu.


Note: This is an automated response to your message  "NumPy-Discussion Digest, Vol 116, Issue 39" sent on 26.05.2016 14:00:01.

This is the only notification you will receive while this person is away.

--

Informationen (einschließlich Pflichtangaben) zu einzelnen, innerhalb der EU tätigen Gesellschaften und Zweigniederlassungen des Konzerns Deutsche Bank finden Sie unter http://www.deutsche-bank.de/de/content/pflichtangaben.htm. Diese E-Mail enthält vertrauliche und/ oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for information (including mandatory corporate particulars) on selected Deutsche Bank branches and group companies registered or incorporated in the European Union. This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Allen Welkie | 25 May 21:35 2016
Picon

Fwd: ifft padding

I'd like to get some feedback on my [pull request](https://github.com/numpy/numpy/pull/7593).

This pull request adds a function `ifftpad` which pads a spectrum by inserting zeros where the highest frequencies would be. This is necessary because the padding that `ifft` does simply inserts zeros at the end of the array. But because of the way the spectrum is laid out, this changes which bins represent which frequencies and in general messes up the result of `ifft`. If you pad with the proposed `ifftpad` function, the zeros will be inserted in the middle of the spectrum and the time signal that results from `ifft` will be an interpolated version of the unpadded time signal. See the discussion in [issue #1346](https://github.com/numpy/numpy/issues/1346).

The following is a script to demonstrate what I mean:

```
import numpy
from numpy import concatenate, zeros
from matplotlib import pyplot

def correct_padding(a, n, scale=True):
    """ A copy of the proposed `ifftpad` function. """
    spectrum = concatenate((a[:len(a) // 2],
                            zeros(n - len(a)),
                            a[len(a) // 2:]))
    if scale:
        spectrum *= (n / len(a))
    return spectrum

def plot_real(signal, label):
    time = numpy.linspace(0, 1, len(signal) + 1)[:-1]
    pyplot.plot(time, signal.real, label=label)

def main():
    spectrum = numpy.zeros(10, dtype=complex)
    spectrum[-1] = 1 + 1j

    signal = numpy.fft.ifft(spectrum)
    signal_bad_padding = numpy.fft.ifft(10 * spectrum, 100)
    signal_good_padding = numpy.fft.ifft(correct_padding(spectrum, 100))

    plot_real(signal, 'No padding')
    plot_real(signal_bad_padding, 'Bad padding')
    plot_real(signal_good_padding, 'Good padding')

    pyplot.legend()
    pyplot.show()


if __name__ == '__main__':
    main()
```

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
G Young | 22 May 03:15 2016
Picon

Re: broadcasting for randint

Hi,

I have had a PR open for quite some time now that allows arguments to broadcast in randint.  While the functionality is fully in-place and very robust, the obstacle at this point is the implementation.

When the dtype parameter was added to randint (see here), a big issue with the implementation was that it created so much duplicate code that it would be a huge maintenance nightmare.  However, this was dismissed in the original PR message because it was believed that template-ing would be trivial, which seemed reasonable at the time.

When I added broadcasting, I introduced a template system to the code that dramatically cut down on the duplication.  However, the obstacle has been whether or not this template system is too ad hoc to be merged into the library.  Implementing a template in Cython was not considered sufficient and is in fact very tricky to do, and unfortunately, I have not received any constructive suggestions from maintainers about how to proceed, so I'm opening this up to the mailing to see whether or not there are better alternatives to what I did, whether this should be merged as it, or whether this should be tabled until a better template can be found.

Thanks!
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Gmane