Nick Eubank | 28 Nov 22:12 2014
Picon

Updates on NA / NaN values for Ints?

Hello Numpy!


Curious if there were any timelines for when NA / NaN support might arrive for Ints. 

(I'm sure this is asked and answered many times, but what I can find seems very out of date.) 

I'm strongly considering migrating from R/Stata/Matlab to Pandas/Numpy, but the inability to handle Na in Int types seems like a big limitation for someone always working with dirty data like me (a social scientist), and the Pandas docs suggest they're waiting for Numpy to add support. Just curious when that might happen! 

Thanks!

nick
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Robert Cimrman | 28 Nov 15:54 2014
Picon

ANN: SfePy 2014.4

I am pleased to announce release 2014.4 of SfePy.

Description
-----------

SfePy (simple finite elements in Python) is a software for solving systems of
coupled partial differential equations by the finite element method or by the
isogeometric analysis (preliminary support). It is distributed under the new
BSD license.

Home page: http://sfepy.org
Mailing list: http://groups.google.com/group/sfepy-devel
Git (source) repository, issue tracker, wiki: http://github.com/sfepy

Highlights of this release
--------------------------

- preliminary support for 1D problems
- data probes using pyVTK library

For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1
(rather long and technical).

Best regards,
Robert Cimrman and Contributors (*)

(*) Contributors to this release (alphabetical order):

Lubos Kejzlar, Vladimir Lukes
Alexander Belopolsky | 28 Nov 04:15 2014
Picon

Finding values in an array

I probably miss something very basic, but how given two arrays a and b, can I find positions in a where elements of b are located?  If a were sorted, I could use searchsorted, but I don't want to get valid positions for elements that are not in a.  In my case, a has unique elements, but in the general case I would accept the first match.  In other words, I am looking for an array analog of list.index() method.
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
David Cournapeau | 26 Nov 09:44 2014
Picon

Setting up a "newcomers" label on the issue tracker ?

Hi,

Would anybody mind if I create a label "newcomers" on GH, and start labelling simple issues ?

This is in anticipation to the bloomberg lab event in London this WE. I will try to give a hand to people interested in numpy/scipy,

David
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Yuxiang Wang | 26 Nov 05:06 2014

Implementation of tensor product and tensor contraction?

Dear all,

I have been doing tensor algebra recently (for continuum mechanics)
and was looking into two common operations: tensor product & tensor
contraction.

1. Tensor product

One common usage is:
a[i1, i2, i3, ..., iN, j1, j2, j3, ..., jM] = b[i1, i2, i3, ..., iN] *
c[j1, j2, j3, ..., jM]

I looked into the current np.outer(), and the only difference is that
it always flattens the input array. So actually, the function for
tensor product is simply

np.outer(a, b, out=out).reshape(a.shape+b.shape)  <-- I think I got
this right but please do correct me if I am wrong

Would anyone think it helpful or harmful to add such a function,
np.tensorprod()? it will simply be like
def tensorprod(a, b, out=None):
    return outer(a, b, out=out).reshape(a.shape+b.shape)

2. Tensor contraction

It is currently the np.tensordot(a, b) and it will do np.tensordot(a,
b, axes=2) by default. I think this is all great, but it would be even
better if we specify in the doc, that:
    i) say explicitly that by default it will be the double-dot or
contraction operator, and
    ii) explain that in cases where axes is an integer-like scalar,
which axes were selected from the two array and in what order. Like:
if axes is an integer-like scalar, it is the number axes to sum over,
equivalent to axes=(list(range(-axes, 0)), list(range(0, axes)))   (or
something like this)

It'd be great to hear what you would think about it,

Shawn

--

-- 
Yuxiang "Shawn" Wang
Gerling Research Lab
University of Virginia
yw5aj <at> virginia.edu
+1 (434) 284-0836
https://sites.google.com/a/virginia.edu/yw5aj/
Pauli Virtanen | 24 Nov 00:13 2014
Picon
Picon

ANN: Scipy 0.15.0 beta 1 release


Dear all,

We have finally finished preparing the Scipy 0.15.0 beta 1 release.
Please try it and report any issues on the scipy-dev mailing list,
and/or on Github.

If no surprises turn up, the final release is planned on Dec 20 in
three weeks.

Source tarballs and full release notes are available at
https://sourceforge.net/projects/scipy/files/SciPy/0.15.0b1/
Binary installers should also be up soon.

Best regards,
Pauli Virtanen

--------------------------------------------

SciPy 0.15.0 is the culmination of 6 months of hard work. It contains
several new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.16.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.2-3.3 and NumPy 1.5.1 or
greater.

New features
============

Linear Programming Interface
----------------------------

The new function ``scipy.optimize.linprog`` provides a generic
linear programming similar to the way ``scipy.optimize.minimize``
provides a generic interface to nonlinear programming optimizers.
Currently the only method supported is *simplex* which provides
a two-phase, dense-matrix-based simplex algorithm. Callbacks
functions are supported,allowing the user to monitor the progress
of the algorithm.

Differential_evolution, a global optimizer
------------------------------------------

A new ``differential_evolution`` function is available in the
``scipy.optimize``
module.  Differential Evolution is an algorithm used for finding the
global
minimum of multivariate functions. It is stochastic in nature (does
not use
gradient methods), and can search large areas of candidate space, but
often
requires larger numbers of function evaluations than conventional gradient
based techniques.

``scipy.signal`` improvements
-----------------------------

The function ``max_len_seq`` was added, which computes a Maximum
Length Sequence (MLS) signal.

``scipy.integrate`` improvements
--------------------------------

It is now possible to use ``scipy.integrate`` routines to integrate
multivariate ctypes functions, thus avoiding callbacks to Python and
providing better performance.

``scipy.linalg`` improvements
-----------------------------

Add function ``orthogonal_procrustes`` for solving the procrustes
linear algebra problem.

``scipy.sparse`` improvements
-----------------------------

``scipy.sparse.linalg.svds`` can now take a ``LinearOperator`` as its
main input.

``scipy.special`` improvements
------------------------------

Values of ellipsoidal harmonic (i.e. Lame) functions and associated
normalization constants can be now computed using ``ellip_harm``,
``ellip_harm_2``, and ``ellip_normal``.

New convenience functions ``entr``, ``rel_entr`` ``kl_div``,
``huber``, and ``pseudo_huber`` were added.

``scipy.sparse.csgraph`` improvements
-------------------------------------

Routines ``reverse_cuthill_mckee`` and ``maximum_bipartite_matching``
for computing reorderings of sparse graphs were added.

``scipy.stats`` improvements
----------------------------

Added a Dirichlet distribution as multivariate distribution.

The new function ``scipy.stats.median_test`` computes Mood's median test.

The new function ``scipy.stats.combine_pvalues`` implements Fisher's
and Stouffer's methods for combining p-values.

``scipy.stats.describe`` returns a namedtuple rather than a tuple,
allowing
users to access results by index or by name.

Deprecated features
===================

The ``scipy.weave`` module is deprecated.  It was the only module
never ported
to Python 3.x, and is not recommended to be used for new code - use Cython
instead.  In order to support existing code, ``scipy.weave`` has been
packaged
separately: `https://github.com/scipy/weave`_.  It is a pure Python
package, and
can easily be installed with ``pip install weave``.

``scipy.special.bessel_diff_formula`` is deprecated.  It is a private
function,
and therefore will be removed from the public API in a following release.

Backwards incompatible changes
==============================

scipy.ndimage
-------------

The functions ``scipy.ndimage.minimum_positions``,
``scipy.ndimage.maximum_positions`` and ``scipy.ndimage.extrema`` return
positions as ints instead of floats.

scipy.integrate
---------------

The format of banded Jacobians in ``scipy.integrate.ode`` solvers is
changed. Note that the previous documentation of this feature was
erroneous.

Ram Rachum | 22 Nov 20:39 2014

ANN: First release of combi, a combinatorics package for Python

Hi everyone,

Just a quick announcement: Last week I made the first release of an open-source package called Combi. It's a package for doing combinatorics in Python.

The pitch:

Combi lets you explore spaces of permutations and combinations as if they were Python sequences, but without generating all the permutations/combinations in advance. It lets you specify a lot of special conditions on these spaces. It also provides a few more classes that might be useful in combinatorics programming.

Information about it: 


I hope this package will be useful to people here.


Thanks,
Ram.
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Marek Wojciechowski | 20 Nov 18:47 2014
X-Face
Picon

return type from ufuncs

Hi!

I wrote a simple subclass of np.ndarray and now i do call np.sum() on it. I 
expected that the result will be a python float (or int) just like when summing 
up regular arrays. Instead i obtain the (scalar) view of my subclass. How can 
i change this behavior? I tried writing __array_wrap__ method like this: 

def __array_wrap__(self, out_arr, context=None):
	selfv = self.view(np.ndarray)
        return np.ndarray.__array_wrap__(selfv, out_arr, context)

but this just returns np.ndarray type and not float.

Regards,
--

-- 
Marek Wojciechowski 
Geoffrey M├ęgardon | 19 Nov 17:06 2014
Picon

Problem with _dotblas.pyd when using Matplotlib for 3d plot

Hi,

I am on Windows 8.1, 64-bit, using the usual version of Anaconda-64-bit for windows.

I just come from the mailing list of matplotlib: my PC is not able to do 3D plot with matplotlib.
Python crashes completely if I try to plot something in 3D and to show it: "Python.exe has stopped working..."

The main problem in resolving my issue is that the python traceback and the faulthandler package do not succeed in catching the errors.
Only the Windows crash report allows us to see what happened before the crash.
The windows report tells that the file _dotblas.pyd from numpy is likely responsible for the crash.

So I tried a simple:
import numpy; numpy.test()

numpy.test() does not work! I dont get any traceback from python. It just crashes as before: "Python.exe has stopped working..."

And again, the Windows report blames  _dotblas.pyd :)

Here the new Windows report for the numpy.test(), I highlighted the lines which blame _dotblas.pyd:
Version=1
EventType=APPCRASH
EventTime=130608138227608275
ReportType=2
Consent=1
UploadTime=130608138229728384
ReportIdentifier=216b16d2-6f5c-11e4-bec3-48d22435da2b
IntegratorReportIdentifier=216b16d1-6f5c-11e4-bec3-48d22435da2b
NsAppName=python.exe
Response.BucketId=398b7eee350a0fd8a7a96f705d3488f6
Response.BucketTable=4
Response.LegacyBucketId=85979911964
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=python.exe
Sig[1].Name=Application Version
Sig[1].Value=0.0.0.0
Sig[2].Name=Application Timestamp
Sig[2].Value=53b4679e
Sig[3].Name=Fault Module Name
Sig[3].Value=_dotblas.pyd

Sig[4].Name=Fault Module Version
Sig[4].Value=0.0.0.0
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=545678cb
Sig[6].Name=Exception Code
Sig[6].Value=c000001d
Sig[7].Name=Exception Offset
Sig[7].Value=0000000000324022
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=6.3.9600.2.0.0.768.101
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=2057
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=f32f
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=f32feb95f950f918532aa47b4372840e
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=9882
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=98823b6c7f579b24e92112ab827fe4a1
UI[2]=C:\Anaconda\python.exe
UI[3]=python.exe has stopped working
UI[4]=Windows can check online for a solution to the problem.
UI[5]=Check online for a solution and close the program
UI[6]=Check online for a solution later and close the program
UI[7]=Close the program
LoadedModule[0]=C:\Anaconda\python.exe
LoadedModule[1]=C:\WINDOWS\SYSTEM32\ntdll.dll
LoadedModule[2]=C:\WINDOWS\system32\KERNEL32.DLL
LoadedModule[3]=C:\WINDOWS\system32\KERNELBASE.dll
LoadedModule[4]=C:\Anaconda\python27.dll
LoadedModule[5]=C:\WINDOWS\WinSxS\amd64_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.8387_none_08e793bfa83a89b5\MSVCR90.dll
LoadedModule[6]=C:\WINDOWS\system32\USER32.dll
LoadedModule[7]=C:\WINDOWS\system32\ADVAPI32.dll
LoadedModule[8]=C:\WINDOWS\system32\SHELL32.dll
LoadedModule[9]=C:\WINDOWS\system32\GDI32.dll
LoadedModule[10]=C:\WINDOWS\system32\msvcrt.dll
LoadedModule[11]=C:\WINDOWS\SYSTEM32\sechost.dll
LoadedModule[12]=C:\WINDOWS\system32\RPCRT4.dll
LoadedModule[13]=C:\WINDOWS\SYSTEM32\combase.dll
LoadedModule[14]=C:\WINDOWS\system32\SHLWAPI.dll
LoadedModule[15]=C:\WINDOWS\system32\IMM32.DLL
LoadedModule[16]=C:\WINDOWS\system32\MSCTF.dll
LoadedModule[17]=C:\Anaconda\DLLs\_socket.pyd
LoadedModule[18]=C:\WINDOWS\system32\WS2_32.dll
LoadedModule[19]=C:\WINDOWS\system32\NSI.dll
LoadedModule[20]=C:\Anaconda\DLLs\_ssl.pyd
LoadedModule[21]=C:\Anaconda\DLLs\_ctypes.pyd
LoadedModule[22]=C:\WINDOWS\system32\ole32.dll
LoadedModule[23]=C:\WINDOWS\system32\OLEAUT32.dll
LoadedModule[24]=C:\Anaconda\DLLs\_hashlib.pyd
LoadedModule[25]=C:\WINDOWS\SYSTEM32\CRYPTSP.dll
LoadedModule[26]=C:\WINDOWS\system32\rsaenh.dll
LoadedModule[27]=C:\WINDOWS\SYSTEM32\bcrypt.dll
LoadedModule[28]=C:\WINDOWS\SYSTEM32\CRYPTBASE.dll
LoadedModule[29]=C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll
LoadedModule[30]=C:\Users\User\AppData\Roaming\Python-Eggs\faulthandler-2.4-py2.7-win-amd64.egg-tmp\faulthandler.pyd
LoadedModule[31]=C:\Anaconda\lib\site-packages\numpy\core\multiarray.pyd
LoadedModule[32]=C:\Anaconda\lib\site-packages\numpy\core\umath.pyd
LoadedModule[33]=C:\Anaconda\lib\site-packages\numpy\core\_dotblas.pyd
LoadedModule[34]=C:\Anaconda\lib\site-packages\numpy\core\libiomp5md.dll
LoadedModule[35]=C:\Anaconda\lib\site-packages\numpy\core\scalarmath.pyd
LoadedModule[36]=C:\Anaconda\lib\site-packages\numpy\lib\_compiled_base.pyd
LoadedModule[37]=C:\Anaconda\lib\site-packages\numpy\linalg\lapack_lite.pyd
LoadedModule[38]=C:\Anaconda\lib\site-packages\numpy\linalg\_umath_linalg.pyd
LoadedModule[39]=C:\Anaconda\lib\site-packages\numpy\fft\fftpack_lite.pyd
LoadedModule[40]=C:\Anaconda\lib\site-packages\numpy\random\mtrand.pyd
LoadedModule[41]=C:\Anaconda\DLLs\_multiprocessing.pyd
State[0].Key=Transport.DoneStage1
State[0].Value=1
FriendlyEventName=Stopped working
ConsentKey=APPCRASH
AppName=python.exe
AppPath=C:\Anaconda\python.exe
NsPartner=windows
NsGroup=windows8
ApplicationIdentity=5B036AF1EC2E20F320DBF28D119DE93D


Any idea, of what I can do?
I try to reinstall anaconda,
I also try conda remove numpy and conda install numpy.
​That does not seem to change anything.

--
--
MEGARDON Geoffrey
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
David Cournapeau | 18 Nov 19:20 2014
Picon

Numpy 1.9.1, zeros and alignement

Hi,

I have not followed closely the changes that happen in 1.9.1, but was surprised by the following:

x = np.zeros(12, "d")
assert x.flags.aligned # fails

This is running numpy 1.9.1 built on windows with VS 2008. Is it expected that zeros may return a non-aligned array ?

David
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Andrea Arteaga | 16 Nov 18:42 2014
Picon

Initializing array from buffer

Hello.
Using the numpy.frombuffer function [1] one can initialize a numpy array using an existing python object that implements the buffer protocol [2]. This is great, but currently this function supports only 1D buffers, even if the provided buffer is multidimensional and it exposes all information about its structure (shape, strides, data type).

Apparently, one can extract every kind of buffer information out of a buffer of a numpy array (pointer, number of dimensions, shape, strides, suboffsets,...), but the other way around is only partially implemented: providing a multidimensional buffer does not mean being able of creating a numpy array the uses that buffer with the desired structure.

My use case is the following: we have a some 3D arrays in our C++ framework. The ordering of the elements in these arrays is neither C nor Fortran style: it might be IJK (i.e. C style, 3rd dimension contiguous in memory), KJI (i.e. Fortran style, first dimension contiguous) or, e.g. IKJ. Moreover we put some padding to optimize aligned access. This kind of memory structure cannot be just expressed as 'C' or 'Fortran', but it can be perfectly expressed using the Python buffer protocol by providing the shape and the strides. We would like to export this structure to a numpy array that should be able of accessing the same memory locations in a consistent way and make some operations like initializing the content or plotting it.

Is this currently possible?
If not, is it planned to implement such a feature?

==========

Maybe just to clarify I could show an example entirely in python. Assume a in a 2D numpy array:

a = np.ones((10,20))

It contains information about its structure which can be portably accessed using its data member:

a.data.format == 'd'
a.data.ndim == 2
a.data.shape == (10,20)
a.data.strides == (160,8)

Unfortunately, when initializing an array b from this buffer, the structure of the buffer is "downgraded" to unidimensional shape:

b = np.frombuffer(a.data)

b.ndim == 1
b.shape == (200,)
b.strides == (8,)

I wished b had the same multi-dimensional structure of a.

(This is of course a very simple example. In my use case I would initialize b with my own buffer instead of that of another numpy array).

Best regards
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion <at> scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Gmane