Sri K | 8 May 21:09 2014
Picon

Re: apply, applymap, map, still confused

Can I apply applymap or apply on elements of only one column while ignoring the rest? I also want the data applied so that the original dataframe is changed. Is this possible?

On Wednesday, March 21, 2012 8:25:23 AM UTC-7, Aman wrote:
Hi Mihai,

I've whipped up a quick example that should clarify things.

import numpy as np
import pandas

df = pandas.DataFrame(np.random.rand(10,3),columns=['A','B','C'])
print df

print "add 1 to every element"
print df.applymap(lambda x: x+1)

print "add 2 to row 3 and return the series"
print df.apply(lambda x: x[3]+2,axis=0)

print "add 3 to col A and return the series"
print df.apply(lambda x: x['A']+1,axis=1)


On Wed, Mar 21, 2012 at 9:52 AM, Mihai Ionescu <if.m...-/E1597aS9LQAvxtiuMwx3w@public.gmane.org> wrote:
as a beginner,
i'm still confused with these methods names,

can we find some more descriptive names for these?
applymap is the most confusing,
changing this method's name may be enough

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To view this discussion on the web visit https://groups.google.com/d/msg/pydata/-/8bVhwF5fO1wJ.
To post to this group, send email to pyd...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To unsubscribe from this group, send email to pydata+un... <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pydata?hl=en.

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Brendan Barnwell | 8 May 07:59 2014
Picon

Equivalent of Series.map for DataFrame

I asked this question a while ago on StackOverflow but received no fully satisfying reply (http://stackoverflow.com/questions/22293683/equivalent-of-series-map-for-dataframe).

The question is, given a DataFrame with some columns, how can I get the equivalent of Series.map, to map into a Series with a MultiIndex?  That is, if the data in the DataFrame is equivalent to

[
  ["A", 1]
  ["B", 2]
  ["C", 3]
]

And I have a series that has a MultiIndex where ("A", 1), ("B", 2), and ("C", 3) are valid indices, what is the way to do myDataFrame.map(mySeries), to grab the elements from the series whose indices correspond to the data in the DataFrame?

In the question there I noted a few ways of doing this, such as converting the DataFrame to lists of lists or row-wise applying a lambda that grabs the elements from the Series.  Andy Hayden suggested another possibility, namely making a MultiIndex from the DataFrame.  However, these all seem unnecessarily cumbersome.  Shouldn't there be a one-shot way to map multiple DataFrame columns *directly* into a MultiIndexed object?

Incidentally, the fastest solution I found at that time involved using df.apply(tuple, axis=1).  However, this no longer seems to work, because apparently pandas now expands tuple results into multiple columns.  Was this change documented?  Why was it made?

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Matthew Brett | 4 May 04:22 2014
Picon

Some test failures for OSX, can I safely ignore them?

Hi,

I have been experimenting building a scipy stack installation using OSX wheels, but I've run into some errors testing pandas, here:

http://nipy.bic.berkeley.edu/builders/scipy-stack-2.7.6-wheel-staging/builds/1/steps/shell_16/logs/stdio

I just wanted to confirm with you that I can safely ignore these:

ERROR: test_round_trip_frame (pandas.io.tests.test_clipboard.TestClipboard)
ERROR: test_round_trip_frame_sep (pandas.io.tests.test_clipboard.TestClipboard)
ERROR: test_round_trip_frame_string (pandas.io.tests.test_clipboard.TestClipboard)

as being related to https://github.com/pydata/pandas/issues/6317

and these:

FAIL: test_fred_multi (pandas.io.tests.test_data.TestFred)
-> AssertionError: expected 217.47800 but got 217.46600
FAIL: test_fred_parts (pandas.io.tests.test_data.TestFred)
-> AssertionError: 217.29900000000001 != 217.23

because of this explanation:

http://stackoverflow.com/questions/22672409/nosetests-fails-with-pandas

Thanks a lot,

Matthew


--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Vigen Isayan | 24 Apr 23:18 2014
Picon

can not import pandas in Python 2.7.3

Hi all,

 I installed pandas 0.13.1 version in my python 2.7.3.
My question is when i try to import pandas in Python 2.7.3 i get the following error messages:

>>>import pandas
cannot import name hashtable
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "C:\Python27\lib\site-packages\pandas\__init__.py", line 6, in <module>
from . import hashtable, tslib, lib
ImportError: cannot import name hashtable

By the way i also have ipython installed and when i try to import pandas (which came with the installation of ipython) in ipython i don't have that problem.

thanks in advance

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Michael | 24 Apr 18:31 2014
Picon

how to emulate itertools.groupby with a series/dataframe?

given a list L = [1, 1, 1, 2, 2, 2, 3, 3, 1, 1, 3, 3,  2, 2, ]
with itertools.groupby I would get 6 groups

I'm trying to get the same result with pandas, 
but I'm not yet aware of a method

Is anyone able to generate a solution for pandas?

If I think of something, i will write it here...

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Bryan Van de Ven | 17 Apr 17:39 2014

ANN: Bokeh 0.4.4 released

I am happy to announce the release of Bokeh version 0.4.4!

Bokeh is a Python library for visualizing large and realtime datasets on the web. Its goal is to provide
elegant, concise construction of novel graphics in the style of Protovis/D3, while delivering
high-performance interactivity to thin clients. Bokeh includes its own Javascript library (BokehJS)
that implements a reactive scenegraph representation of the plot, and renders efficiently to HTML5
Canvas. Bokeh works well with IPython Notebook, but can generate standalone graphics that embed into
regular HTML. If you are a Matplotlib user, you can just use %bokeh magic to start interacting with your
plots in the notebook immediately!

Check out the full documentation, interactive gallery, and tutorial at

	http://bokeh.pydata.org

If you are using Anaconda, you can install with conda:

	conda install bokeh

Alternatively, you can install with pip:

	pip install bokeh

We are still working on some bigger features but want to get new fixes and functionality out to users as soon
as we can. Some notable features of this release are:

	* Additional Matplotlib, ggplot, and Seaborn compatibility (styling, more examples)
 	* TravisCI testing integration at https://travis-ci.org/ContinuumIO/bokeh
	* Tool enhancements, constrained pan/zoom, more hover glyphs 
	* Server remote data and downsampling examples 
	* Initial work for Bokeh "app" concept 

Also, we've also made lots of little bug fixes and enhancements - see the CHANGELOG for full details.

BokehJS is also available by CDN for use in standalone javascript applications:

	http://cdn.pydata.org/bokeh-0.4.4.js
	http://cdn.pydata.org/bokeh-0.4.4.css
	http://cdn.pydata.org/bokeh-0.4.4.min.js
	http://cdn.pydata.org/bokeh-0.4.4.min.css

Some examples of BokehJS use can be found on the Bokeh JSFiddle page:

	http://jsfiddle.net/user/bokeh/fiddles/

The release of Bokeh 0.5 is planned for early May. Some notable features we plan to include are:

	* Abstract Rendering for semantically meaningful downsampling of large datasets
	* Better grid-based layout system, using Cassowary.js
	* More MPL/Seaborn/ggplot.py compatibility and examples, using MPLExporter
	* Additional tools, improved interactions, and better plot frame
	* Touch support

Issues, enhancement requests, and pull requests can be made on the Bokeh Github page: https://github.com/continuumio/bokeh

Questions can be directed to the Bokeh mailing list: bokeh@...

If you have interest in helping to develop Bokeh, please get involved! Special thanks to recent
contributors: Amy Troschinetz and Gerald Dalley

Bryan Van de Ven
Continuum Analytics
http://continuum.io

--

-- 
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe@...
For more options, visit https://groups.google.com/d/optout.

Ralf Gommers | 15 Apr 21:03 2014
Picon

EuroSciPy 2014 abstract submission deadline extended

Hi,

In response to a number of requests, the organizing committee of the EuroSciPy 2014 conference has extended the deadline for abstract submission by 12 days, to Sunday April 27th 2014, 23:59:59 UTC. Up to then, new abstracts may be submitted on https://www.euroscipy.org/2014/ . We are very much looking forward to your submissions to the conference.

EuroSciPy 2014 is the annual European conference for scientists using Python. It will be held August 27-30 2014 in Cambridge, UK. This year promises to be an exciting event again. The number of abstract submissions is already above the level of the 2013 edition, and the first keynote has been announced:

    Steven G. Johnson (MIT), Crossing Language Barriers with Julia, Scipy, and IPython

Questions regarding abstract submission may be addressed to euroscipy-org-+ZN9ApsXKcEdnm+yROfE0A@public.gmane.org

Best regards,
Ralf Gommers (Program Chair)

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Sidon Duarte | 13 Apr 03:42 2014
Picon

Hierarchical Keys error

How to create  Hierarchical Keys on HDFStore?
I get an error when i try to create them, my code ( and error) is listed below


import pandas as pd
import tables as tb

pd.version.version
Out[45]: '0.13.1'

tb.__version__
Out[46]: '2.4.0'

l1 = [{'Images' : 'Lymphoma', 'Res' : '1024 x 767'}]
df1 = pd.DataFrame(l1)
store = pd.HDFStore('my_store1.h5')
store['MCL'] = df1

m1 = [[1,2,3],[4,5,6]]

store['MCL/Matrix'] = m1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-55-1681f898f2b9> in <module>()
----> 1 store['MCL/Matrix'] = m1

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in __setitem__(self, key, value)
    412 
    413     def __setitem__(self, key, value):
--> 414         self.put(key, value)
    415 
    416     def __delitem__(self, key):

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in put(self, key, value, format, append, **kwargs)
    819             format = get_option("io.hdf.default_format") or 'fixed'
    820         kwargs = self._validate_format(format, kwargs)
--> 821         self._write_to_group(key, value, append=append, **kwargs)
    822 
    823     def remove(self, key, where=None, start=None, stop=None):

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in _write_to_group(self, key, value, format, index, append, complib, encoding, **kwargs)
   1253 
   1254         s = self._create_storer(group, format, value, append=append,
-> 1255                                 encoding=encoding, **kwargs)
   1256         if append:
   1257             # raise if we are trying to append to a Fixed format,

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in _create_storer(self, group, format, value, append, **kwargs)
   1167                     pt = _TYPE_MAP[type(value)]
   1168                 except:
-> 1169                     error('_TYPE_MAP')
   1170 
   1171                 # we are actually a table

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in error(t)
   1143                 "cannot properly create the storer for: [%s] [group->%s,"
   1144                 "value->%s,format->%s,append->%s,kwargs->%s]"
-> 1145                 % (t, group, type(value), format, append, kwargs)
   1146             )
   1147 

TypeError: cannot properly create the storer for: [_TYPE_MAP]

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Sidon Duarte | 13 Apr 03:44 2014
Picon

Pandas HDFStore Hierarchical Keys error

How to create  Hierarchical Keys on HDFStore?
I get an error when i try to create them, my code ( and error) is listed below



import pandas as pd
import tables as tb

pd.version.version
Out[45]: '0.13.1'

tb.__version__
Out[46]: '2.4.0'

l1 = [{'Images' : 'Lymphoma', 'Res' : '1024 x 767'}]
df1 = pd.DataFrame(l1)
store = pd.HDFStore('my_store1.h5')
store['MCL'] = df1

m1 = [[1,2,3],[4,5,6]]

store['MCL/Matrix'] = m1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-55-1681f898f2b9> in <module>()
----> 1 store['MCL/Matrix'] = m1

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in __setitem__(self, key, value)
    412 
    413     def __setitem__(self, key, value):
--> 414         self.put(key, value)
    415 
    416     def __delitem__(self, key):

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in put(self, key, value, format, append, **kwargs)
    819             format = get_option("io.hdf.default_format") or 'fixed'
    820         kwargs = self._validate_format(format, kwargs)
--> 821         self._write_to_group(key, value, append=append, **kwargs)
    822 
    823     def remove(self, key, where=None, start=None, stop=None):

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in _write_to_group(self, key, value, format, index, append, complib, encoding, **kwargs)
   1253 
   1254         s = self._create_storer(group, format, value, append=append,
-> 1255                                 encoding=encoding, **kwargs)
   1256         if append:
   1257             # raise if we are trying to append to a Fixed format,

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in _create_storer(self, group, format, value, append, **kwargs)
   1167                     pt = _TYPE_MAP[type(value)]
   1168                 except:
-> 1169                     error('_TYPE_MAP')
   1170 
   1171                 # we are actually a table

/usr/lib/python2.7/dist-packages/pandas/io/pytables.pyc in error(t)
   1143                 "cannot properly create the storer for: [%s] [group->%s,"
   1144                 "value->%s,format->%s,append->%s,kwargs->%s]"
-> 1145                 % (t, group, type(value), format, append, kwargs)
   1146             )
   1147 

TypeError: cannot properly create the storer for: [_TYPE_MAP]

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Skipper Seabold | 8 Apr 14:18 2014
Picon

What happened to the suppress option?

How can I always suppress exponent notation. There used to be a
pd.set_printoptions(suppress=True).

I didn't see anything in set_options docstring, but maybe I missed it.
Those docs are a bit tough to read to my eyes.

It's not always even replicable that the last column below has
exponent notation.

[~/]
[19]: df = pd.DataFrame(np.c_[np.random.randn(10, 10),
np.random.randn(10), np.random.normal(0, 1e-6, size=10)])

[~/]
[20]: df
[20]:
         0         1         2         3         4         5         6   \
0  0.758752  0.841082 -1.255632 -0.261701  0.185757 -0.503107  1.799091
1  0.405298 -0.685786  0.303614 -2.885199 -0.519378  1.840360  1.991629
2  3.036509 -0.128819  0.628272 -0.750843  0.700176 -1.045037  0.271910
3 -0.408124 -1.819260 -0.061597  1.007687 -0.660307 -0.882606 -1.072504
4  0.819452  0.918523  0.067555 -1.297625 -1.289704  0.552133  0.310618
5  0.897767 -0.349175  0.363083 -1.472415  0.962165  0.129417 -0.160296
6  0.630074 -0.373593 -0.361610 -2.283741  0.245379 -0.478192 -0.441849
7  1.266303  1.272007  0.541298  0.638055  0.137964 -1.416108  0.564519
8 -0.781213  1.854782  1.055120  0.236743 -0.460153 -0.079680 -1.293484
9 -0.091647  0.123987  0.472135 -0.968936 -1.143489  0.153881 -1.225102

         7         8         9         10            11
0 -0.535159 -1.777501 -0.921817  0.508602 -8.837914e-07
1 -0.828329  1.762914  0.829432  1.596714  1.154674e-06
2 -1.406737  1.189656 -0.453278 -0.002569 -2.415813e-07
3  0.660344  0.557123 -0.240391  2.194905 -5.047970e-07
4 -0.189786 -1.078186  0.399949  0.954772 -1.515437e-06
5 -0.582136  1.527210  0.090962  0.367698  5.088797e-08
6 -0.537436  0.657869  1.516261  0.150562 -4.539411e-07
7  0.708429 -0.755490 -0.546651 -0.527286  3.034608e-07
8  0.070430  1.930544  0.485097 -2.710529  1.229098e-06
9  0.406528 -2.541766  0.156423 -0.359297 -6.515622e-08

--

-- 
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe@...
For more options, visit https://groups.google.com/d/optout.

Erwan Le Quere | 7 Apr 19:09 2014
Picon

Pandas resample with "seasonal" frequency

Hi,

I am trying to resample data to a "seasonal" frequency (basically 2 quarters) and resample seems to behave bizarrely. Depending upon which quarter the start date of the initial index is, the result is different (and appears wrong to me).

See this SO post (no answers yet ;-( where I describe the problem in more details.

Can anyone point me to what I'm doing incorrectly or else if there is anything wrong with resample?

Cheers

Erwan

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.

Gmane