Janet Jacobsen | 2 Jul 21:29 2009

Re: user's process shows as "idle in transaction"

Hi.  I wanted to let you know that the user added a rollback() to his
code, and now things are working much better.

Prior to his adding the rollback(), deleting as few as 60 rows from
the table (which has 12 M rows) that his script queries took as long
as 4 minutes.  If I tried to create an index on that table, ps aux showed
me that the process was always "waiting".  The only way I could
create indexes - or delete more than 60 rows at a time - was to kill
the user's postgres process.

Now that he has added the rollback() to his code, his postgres
process per "ps aux" no longer shows that it is "idle in transaction",
but instead just "idle".

And I just tried creating an index on the table with 12.5M rows, and
it took less than one minute.  Much, much better.

Thank you for your help,
Janet

Federico Di Gregorio wrote:
> Il giorno lun, 29/06/2009 alle 12.26 -0700, Janet Jacobsen ha scritto:
> [snip]
>   
>> The user told me that he does not close the database connection
>> after the fetchall - instead he has a Python sleep command, so that 
>> he is checking the database every 60 s to see whether new entries
>> have been added to a given table
>> His code is part of an analysis pipeline, whereas the part of the
>> database that I work on is loading processed data into the
(Continue reading)

Russell Hires | 7 Jul 17:27 2009
Picon
Picon

Trying to install psycopg2 into zope/plone

Hello all. I'm trying to install psycopg2 into plone, using buildout,  
but I keep getting an error:

Getting distribution for 'psycopg2'.
error: Setup script exited with error: No such file or directory
An error occured when trying to install psycopg2 2.0.11.Look above  
this message for any errors thatwere output by easy_install.
While:
   Installing zopeskel.
   Getting distribution for 'psycopg2'.
Error: Couldn't install: psycopg2 2.0.11

And I have no idea why.  Below is a copy of my buildout.cfg. I've  
been all over this issue w/ the #plone people on irc, and no one has  
any advice they can offer about why it won't install properly, or how  
to fix this error.

Any advice or direction is appreciated. Running plone 3.2 on OSX 10.4  
Intel.

Russell

############################################
#
# Buildout Configuration File for Standalone Plone
# ------------------------------------------------
# $LastChangedDate: 2009-03-03 16:13:01 -0800 (Tue, 03 Mar 2009) $  
$LastChangedRevision: 25091M $
#
# After making changes in this configuration file,
(Continue reading)

Jonathan Ballet | 7 Jul 19:28 2009
Picon

Re: Trying to install psycopg2 into zope/plone

On Tue, Jul 7, 2009 at 5:27 PM, Russell Hires<rhires@...> wrote:
> Hello all. I'm trying to install psycopg2 into plone, using buildout, but I
> keep getting an error:
>
> Getting distribution for 'psycopg2'.
> error: Setup script exited with error: No such file or directory
> An error occured when trying to install psycopg2 2.0.11.Look above this
> message for any errors thatwere output by easy_install.
> While:
>  Installing zopeskel.
>  Getting distribution for 'psycopg2'.
> Error: Couldn't install: psycopg2 2.0.11

Can you post the output of buildout with logs enabled as well ?

Something like "buildout -o buildout:log-level=DEBUG -v" (this can be
very long, maybe you should try to remove the uninteresting output)
Russell Hires | 7 Jul 20:02 2009
Picon
Picon

Re: Trying to install psycopg2 into zope/plone

This is everything. Not sure what's "uninteresting" but it's not too  
too long...

Thanks!

bin/buildout -o buildout:log-level=DEBUG -v
Installing 'plone.recipe.zope2install'.
We have the distribution that satisfies  
'plone.recipe.zope2install==2.6'.
Installing 'plone.recipe.distros'.
Picked: plone.recipe.distros = 1.5
Installing 'plone.recipe.zope2instance'.
We have the distribution that satisfies  
'plone.recipe.zope2instance==2.7'.
Adding required 'zc.recipe.egg==1.1.0'
We have the distribution that satisfies 'zc.recipe.egg==1.1.0'.
Installing 'plone.recipe.precompiler'.
Picked: plone.recipe.precompiler = 0.3
Installing 'plone.recipe.command'.
Picked: plone.recipe.command = 1.0
Installing 'plone.recipe.unifiedinstaller'.
Picked: plone.recipe.unifiedinstaller = 0.9

Configuration data:
[zope2]
additional-fake-eggs = %(__buildout_space_n__)sZConfig
         pytz
executable = /Applications/Plone/Python-2.4/bin/python
fake-zope-eggs = true
location = /Applications/Plone/Zope-2.10.7-final-py2.4
(Continue reading)

Russell Hires | 16 Jul 05:47 2009
Picon
Picon

Re: Trying to install psycopg2 into zope/plone

I'm guessing no one has any ideas on this. Where else is a good place  
to get more information or assistance with this?

TIA

Russell

On Jul 7, 2009, at 2:02 PM, Russell Hires wrote:

> This is everything. Not sure what's "uninteresting" but it's not  
> too too long...
>
> Thanks!
>
>
> bin/buildout -o buildout:log-level=DEBUG -v
> Installing 'plone.recipe.zope2install'.
> We have the distribution that satisfies  
> 'plone.recipe.zope2install==2.6'.
> Installing 'plone.recipe.distros'.
> Picked: plone.recipe.distros = 1.5
> Installing 'plone.recipe.zope2instance'.
> We have the distribution that satisfies  
> 'plone.recipe.zope2instance==2.7'.
> Adding required 'zc.recipe.egg==1.1.0'
> We have the distribution that satisfies 'zc.recipe.egg==1.1.0'.
> Installing 'plone.recipe.precompiler'.
> Picked: plone.recipe.precompiler = 0.3
> Installing 'plone.recipe.command'.
> Picked: plone.recipe.command = 1.0
(Continue reading)

Stuart Bishop | 16 Jul 06:06 2009
Picon

Re: Trying to install psycopg2 into zope/plone

On Thu, Jul 16, 2009 at 10:47 AM, Russell Hires<rhires@...> wrote:

> I'm guessing no one has any ideas on this. Where else is a good place to get
> more information or assistance with this?

It looks like you are having a buildout problem, so I'd suggest the
main Python list or maybe one of the Zope mailing lists - I'm not
familiar where buildout discussions occur.

--

-- 
Stuart Bishop <stuart@...>
http://www.stuartbishop.net/
Charlie Clark | 16 Jul 12:30 2009

Re: Psycopg Digest, Vol 66, Issue 4

Am 16.07.2009, 12:00 Uhr, schrieb <psycopg-request <at> lists.initd.org>:

> It looks like you are having a buildout problem, so I'd suggest the
> main Python list or maybe one of the Zope mailing lists - I'm not
> familiar where buildout discussions occur.

It's probably worth noting that you don't have to use buildout and eggs  
and can install the required libraries and Zope Products pretty much the  
same as always.

Charlie
--

-- 
Charlie Clark
Helmholtzstr. 20
Düsseldorf
D- 40215
Tel: +49-211-938-5360
GSM: +49-178-782-6226
_______________________________________________
Psycopg mailing list
Psycopg <at> lists.initd.org
http://lists.initd.org/mailman/listinfo/psycopg
Brian Sutherland | 16 Jul 12:54 2009
Picon

Re: Trying to install psycopg2 into zope/plone

On Tue, Jul 07, 2009 at 11:27:33AM -0400, Russell Hires wrote:
> Hello all. I'm trying to install psycopg2 into plone, using buildout,  
> but I keep getting an error:
>
> Getting distribution for 'psycopg2'.
> error: Setup script exited with error: No such file or directory

Do you have PostgreSQL installed?

This happened to me when the pg_config binary was not on the $PATH. On
Debian Lenny, the pg_config binary is in the libpq-dev package.

> An error occured when trying to install psycopg2 2.0.11.Look above this 
> message for any errors thatwere output by easy_install.
> While:
>   Installing zopeskel.
>   Getting distribution for 'psycopg2'.
> Error: Couldn't install: psycopg2 2.0.11
>
> And I have no idea why.  Below is a copy of my buildout.cfg. I've been 
> all over this issue w/ the #plone people on irc, and no one has any 
> advice they can offer about why it won't install properly, or how to fix 
> this error.
>
> Any advice or direction is appreciated. Running plone 3.2 on OSX 10.4  
> Intel.
>
> Russell
>
> ############################################
(Continue reading)

Andrew Klaassen | 22 Jul 21:53 2009
Picon

table.column in cursor.description?

Hi,

Is it possible to get psycopg2 to return full column names (i.e. "table.column" instead of just "column")
in cursor.description?  This would be handy when executing joins.

I.e.

>>> cursor.execute("select * from document, element")
>>> print cursor.description
[('document.id', ...), ('document.name', ...), ('element.id', ...), ('element.text', ...)]

...instead of the current behaviour, which is:

>>> print cursor.description
[('id', ...), ('name', ...), ('id', ...), ('text', ...)]

The SQLite Python module has something like this with "pragma
full_column_names=ON;":

http://mail.python.org/pipermail/python-bugs-list/2009-January/069202.html

Possible?

Thanks.

Andrew

      __________________________________________________________________
Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now
http://ca.toolbar.yahoo.com.
(Continue reading)

James Henstridge | 23 Jul 09:18 2009
Picon

Re: table.column in cursor.description?

On Thu, Jul 23, 2009 at 3:53 AM, Andrew Klaassen<clawsoon@...> wrote:
> Hi,
>
> Is it possible to get psycopg2 to return full column names (i.e. "table.column" instead of just "column")
in cursor.description?  This would be handy when executing joins.
>
> I.e.
>
>>>> cursor.execute("select * from document, element")
>>>> print cursor.description
> [('document.id', ...), ('document.name', ...), ('element.id', ...), ('element.text', ...)]
>
> ...instead of the current behaviour, which is:
>
>>>> print cursor.description
> [('id', ...), ('name', ...), ('id', ...), ('text', ...)]

The column names for result sets come directly from PostgreSQL.
Psycopg2 isn't adding or removing any information here.  I don't know
if it is possible to change PostgreSQL's behaviour here, but if you
can it is likely that the change would be reflected in Psycopg2 with
no extra effort.

Perhaps someone on one of the PostgreSQL lists will know the answer?

> The SQLite Python module has something like this with "pragma
> full_column_names=ON;":
>
> http://mail.python.org/pipermail/python-bugs-list/2009-January/069202.html

(Continue reading)


Gmane