johnf | 1 May 15:29 2009

stop a long running select

Hi,
Is there a way to stop a long running select statement without losing the 
connection, cursor from python?
--

-- 
John Fabiani
James Henstridge | 1 May 17:59 2009
Picon

Re: stop a long running select

On Fri, May 1, 2009 at 9:29 PM, johnf <jfabiani@...> wrote:
> Hi,
> Is there a way to stop a long running select statement without losing the
> connection, cursor from python?

There is a connection variable called statement_timeout, which you can
use to set the maximum length of time any particular statement issued
can take in milliseconds.

So if you issue the command "SET statement_timeout TO 1000", and then
a subsequent execute() call takes longer than 1 second, you'll get a
psycopg2.extensions.QueryCanceledError exception.  At that point you
can roll back the connection and do what ever recovery is appropriate
for your application.

The transaction will have failed, but that is unavoidable: a query
being canceled is the same as a query that fails.

James.
theodore test | 1 May 22:57 2009
Picon

Installation Error on OS X

Hello everyone,

I recently attempted to install psycopg2-2.0.10 on my Intel OS X 10.4.11 system.  Both the "easy_install ." and regular "python setup.py install" methods produced similar errors.  Any thoughts or advice?

Thanks in advance,

Theodore Test

===EASY_INSTALL VERSION===
$ easy_install .
Processing .
Running setup.py -q bdist_egg --dist-dir /Users/TheoTest/Desktop/Buffer/psycopg2-2.0.10/egg-dist-tmp-Pu9Bo4
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/Current/bin/easy_install", line 8, in <module>
    load_entry_point('setuptools==0.6c8.0002', 'console_scripts', 'easy_install')()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 1671, in main
    with_ei_usage(lambda:
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 1659, in with_ei_usage
    return f()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 1675, in <lambda>
    distclass=DistributionWithoutHelpCommands, **kw
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 211, in run
    self.easy_install(spec, not self.no_deps)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 427, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 476, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 655, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 930, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/easy_install.py", line 919, in run_setup
    run_setup(setup_script, args)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/sandbox.py", line 27, in run_setup
    lambda: execfile(
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/sandbox.py", line 63, in run
    return func()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/sandbox.py", line 29, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'}
  File "setup.py", line 410, in <module>
    ext_modules=ext)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/bdist_egg.py", line 167, in run
    self.run_command("egg_info")
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/egg_info.py", line 171, in run
    self.find_sources()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/egg_info.py", line 252, in find_sources
    mm.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/egg_info.py", line 306, in run
    self.add_defaults()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/site-packages/setuptools-0.6c8.0002-py2.5.egg/setuptools/command/egg_info.py", line 330, in add_defaults
    sdist.add_defaults(self)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/command/sdist.py", line 300, in add_defaults
    build_ext = self.get_finalized_command('build_ext')
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/cmd.py", line 319, in get_finalized_command
    cmd_obj.ensure_finalized()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/cmd.py", line 117, in ensure_finalized
    self.finalize_options()
  File "setup.py", line 219, in finalize_options
    except (Warning, w):
NameError: global name 'w' is not defined


===PYTHON SETUP.PY INSTALL VERSION===

$python setup.py install
Password:
running install
running build
running build_py
running build_ext
Traceback (most recent call last):
  File "setup.py", line 410, in <module>
    ext_modules=ext)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/command/install.py", line 506, in run
    self.run_command('build')
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/command/build.py", line 112, in run
    self.run_command(cmd_name)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/dist.py", line 993, in run_command
    cmd_obj.ensure_finalized()
  File "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/cmd.py", line 117, in ensure_finalized
    self.finalize_options()
  File "setup.py", line 219, in finalize_options
    except (Warning, w):
NameError: global name 'w' is not defined

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg
James Henstridge | 2 May 06:24 2009
Picon

Re: Installation Error on OS X

On Sat, May 2, 2009 at 4:57 AM, theodore test <tteststudent@...> wrote:
> Hello everyone,
>
> I recently attempted to install psycopg2-2.0.10 on my Intel OS X 10.4.11
> system.  Both the "easy_install ." and regular "python setup.py install"
> methods produced similar errors.  Any thoughts or advice?
>
> Thanks in advance,
>
> Theodore Test
>

> ===PYTHON SETUP.PY INSTALL VERSION===
>
> $python setup.py install
[snip]
> "/Library/Frameworks/Python.framework/Versions/2.5.2001/lib/python2.5/distutils/cmd.py",
> line 117, in ensure_finalized
>     self.finalize_options()
>   File "setup.py", line 219, in finalize_options
>     except (Warning, w):
> NameError: global name 'w' is not defined

Looks like a bug in psycopg2's setup.py file.  Try editing it and
removing the parentheses on line 219 (so it just reads "except
Warning, w:".

This is an error/warning code path, so it is likely that there is
something else you'll need to do to install psycopg2, but this should
make the error display.

James.
theodore test | 2 May 18:57 2009
Picon

Re: Installation Error on OS X

Thank you for the advice.  Ultimately, the install procedure seemed to have been having trouble finding related PostgreSQL command files. 

Solved by running:  PATH=$PATH:/Library/PostgresPlus/8.3/bin/ sudo easy_install psycopg2


Thanks again,

Theo

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg
Marko Kreen | 4 May 15:22 2009
Picon

decrease DictRow memory usage

Use "__slots__ = ('_index',)" to reserve single attribute slot for _index.
Otherwise a per-row dict is created to store object attributes...

The only danger of doing it is that if somebody uses plain DictRow and
adds new attrubutes into it, it will get error:

  row.foo = x

I'm really dubious if anybody does this.  NB, this does not apply if
somebody has subclassed DictRow, then the parent class' __slots__
is ignored.

Attached patch does the same for RealDictRow too, although I personally
do not use it.  But it seems good idea to do it there too.

On older python versions that do not support __slots__ it should be
taken as random class attribute, so should be no problem.

--

-- 
marko
Attachment (dictrow.diet.diff): text/x-patch, 948 bytes
_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg
Federico Di Gregorio | 5 May 11:33 2009

Re: decrease DictRow memory usage

Il giorno lun, 04/05/2009 alle 16.22 +0300, Marko Kreen ha scritto:
> Use "__slots__ = ('_index',)" to reserve single attribute slot for
> _index.
> Otherwise a per-row dict is created to store object attributes...
> 
> The only danger of doing it is that if somebody uses plain DictRow and
> adds new attrubutes into it, it will get error:
> 
>   row.foo = x
> 
> I'm really dubious if anybody does this.  NB, this does not apply if
> somebody has subclassed DictRow, then the parent class' __slots__
> is ignored.

I agree. I someone uses custom attributes on the DictRow let me know or
I'll apply the patch. ;)

federico

--

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                fog@...
INIT.D Developer                                           fog@...
  Spesso crescere ed andare a vivere da soli è l'unico modo di restare
   bambini.                                             -- Alice Fontana
_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg
Oswaldo Hernández | 5 May 12:33 2009
Picon

Re: decrease DictRow memory usage

Federico Di Gregorio escribió:
> Il giorno lun, 04/05/2009 alle 16.22 +0300, Marko Kreen ha scritto:
>> Use "__slots__ = ('_index',)" to reserve single attribute slot for
>> _index.
>> Otherwise a per-row dict is created to store object attributes...
>>
>> The only danger of doing it is that if somebody uses plain DictRow and
>> adds new attrubutes into it, it will get error:
>>
>>   row.foo = x
>>
>> I'm really dubious if anybody does this.  NB, this does not apply if
>> somebody has subclassed DictRow, then the parent class' __slots__
>> is ignored.
> 
> I agree. I someone uses custom attributes on the DictRow let me know or
> I'll apply the patch. ;)
> 

Hi,

I use a custom DictRow class derived from extras.DictRow. I will do this afternoon some tests to 
overload '__slots__'.

I will put here the results tomorrow.

--

-- 
*****************************************
Oswaldo Hernández
oswaldo ( <at> ) soft-com (.) es
*****************************************
PD:
Antes de imprimir este mensaje, asegúrese de que es necesario.
El medio ambiente está en nuestra mano.
_______________________________________________
Psycopg mailing list
Psycopg <at> lists.initd.org
http://lists.initd.org/mailman/listinfo/psycopg
James Henstridge | 5 May 14:56 2009
Picon

Re: decrease DictRow memory usage

2009/5/5 Oswaldo Hernández <listas@...>:
> Federico Di Gregorio escribió:
>>
>> Il giorno lun, 04/05/2009 alle 16.22 +0300, Marko Kreen ha scritto:
>>>
>>> Use "__slots__ = ('_index',)" to reserve single attribute slot for
>>> _index.
>>> Otherwise a per-row dict is created to store object attributes...
>>>
>>> The only danger of doing it is that if somebody uses plain DictRow and
>>> adds new attrubutes into it, it will get error:
>>>
>>>  row.foo = x
>>>
>>> I'm really dubious if anybody does this.  NB, this does not apply if
>>> somebody has subclassed DictRow, then the parent class' __slots__
>>> is ignored.
>>
>> I agree. I someone uses custom attributes on the DictRow let me know or
>> I'll apply the patch. ;)
>>
>
> Hi,
>
> I use a custom DictRow class derived from extras.DictRow. I will do this
> afternoon some tests to overload '__slots__'.
>
> I will put here the results tomorrow.

If you're subclassing DictRow then this change shouldn't have any
effect on you.  Your subclass will have an instance dictionary though,
so you won't see the memory savings.

If you want to see the memory savings though, you'll need to define
__slots__ for your subclass.  If you aren't adding any attributes,
setting it to an empty tuple will do the trick.

James.
Oswaldo Hernández | 5 May 15:55 2009
Picon

Re: decrease DictRow memory usage

James Henstridge escribió:
> 2009/5/5 Oswaldo Hernández <listas@...>:
>> Federico Di Gregorio escribió:
>>> Il giorno lun, 04/05/2009 alle 16.22 +0300, Marko Kreen ha scritto:
>>>> Use "__slots__ = ('_index',)" to reserve single attribute slot for
>>>> _index.
>>>> Otherwise a per-row dict is created to store object attributes...
>>>>
>>>> The only danger of doing it is that if somebody uses plain DictRow and
>>>> adds new attrubutes into it, it will get error:
>>>>
>>>>  row.foo = x
>>>>
>>>> I'm really dubious if anybody does this.  NB, this does not apply if
>>>> somebody has subclassed DictRow, then the parent class' __slots__
>>>> is ignored.
>>> I agree. I someone uses custom attributes on the DictRow let me know or
>>> I'll apply the patch. ;)
>>>
>> Hi,
>>
>> I use a custom DictRow class derived from extras.DictRow. I will do this
>> afternoon some tests to overload '__slots__'.
>>
>> I will put here the results tomorrow.
> 
> If you're subclassing DictRow then this change shouldn't have any
> effect on you.  Your subclass will have an instance dictionary though,
> so you won't see the memory savings.
> 
> If you want to see the memory savings though, you'll need to define
> __slots__ for your subclass.  If you aren't adding any attributes,
> setting it to an empty tuple will do the trick.
> 

Yes i add one attribute in my custom class.

I apply the patch, then on an initial test:

. If i do nothing on my subclass, python provides a __dict__ for it. Works fine.

. Adding in the subclass:
      __slots__ = ('myattr',)
   also works fine.

Seems no problems about using slots on main DictRow and subclasing it.

Thanks,

--

-- 
*****************************************
Oswaldo Hernández
oswaldo ( <at> ) soft-com (.) es
*****************************************
PD:
Antes de imprimir este mensaje, asegúrese de que es necesario.
El medio ambiente está en nuestra mano.

Gmane