error in make

Hello again,

Thanks for pointing out my mistake with the includes option!  Now I am
getting an error in the make procedure...
-------------------------------------------------------------------------

[irving@... psycopg-1.1.11]$ ./configure
--with-postgres-libraries=/home/irving/postgresql-7.4.1/src/interfaces/libpq
--with-postgres-includes=/home/irving/postgresql-7.4.1/src/include
--with-zope=/opt/Zope-2.7
checking for python... /usr/local/bin/python
checking python version... 2.3
checking python installation prefix... /usr/local
checking python installation exec_prefix... /usr/local
checking definitions in Python library makefile... done
checking location of python library... $(prefix)/lib/python2.3/site-packages
checking location of python shared modules...
$(exec_prefix)/lib/python2.3/site-packages
checking for gcc... gcc -pthread
checking for C compiler default output... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc -pthread accepts -g... yes
checking for gcc -pthread option to accept ANSI C... none needed
checking how to run the C preprocessor... gcc -pthread -E
checking whether make sets $(MAKE)... yes
checking for inline... inline

Re: Connect pooling problem

On Sunday 29 February 2004 16:15, Federico Di Gregorio wrote:
> Il dom, 2004-02-29 alle 16:04, Ricardo Caesar Lenzi ha scritto:
> > Hi all,
> >
> > I am having problems with the number of connections opened between the my
> > application and database. Some times the number of connections grow util
> > the limit of the psycopg max connections, and break the application.
> >
> > It seems that the minimum number of connections (minconn argument in
> > psycopg.connect) is not being respected, therefore when a cursor is
> > closed, the pool of connections seems not to reduce, keeping the physical
> > database connection open.
>
> the short answer is: don't use psycopg own connection pooling. use
> external pooling code like one in twisted, webware, etc.
>
> connection pooling does not belong to the db adapter layer and has
> already been removed from psycopg 2.

Hi Federico.

What is the correct way in order to avoid using psycopg connection pool?

The module.h defines MINCONN (the default value) as 8. Therefore, the pool 
size in 8, and 8 connection will always be available for reuse. But I think 
this should be not the case for Zope, where the threads do the pool.

In a typical Zope setup, where you have (let's say) 4 threads, I think minconn 
should be 0, and maxconn should be 4. Am I right here? Otherwise, what should 
be the right parameters?

Re: Connect pooling problem

Il lun, 2004-03-01 alle 19:28, Paolo Bizzarri ha scritto:
> What is the correct way in order to avoid using psycopg connection pool?

never pass serialize=0 to .connect().

> The module.h defines MINCONN (the default value) as 8. Therefore, the pool 
> size in 8, and 8 connection will always be available for reuse. But I think 
> this should be not the case for Zope, where the threads do the pool.
> 
> In a typical Zope setup, where you have (let's say) 4 threads, I think minconn 
> should be 0, and maxconn should be 4. Am I right here? Otherwise, what should 
> be the right parameters?

in Zope the connection object is put in serialized mode (the default)
and there is a 1-1 correspondance between connection objects and
postgresql connections.

--

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                fog@...
INIT.D Developer                                           fog@...
  Those who do not study Lisp are doomed to reimplement it. Poorly.
                                     -- from Karl M. Hegbloom .signature

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

exceptions within & memory leaks between transactions

Greetings;

Looking for a way to try inserting all lines from a CSV file and then
only handling any say as an update if it raises an exception because of
a duplicate key, all without disrupting the immediate transaction.

Also wondering if there's an obvious enough way to track down an
apparent memory leak when trying to import a number of CSV files even
though this function only loops through them one at a time, before
committing the transaction and then even going out of scope.  The system
does periodically log something about a "recycled transaction file" to
standard output, if that offers any clues...

Thanks,

Re: exceptions within & memory leaks between transactions

Il mar, 2004-03-02 alle 20:40, DCS ha scritto:
> Greetings;
> 
> Looking for a way to try inserting all lines from a CSV file and then
> only handling any say as an update if it raises an exception because of
> a duplicate key, all without disrupting the immediate transaction.

you can't. postgresql (not psycopg, postgresql!) invalidates the *whole*
transaction at the first error.

> Also wondering if there's an obvious enough way to track down an
> apparent memory leak when trying to import a number of CSV files even
> though this function only loops through them one at a time, before
> committing the transaction and then even going out of scope.  The system
> does periodically log something about a "recycled transaction file" to
> standard output, if that offers any clues...

that's a postgresql log. do you mean a postgresql memory leak or a
psycopg one?

--

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                fog@...
INIT.D Developer                                           fog@...
  Try the Joy of TeX [http://www.tug.org]
                                      -- brought to you by One Line Spam

_______________________________________________
Psycopg mailing list

Hanging connections

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

Re: Hanging connections

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

Re: Hanging connections

Il mer, 2004-03-03 alle 21:32, Nick Bower ha scritto:
> Hello.  Is there a status update on the hanging connections from Zope
> to Postgresql problem?

marco bizzarri from icube sent me a nice procedure to create some
hanging connections. open a connection, use it a little bit and then
refresh a product. repeat some times and some hangin' connections
appear. can someone using other DAs try this and tell me if it is a
generic zope problem or just a psycopg one?

> When simply opening and closing a zpsycopg connection from the Zope
> management interface, I'm always left with a hanging connection to
> Postgresql (as reported by a select on pg_stat_activity).

this is very strange and does not happen here.

> When opening, web stressing with repeated ZSQL calls and closing a
> zsycopg connection from the Zope management interface, I'm left with
> several hanging connections.

argh.

> Maybe this problem is common and being worked on?  Do we have an
> explanation and fix date (assuming I'm actually seeing the problem
> correctly)?  The reason I ask is that I'm 2 weeks away from a serious
> launch and I've just discovered we're seeing this issue.  =-\

i am working on it. note that our site never showed such a problem but
we are running with very high cache parameters and low load and i am
thinking that the problem is cache related. anyway, i'll start to post
updates about this problem every 48h.

please, compile with --enable-devel, look at the logs, look at zope
logs, even at postgresql ones and report anything suspicious.

many thanks,
federico

--

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                fog@...
INIT.D Developer                                           fog@...
                        Su grappe nessuno è innocente. -- Natale Titotto

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

Re: Hanging connections

On Thursday 04 March 2004 00:44, Federico Di Gregorio wrote:
> Il mer, 2004-03-03 alle 21:32, Nick Bower ha scritto:
> > Hello.  Is there a status update on the hanging connections from Zope
> > to Postgresql problem?
>
> marco bizzarri from icube sent me a nice procedure to create some
> hanging connections. open a connection, use it a little bit and then
> refresh a product. repeat some times and some hangin' connections
> appear. can someone using other DAs try this and tell me if it is a
> generic zope problem or just a psycopg one?
>
> > When simply opening and closing a zpsycopg connection from the Zope
> > management interface, I'm always left with a hanging connection to
> > Postgresql (as reported by a select on pg_stat_activity).
>
> this is very strange and does not happen here.
>
> > When opening, web stressing with repeated ZSQL calls and closing a
> > zsycopg connection from the Zope management interface, I'm left with
> > several hanging connections.
>
> argh.
>
> > Maybe this problem is common and being worked on?  Do we have an
> > explanation and fix date (assuming I'm actually seeing the problem
> > correctly)?  The reason I ask is that I'm 2 weeks away from a serious
> > launch and I've just discovered we're seeing this issue.  =-\
>
> i am working on it. note that our site never showed such a problem but
> we are running with very high cache parameters and low load and i am
> thinking that the problem is cache related. anyway, i'll start to post
> updates about this problem every 48h.

Yes, most probably this is related to the cache, since we had the problem in 
two different environment: one is a production environment, with a lot of 
access and very small cache setup (I think we will try to increase the cache 
size, and then provide some updates). The second one is our "normal" 
development environment, where we have little access but A LOT of refresh. I 
can remember from Zope 2.6.1 Changelog that there were some fixes for ZODB 
exactly on this issue.

> please, compile with --enable-devel, look at the logs, look at zope
> logs, even at postgresql ones and report anything suspicious.

Oh... so this was the RIGHT way to have PSYCOTIC_DEBUG set and all the other 
stuff required? I spent 1 hour to compile psycopg in order to have the proper 
output :)

> many thanks,
> federico

Regards
Marco

--

-- 
Paolo Bizzarri - President - Icube S.r.l.
Address:   	Via Ridolfi 15 - 56124 Pisa (PI), Italy
E-mail:	p.bizzarri@...	Web:	http://www.icube.it
Phone:	(+39) 050 97 02 07	Fax: 	(+39) 050 31 36 588

Re: Hanging connections

Il gio, 2004-03-04 alle 09:22, Paolo Bizzarri ha scritto:
> > please, compile with --enable-devel, look at the logs, look at zope
> > logs, even at postgresql ones and report anything suspicious.
> 
> Oh... so this was the RIGHT way to have PSYCOTIC_DEBUG set and all the other 
> stuff required? I spent 1 hour to compile psycopg in order to have the proper 
> output :)

ghgh. ./configure --help is your friend. :)

--

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                fog@...
INIT.D Developer                                           fog@...
  Qu'est ce que la folie? Juste un sentiment de liberté si
   fort qu'on en oublie ce qui nous rattache au monde... -- J. de Loctra

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg