Re: How to return sets from plpythonu functions

2006/4/21, Clodoaldo Pinto <clodoaldo.pinto@...>:
> 2006/4/21, Tim Roberts <timr@...>:
> > How can that possibly be?  The psycopg code is making a brand new
> > connection to EXACTLY the same database server, making the same request
> > in SQL.  I mean, I hear what you are saying, but unless the procedure is
> > doing a lot of extra processing beyond simple SQL statements, I find it
> > almost impossible to believe that adding a Python layer could improve
> > the performance.
>
> The function does a lot of distinct, max(), min() and avg(). I
> suspect, given what I have already seen, that doing it in Python will
> be more efficient.
>
> I have some plpythonu functions, none returning rows. They do lots of
> computing and are sure much faster than sql or plpgsql. They are
> called only once in a batch processing so the psycopg load is not
> important. This new one will be called from a web page so your point
> is significant. I did not know about plpy. I will give it a try.
>
> One of the functions was greatly accelerated just by changing from
> PyGreSQL to psycopg:
>
> http://lists.initd.org/pipermail/psycopg/2005-September/003816.html
> http://lists.initd.org/pipermail/psycopg/2005-October/003994.html
>

My findings:

The pure sql function sql() takes 630 ms.
The plpythonu function py() takes 319 ms.

Re: [psycopg] #103: easy_install psycopg2 fails because cheeseshop entry is out-of-date

#103: easy_install psycopg2 fails because cheeseshop entry is out-of-date
-----------------------------+----------------------------------------------
 Reporter:  rmunn <at> pobox.com  |        Owner:  fog   
     Type:  defect           |       Status:  new   
 Priority:  normal           |    Milestone:        
Component:  psycopg2         |      Version:  1.1.21
 Severity:  normal           |   Resolution:        
 Keywords:                   |  
-----------------------------+----------------------------------------------
Comment (by lbruno):

 Maybe having a psycopg2-latest.tar.gz symlink is a good idea?

-- 
Ticket URL: <http://initd.org/tracker/psycopg/ticket/103>
psycopg <http://initd.org/>
psycopg

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

Re: How to return sets from plpythonu functions

	Just a thought, but...

	When writing stored procedures for postgres, the following problems arise  
:

	- plpgsql has all the database features, but the language SUCKS horribly,  
and it's slow.
	- plpython is python, so it's gorgeous, but the database interaction  
features are missing (you have to use plpy (yuk) or import psycopg2 and  
open another connection).

	Therefore, my question would be :

	Is it possible to have a customized version of psycopg2 which would be  
specialized for use inside pl-python functions. It would be the mutant  
offspring of plpy and psycopg.
	All the type conversions and stuff would be kept intact. Even for return  
values. (this would be gorgeous).
	It would address the database directly instead of using libpq (being able  
to work inside the current transaction). Thus, it would be insanely fast.
	It would cache prepared statements : using SQL inside Python would be  
simple, natural, and extremely fast (as fast as plpgsql for doing queries).

	Any thoughts from the developers ?

	Regards,
	Pierre

Re: How to return sets from plpythonu functions

PFC wrote:

>	Is it possible to have a customized version of psycopg2 which would be  
>specialized for use inside pl-python functions. It would be the mutant  
>offspring of plpy and psycopg.
>	All the type conversions and stuff would be kept intact. Even for return  
>values. (this would be gorgeous).
>	It would address the database directly instead of using libpq (being able  
>to work inside the current transaction). Thus, it would be insanely fast.
>	It would cache prepared statements : using SQL inside Python would be  
>simple, natural, and extremely fast (as fast as plpgsql for doing queries).
>

This requires more than psycopg: it requires that Postgres actually
expose its internal object model to its embedded Python interpreter, so
that those objects are accessible.  Psycopg2 cannot invent hooks into
Postgres, and I do not know whether the PG language interface actually
does that.

This maybe a question for the Postgres developer's list.

--

-- 
Tim Roberts, timr@...
Providenza & Boekelheide, Inc.

how to handle large object in psycopg2?

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

Re: how to handle large object in psycopg2?

Il giorno mar, 09/05/2006 alle 18.14 +0800, 王磊 ha scritto:
> lobject-branch is the only way?

Yes, and it was even abandoned by its author.

federico

--

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                fog@...
INIT.D Developer                                           fog@...
 And anyone who yells "fork" deserves to get one stuck in them.
                                                          -- Dan Winship

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

Re: how to handle large object in psycopg2?

On Tuesday 09 May 2006 4:03 pm, Federico Di Gregorio wrote:
> Il giorno mar, 09/05/2006 alle 18.14 +0800, 王磊 ha scritto:
> > lobject-branch is the only way?
>
> Yes, and it was even abandoned by its author.

LOL
--

-- 
regards
kg

http://www.livejournal.com/users/lawgon
tally ho! http://avsap.org.in
ಇಂಡ್ಲಿನಕ್ಸ வாழ்க!
_______________________________________________
Psycopg mailing list
Psycopg <at> lists.initd.org
http://lists.initd.org/mailman/listinfo/psycopg

Re: how to handle large object in psycopg2?

Il giorno mar, 09/05/2006 alle 16.14 +0530, Kenneth Gonsalves ha
scritto:
> On Tuesday 09 May 2006 4:03 pm, Federico Di Gregorio wrote:
> > Il giorno mar, 09/05/2006 alle 18.14 +0800, 王磊 ha scritto:
> > > lobject-branch is the only way?
> >
> > Yes, and it was even abandoned by its author.
> 
> LOL

Well.. this is the first time I talk about psycopg's author in third
person and it is not me. :D Marco! Is that branch really orphaned?

federico

--

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                fog@...
INIT.D Developer                                           fog@...
 A desobediência é uma virtude necessária à criatividade. -- Raul Seixas

_______________________________________________
Psycopg mailing list
Psycopg@...
http://lists.initd.org/mailman/listinfo/psycopg

Re: how to handle large object in psycopg2?

On Tuesday 09 May 2006 4:21 pm, Federico Di Gregorio wrote:
> > LOL
>
> Well.. this is the first time I talk about psycopg's author in
> third person and it is not me. :D

not you? didnt know that

--

-- 
regards
kg

http://www.livejournal.com/users/lawgon
tally ho! http://avsap.org.in
ಇಂಡ್ಲಿನಕ್ಸ வாழ்க!
_______________________________________________
Psycopg mailing list
Psycopg <at> lists.initd.org
http://lists.initd.org/mailman/listinfo/psycopg

Re: how to handle large object in psycopg2?

Hi Federico.

Up to now I did not consider working again on that project... mostly
because of lack of time rather than of interest.

Actually, a good largeobject support for psycopg2 would be great for our
(Zope) applications (we had to devise a workaround at the moment, which
work, but is not very efficient at all).

However, I will try to work again on it... no promise 

Regards

Federico Di Gregorio wrote:

>Il giorno mar, 09/05/2006 alle 16.14 +0530, Kenneth Gonsalves ha
>scritto:
>  
>
>>On Tuesday 09 May 2006 4:03 pm, Federico Di Gregorio wrote:
>>    
>>
>>>Il giorno mar, 09/05/2006 alle 18.14 +0800, 王磊 ha scritto:
>>>      
>>>
>>>>lobject-branch is the only way?
>>>>        
>>>>
>>>Yes, and it was even abandoned by its author.
>>>      
>>>
>>LOL
>>    
>>
>
>Well.. this is the first time I talk about psycopg's author in third
>person and it is not me. :D Marco! Is that branch really orphaned?
>
>federico
>
>  
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Psycopg mailing list
>Psycopg <at> lists.initd.org
>http://lists.initd.org/mailman/listinfo/psycopg
>  
>

_______________________________________________
Psycopg mailing list
Psycopg <at> lists.initd.org
http://lists.initd.org/mailman/listinfo/psycopg