Re: using COPY FROM

I'm experimenting with curs.copy_from(), with no success.

My Python code is very simple, much like the copy_from example from the
docs. Yet my little program, which should load 3 lines of data in a test
table, completes without errors, but the data never appear in the db table.

One thing that bothers me a lot is the lack of information about what is
going on. Which kind of information is available on the completion (or lack
of..) of a copy operation? Is it there an exit code, at the very least?
This sort of information is imperative for successfully writing dependable
data load procedures.
I'm probably overlooking something, any help?

ciao
Guido

Re: using COPY FROM

Il mar, 2002-11-05 alle 22:59, gnotari@... ha scritto:
> 
> I'm experimenting with curs.copy_from(), with no success.
> 
> My Python code is very simple, much like the copy_from example from the
> docs. Yet my little program, which should load 3 lines of data in a test
> table, completes without errors, but the data never appear in the db table.

are you calling .commit() on the connection?

--

-- 
Federico Di Gregorio
Debian GNU/Linux Developer & Italian Press Contact        fog@...
INIT.D Developer                                           fog@...
     One key. One input. One enter. All right. -- An american consultant
           (then the system crashed and took down the *entire* network)

Re: using COPY FROM

On 06/11/2002 01.22.52 psycopg-admin wrote:

>  Il mar, 2002-11-05 alle 22:59, gnotari@... ha scritto:
>  >
>  > I'm experimenting with curs.copy_from(), with no success.
>  >
>  > My Python code is very simple, much like the copy_from example from
the
>  > docs. Yet my little program, which should load 3 lines of data in a
test
>  > table, completes without errors, but the data never appear in the db
table.
>
>  are you calling .commit() on the connection?

Indeed not. That solved the problem, lines got written. It wasn't really
obvious, to me at least...

I've had no problem in starting a child process with popen2 and having it
write into the database (with copy_from) data the parent process writes to
it.

Again, the problem is with feedback. COPY, as implemented on Postgres,
gives next no none.
I think the most practical way is to SELECT afterwards, counting the rows
actually written. You can then report or act on the operation result.

ciao
Guido

Re: using COPY FROM

Il mer, 2002-11-06 alle 11:45, Guido Notari ha scritto:

> Indeed not. That solved the problem, lines got written. It wasn't really
> obvious, to me at least...

good.

> I've had no problem in starting a child process with popen2 and having it
> write into the database (with copy_from) data the parent process writes to
> it.

nice.

> Again, the problem is with feedback. COPY, as implemented on Postgres,
> gives next no none.
> I think the most practical way is to SELECT afterwards, counting the rows
> actually written. You can then report or act on the operation result.

i think so. anyway, not that if a method doesnot raise an exception
everything went well. ot it should be so, if it does not happen, please
report it as a bug.

--

-- 
Federico Di Gregorio
Debian GNU/Linux Developer & Italian Press Contact        fog@...
INIT.D Developer                                           fog@...
  We are all dust, Saqi, so play the lute
                    We are all wind, Saqi, so bring wine. -- Omar Khayam

Small patch for ZPsycopgDA

Hi,

here's a small patch to import ImageFile from Globals, not from the 
ImageFile module (usually deleted for using PIL with Zope).

Thanks,

Yves

diff -ru psycopg-1.0.13.orig/ZPsycopgDA/DA.py psycopg-1.0.13/ZPsycopgDA/DA.py
--- psycopg-1.0.13.orig/ZPsycopgDA/DA.py	Wed Nov  6 14:38:43 2002
+++ psycopg-1.0.13/ZPsycopgDA/DA.py	Wed Nov  6 14:39:13 2002
 <at>  <at>  -91,8 +91,7  <at>  <at> 

 from db import DB
 import Shared.DC.ZRDB.Connection, sys, DABase, time
-from Globals import HTMLFile
-from ImageFile import ImageFile
+from Globals import HTMLFile, ImageFile
 from ExtensionClass import Base
 from string import find, join, split, rindex
 try:
diff -ru psycopg-1.0.13.orig/ZPsycopgDA/DABase.py psycopg-1.0.13/ZPsycopgDA/DABase.py
--- psycopg-1.0.13.orig/ZPsycopgDA/DABase.py	Wed Nov  6 14:38:53 2002
+++ psycopg-1.0.13/ZPsycopgDA/DABase.py	Wed Nov  6 14:39:24 2002
 <at>  <at>  -90,8 +90,7  <at>  <at> 

 import Shared.DC.ZRDB.Connection, sys
 from App.Dialogs import MessageDialog

Re: Small patch for ZPsycopgDA

Il mer, 2002-11-06 alle 14:53, Yves Bastide ha scritto:
> Hi,
> 
> here's a small patch to import ImageFile from Globals, not from the 
> ImageFile module (usually deleted for using PIL with Zope).

applied. thank you very much.

--

-- 
Federico Di Gregorio
Debian GNU/Linux Developer & Italian Press Contact        fog@...
INIT.D Developer                                           fog@...
   God is real. Unless declared integer. -- Anonymous FORTRAN programmer

Exception handling

Hello all,

There hasn't  been much (any ?) discussion or comments (at least 
not on the list) regarding intelligent exception handling that 
would enable more PostgreSQL cleverness and less scripting. I 
was hoping Edmund Lian's comments would be followed by more 
discussion.

Perhaps Frederico's been getting some input to his request for 
mapping from postgres errors to DBAPI-2.0 exceptions. But 
mapping from postgres errors to DBAPI-2.0 exceptions, doesn't 
really address the issue.

Here's an illustration:
____________________________________________________________

CREATE TABLE tentest (
    numero    int
);

CREATE FUNCTION maxten () RETURNS OPAQUE AS '
     BEGIN
         IF NEW.numero > 10 THEN
             RAISE EXCEPTION ''Woo Hoo !'';
         END IF;

         RETURN NEW;
     END;
' LANGUAGE 'plpgsql';

Re: Exception handling

On 11/08/2002 01:03:48 PM Joel wrote:

>There hasn't  been much (any ?) discussion or comments (at least
>not on the list) regarding intelligent exception handling that
>would enable more PostgreSQL cleverness and less scripting. I
>was hoping Edmund Lian's comments would be followed by more
>discussion.
>
>Perhaps Frederico's been getting some input to his request for
>mapping from postgres errors to DBAPI-2.0 exceptions. But
>mapping from postgres errors to DBAPI-2.0 exceptions, doesn't
>really address the issue.

I've been quietly contemplating my navel... Actually, I took a step back to
see what better exception handling in psycopy would buy us. My conclusion:
not much, but it depends on the scope of solution that you're talking
about.

On the PostgreSQL specific level, more refined exceptions would allow us to
exploit PostgreSQL more fully without resorting to application level
scripting. But you can do this right now by trapping the ProgrammingError
exceptions that psycopg now generates, parsing the error message, and then
raising a more detailed exception.

In the absence of standards, I've just gone and created an abstraction of
my own, so that any code I build on top of this receives filtered
exceptions rather than whatever the underlying DB module generates.

The wider problem isn't that psycopg generates few standardized exceptions,

Re: Exception handling

Joel Rodrigues wrote:
> Hello all,
> 
> There hasn't  been much (any ?) discussion or comments (at least not on 
> the list) regarding intelligent exception handling that would enable 
> more PostgreSQL cleverness and less scripting. I was hoping Edmund 
> Lian's comments would be followed by more discussion.
> 

What I think may be interesting (not sure as I just thought of it and 
haven't contemplated it for long) is for the various PostgreSQL adaptors 
to come to some common exception handling scheme.  We have PyPgSQL which 
is nice, psycopg which is nice, popy and pygresql which i don't use.

Maybe cross-database is a bit too much to bite off at once?

Tom

Re: Exception handling

My apologies for my example not being complete. I forgot to 
include the result of running amelie.py from the command line 
with  a value larger than 10. The function, remember, is :

IF NEW.numero > 10 THEN
     RAISE EXCEPTION ''Woo Hoo !'';

Passing CGI values to the script as it stood with:

iso3dnum = form.getvalue("iso3dnum", "11")

curs.execute("""INSERT INTO tmp3dtest VALUES (%s)""", (iso3dnum))

RESULTS IN:

[localhost:WebServer/CGI-Executables/hypatiacgi] mark% pychecker 
amelie.py
Processing amelie...
   Caught exception importing module amelie:
     File "/usr/local/lib/python2.2/site-
packages/pychecker/checker.py", line 505, in setupMainCode()
       module = imp.load_module(self.moduleName, file, filename, smt)
     File "amelie.py", line 11
       curs.execute("""INSERT INTO tmp3dtest VALUES (%s)""", (iso3dnum))
   ProgrammingError: not all arguments converted

Warnings...

amelie:1: NOT PROCESSED UNABLE TO IMPORT