Ron St-Pierre | 1 Sep 02:18 2004

Re: Table UPDATE is too slow

Thanks for everyone's comments (Thomas, Steinar, Frank, Matt, William). 
Right now I'm bench-marking the time it takes for each step
in the end of day update process and then I am going to test a few things:
- dropping most indexes, and check the full processing time and see if 
there is any noticeable performance degradation on the web-end
- wrapping the updates in a transaction, and check the timing
- combining the two
- reviewing my shared_buffers and sort_memory settings

Thanks
Ron

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo <at> postgresql.org)

Tom Lane | 1 Sep 07:32 2004
Picon

Re: Table UPDATE is too slow

Ron St-Pierre <rstpierre <at> syscor.com> writes:
>  Does anyone have some idea on how we can increase speed, either by 
> changing the updates, designing the database
>  differently, etc, etc? This is currently a big problem for us.

>  Other notables:
>    The UPDATE is run from a within a function: FOR rec IN SELECT ...LOOP 
> RETURN NEXT rec; UPDATE dataTable.....

One point that I don't think was made before: by doing a collection of
updates in this serial one-at-a-time fashion, you are precluding any
possibility of query optimization over the collection of updates.  It
might win to dump the update data into a temp table and do a single
UPDATE command joining to the temp table.  Or not --- quite possibly not
--- but I think it's something to think about.

			regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

               http://archives.postgresql.org

Michael Ryan S. Puncia | 1 Sep 08:42 2004
Picon

Changing the column length

Hi ,

            I am sorry that my question is out of line with this group(performance) but I need

an urgent help L …pls .. I need to know how  to change the length of the column.

 

Thanks and hoping that u will not ignore my question

 

Stefano Bonnin | 1 Sep 09:56 2004
Picon

Fw: Query performance issue with 8.0.0beta1


----- Original Message ----- 
From: "Stefano Bonnin" <stefano.bonnin <at> comai.to>
To: "Josh Berkus" <josh <at> agliodbs.com>
Sent: Monday, August 30, 2004 4:13 PM
Subject: Re: [PERFORM] Query performance issue with 8.0.0beta1

> This is my postgres.conf, I have changed only the work_mem and
> shared_buffers parameters.
>
> >DID you
> > configure it for the 8.0 database?
>
> What does it mean? Is in 8.0 some important NEW configation parameter ?
>
> # pgdata = '/usr/local/pgsql/data'              # use data in another
> directory
> # hba_conf = '/etc/pgsql/pg_hba.conf'           # use hba info in another
> directory
> # ident_conf = '/etc/pgsql/pg_ident.conf'       # use ident info in
another
> directory
> # external_pidfile= '/var/run/postgresql.pid'   # write an extra pid file
> #listen_addresses = 'localhost' # what IP interface(s) to listen on;
>                                 # defaults to localhost, '*' = any
> #port = 5432
> max_connections = 100
> #superuser_reserved_connections = 2
> #unix_socket_directory = ''
> #unix_socket_group = ''
> #unix_socket_permissions = 0777 # octal
> #rendezvous_name = ''           # defaults to the computer name
> #authentication_timeout = 60    # 1-600, in seconds
> #ssl = false
> #password_encryption = true
> #krb_server_keyfile = ''
> #db_user_namespace = false
>
> shared_buffers = 2048           # min 16, at least max_connections*2, 8KB
> each
> work_mem = 2048                 # min 64, size in KB
> #maintenance_work_mem = 16384   # min 1024, size in KB
> #max_stack_depth = 2048         # min 100, size in KB
>
> #max_fsm_pages = 20000          # min max_fsm_relations*16, 6 bytes each
> #max_fsm_relations = 1000       # min 100, ~50 bytes each
>
> #max_files_per_process = 1000   # min 25
> #preload_libraries = ''
>
> #vacuum_cost_delay = 0          # 0-1000 milliseconds
> #vacuum_cost_page_hit = 1       # 0-10000 credits
> #vacuum_cost_page_miss = 10     # 0-10000 credits
> #vacuum_cost_page_dirty = 20    # 0-10000 credits
> #vacuum_cost_limit = 200        # 0-10000 credits
>
> #bgwriter_delay = 200           # 10-5000 milliseconds
> #bgwriter_percent = 1           # 1-100% of dirty buffers
> #bgwriter_maxpages = 100        # 1-1000 buffers max at once
>
> #fsync = true                   # turns forced synchronization on or off
> #wal_sync_method = fsync        # the default varies across platforms:
>                                 # fsync, fdatasync, open_sync, or
> open_datasync
> #wal_buffers = 8                # min 4, 8KB each
> #commit_delay = 0               # range 0-100000, in microseconds
> #commit_siblings = 5            # range 1-100
> #checkpoint_segments = 3        # in logfile segments, min 1, 16MB each
> #checkpoint_timeout = 300       # range 30-3600, in seconds
> #checkpoint_warning = 30        # 0 is off, in seconds
>
> #archive_command = ''           # command to use to archive a logfile
> segment
>
> #enable_hashagg = true
> #enable_hashjoin = true
> #enable_indexscan = true
> #enable_mergejoin = true
> #enable_nestloop = true
> #enable_seqscan = true
> #enable_sort = true
> #enable_tidscan = true
>
> #effective_cache_size = 1000    # typically 8KB each
> #random_page_cost = 4           # units are one sequential page fetch cost
> #cpu_tuple_cost = 0.01          # (same)
> #cpu_index_tuple_cost = 0.001   # (same)
> #cpu_operator_cost = 0.0025     # (same)
>
> #geqo = true
> #geqo_threshold = 12
> #geqo_effort = 5                # range 1-10
> #geqo_pool_size = 0             # selects default based on effort
> #geqo_generations = 0           # selects default based on effort
> #geqo_selection_bias = 2.0      # range 1.5-2.0
>
> default_statistics_target = 20  # range 1-1000
> #from_collapse_limit = 8
> #join_collapse_limit = 8        # 1 disables collapsing of explicit JOINs
>
> #log_destination = 'stderr'     # Valid values are combinations of stderr,
>                                 # syslog and eventlog, depending on
>                                 # platform.
>
> # This is relevant when logging to stderr:
> #redirect_stderr = false    # Enable capturing of stderr into log files.
> # These are only relevant if redirect_stderr is true:
> #log_directory = 'pg_log'   # Directory where logfiles are written.
>                             # May be specified absolute or relative to
> PGDATA
> #log_filename_prefix = 'postgresql_' # Prefix for logfile names.
> #log_rotation_age = 1440    # Automatic rotation of logfiles will happen
> after
>                             # so many minutes.  0 to disable.
> #log_rotation_size = 10240  # Automatic rotation of logfiles will happen
> after
>                             # so many kilobytes of log output.  0 to
> disable.
>
> # These are relevant when logging to syslog:
> #syslog_facility = 'LOCAL0'
> #syslog_ident = 'postgres'
>
>
> # - When to Log -
>
> #client_min_messages = notice   # Values, in order of decreasing detail:
>                                 #   debug5, debug4, debug3, debug2,
debug1,
>                                 #   log, notice, warning, error
>
> #log_min_messages = notice      # Values, in order of decreasing detail:
>                                 #   debug5, debug4, debug3, debug2,
debug1,
>                                 #   info, notice, warning, error, log,
> fatal,
>                                 #   panic
>
> #log_error_verbosity = default  # terse, default, or verbose messages
>
> #log_min_error_statement = panic # Values in order of increasing severity:
>                                  #   debug5, debug4, debug3, debug2,
debug1,
>                                  #   info, notice, warning, error,
> panic(off)
>
> #log_min_duration_statement = -1 # -1 is disabled, in milliseconds.
>
> #silent_mode = false             # DO NOT USE without syslog or
> redirect_stderr
>
> # - What to Log -
>
> #debug_print_parse = false
> #debug_print_rewritten = false
> #debug_print_plan = false
> #debug_pretty_print = false
> #log_connections = false
> #log_disconnections = false
> #log_duration = false
> #log_line_prefix = '%t %u %d '  # e.g. '<%u%%%d> '
>                                 # %u=user name %d=database name
>                                 # %r=remote host and port
>                                 # %p=PID %t=timestamp %i=command tag
>                                 # %c=session id %l=session line number
>                                 # %s=session start timestamp
>                                 # %x=stop here in non-session processes
>                                 # %%='%'
> log_statement = 'all'           # none, mod, ddl, all
> #log_hostname = false
>
>
>
#---------------------------------------------------------------------------
> # RUNTIME STATISTICS
>
#---------------------------------------------------------------------------
>
> # - Statistics Monitoring -
>
> #log_parser_stats = false
> #log_planner_stats = false
> #log_executor_stats = false
> #log_statement_stats = false
>
> # - Query/Index Statistics Collector -
>
> #stats_start_collector = true
> #stats_command_string = false
> #stats_block_level = false
> #stats_row_level = false
> #stats_reset_on_server_start = true
>
#---------------------------------------------------------------------------
> # CLIENT CONNECTION DEFAULTS
>
#---------------------------------------------------------------------------
>
> # - Statement Behavior -
>
> #search_path = '$user,public'   # schema names
> #check_function_bodies = true
> #default_transaction_isolation = 'read committed'
> #default_transaction_read_only = false
> #statement_timeout = 0          # 0 is disabled, in milliseconds
>
> # - Locale and Formatting -
>
> #datestyle = 'iso, mdy'
> #timezone = unknown             # actually, defaults to TZ environment
> setting
> #australian_timezones = false
> #extra_float_digits = 0         # min -15, max 2
> #client_encoding = sql_ascii    # actually, defaults to database encoding
>
> # These settings are initialized by initdb -- they might be changed
> lc_messages = 'it_IT.UTF-8'             # locale for system error message
> strings
> lc_monetary = 'it_IT.UTF-8'             # locale for monetary formatting
> lc_numeric = 'it_IT.UTF-8'              # locale for number formatting
> lc_time = 'it_IT.UTF-8'                 # locale for time formatting
>
> # - Other Defaults -
>
> #explain_pretty_print = true
> #dynamic_library_path = '$libdir'
>
>
>
#---------------------------------------------------------------------------
> # LOCK MANAGEMENT
>
#---------------------------------------------------------------------------
>
> #deadlock_timeout = 1000        # in milliseconds
> #max_locks_per_transaction = 64 # min 10, ~260*max_connections bytes each
>
>
>
#---------------------------------------------------------------------------
> # VERSION/PLATFORM COMPATIBILITY
>
#---------------------------------------------------------------------------
>
> # - Previous Postgres Versions -
> #add_missing_from = true
> #regex_flavor = advanced        # advanced, extended, or basic
> #sql_inheritance = true
> #default_with_oids = true
>
> # - Other Platforms & Clients -
>
> #transform_null_equals = false
>
>
>
> ************************
> Thanks
>
> PS. I'm sorry for the late answer but I was not in office.
>
> ----- Original Message ----- 
> From: "Josh Berkus" <josh <at> agliodbs.com>
> To: "Stefano Bonnin" <stefano.bonnin <at> comai.to>;
> <pgsql-performance <at> postgresql.org>
> Sent: Friday, August 27, 2004 7:14 PM
> Subject: Re: [PERFORM] Query performance issue with 8.0.0beta1
>
>
> > Stefano,
> >
> > > Hi, I have just installed 8.0.0beta1 and I noticed that some query are
> > > slower than 7.4.2 queries.
> >
> > Seems unlikely.   How have you configured postgresql.conf?    DID you
> > configure it for the 8.0 database?
> >
> > -- 
> > Josh Berkus
> > Aglio Database Solutions
> > San Francisco
> >
>

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

gnari | 1 Sep 10:47 2004
Picon

Re: Changing the column length

From: "Michael Ryan S. Puncia" <mpuncia <at> census.gov.ph>

>             I am sorry that my question is out of line with this
> group(performance) but I need

-general might be more appropriate

>
> an urgent help :-( .pls .. I need to know how  to change the length of the
> column.

add a new column, use update to copy values from old column,
use alter table to rename columns

gnari

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Stefano Bonnin | 1 Sep 11:18 2004
Picon

Re: Query performance issue with 8.0.0beta1

Server HP: Intel(R) Pentium(R) 4 CPU 2.26GHz
RAM 1GB
OS: RedHat 8
And the disk:

    kernel: megaide: driver version 05.04f (Date: Nov 11, 2002; 18:15 EST)
    kernel: megaide: bios version 02.06.07221605
    kernel: megaide: LD 0 RAID1 status=ONLINE sectors=156297343
capacity=76317 MB drives=2
    kernel: scsi0 : LSI Logic MegaIDE RAID BIOS Version 2.6.07221605, 8
targs 1 chans 8 luns
    kernel:   Vendor: LSILOGIC  Model:  LD  0  IDERAID   Rev:
    kernel:   Type:   Direct-Access                      ANSI SCSI revision:
02

Thanks.
RedS

----- Original Message ----- 
From: "Josh Berkus" <josh <at> agliodbs.com>
To: "Stefano Bonnin" <stefano.bonnin <at> comai.to>
Sent: Monday, August 30, 2004 6:54 PM
Subject: Re: [PERFORM] Query performance issue with 8.0.0beta1

> Stefano,
>
> > This is my postgres.conf, I have changed only the work_mem and
> > shared_buffers parameters.
>
> And not very much, I see.
>
> > >DID you
> > > configure it for the 8.0 database?
> >
> > What does it mean? Is in 8.0 some important NEW configation parameter ?
>
> Well, there are but they shouldn't affect your case.     However, there
are a
> lot of other settings that need to be adjusted.   Please post your
hardware
> plaform information:  OS, CPU, RAM, and disk setup.
>
> -- 
> Josh Berkus
> Aglio Database Solutions
> San Francisco
>

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo <at> postgresql.org

G u i d o B a r o s i o | 1 Sep 12:06 2004
Picon

slower every day

Dear all,

  I am currently experiencing troubles with the performance of my critical's database.

  The problem is the time that the postgres takes to perform/return a query. For example, trying the \d
<tablename> command takes between 4 or 5 seconds. This table is very big, but I am not asking for the rows,
only asking the table schema, so...why is this so slow?!?!? My last administrative action into this table
was a reindex to all the indexes via the BKI in standalone mode. I thought I suceed, but this was las
saturday. Today I am in the same situation again.

  The only change that I've done was a highest level of debug in the conf file (loggin lot of stuff). 
  I understand that this could lack on performance, but when I've changed the .conf file to the usual .conf
file (with less debug), and pg_ctl reload(ed) it, it goes on debuging as in the first state, in the higher
level. Is this a known issue? 

  My conclusion is that I can aquire high levels of debug while the server is running, editing the .conf file,
and pg_reload(ing) it, but I can go back then, unless I pg_restart the server. Is this ok?

Some info
-----------------------------------------------------------
PostgreSQL 7.4.2
[postgres <at> lmnukmis02 data]$ pg_config --configure
'--enable-thread-safety' '--with-perl'
Intel(R) Xeon(TM) MP CPU 2.80GHz
Linux 2.4.24-ck1 #5 SMP Fri Mar 12 23:41:51 GMT 2004 i686 unknown
RAM 4 Gb.
-----------------------------------------------------------

Thanks, Guido.

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo <at> postgresql.org

Shridhar Daithankar | 1 Sep 12:25 2004
Picon

Re: slower every day

On Wednesday 01 Sep 2004 3:36 pm, G u i d o B a r o s i o wrote:
> Dear all,
>
>   I am currently experiencing troubles with the performance of my
> critical's database.
>
>   The problem is the time that the postgres takes to perform/return a
> query. For example, trying the \d <tablename> command takes between 4 or 5
> seconds. This table is very big, but I am not asking for the rows, only
> asking the table schema, so...why is this so slow?!?!? My last
> administrative action into this table was a reindex to all the indexes via
> the BKI in standalone mode. I thought I suceed, but this was las saturday.
> Today I am in the same situation again.

Is this database vacuumed and analyzed recently? I would suggest database-wide 
vacuum full analyze.

If your queries are getting slower, then checking the explain analyze output 
is a good starting point. To see queries issued by psql, start it as psql -E.

HTH

 Shridhar

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Peter Eisentraut | 1 Sep 12:28 2004
Picon
Picon

Re: slower every day

Am Mittwoch, 1. September 2004 12:06 schrieb G u i d o B a r o s i o:
>   The problem is the time that the postgres takes to perform/return a
> query. For example, trying the \d <tablename> command takes between 4 or 5
> seconds. This table is very big, but I am not asking for the rows, only
> asking the table schema, so...why is this so slow?!?!? My last
> administrative action into this table was a reindex to all the indexes via
> the BKI in standalone mode. I thought I suceed, but this was las saturday.

Do you regularly vacuum and analyze the database?

--

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

G u i d o B a r o s i o | 1 Sep 12:57 2004
Picon

Re: [ADMIN] slower every day

The solution appeared as something I didn't know

  On the .conf file

Previous situation:

#log_something=false
log_something=true

Worst situation 
#log_something=false
#log_something=true 

Nice situation
log_something=false
#log_something=true

Ok, the problem was that I assumed that commenting a value on
the conf file will set it up to a default (false?). I was wrong.
My server was writting tons of log's.

Is this the normal behavior for pg_ctl reload? It seems that looks for new values, remembering the last
state on the ones that actually are commented. Although it's my fault to have 2 (tow) lines for the same
issue, and that I should realize that this is MY MISTAKE, the log defaults on a reload, if commented, tend to
be the last value entered?

Regards,
Guido

> Am Mittwoch, 1. September 2004 12:06 schrieb G u i d o B a r o s i o:
> >   The problem is the time that the postgres takes to perform/return a
> > query. For example, trying the \d <tablename> command takes between 4 or 5
> > seconds. This table is very big, but I am not asking for the rows, only
> > asking the table schema, so...why is this so slow?!?!? My last
> > administrative action into this table was a reindex to all the indexes via
> > the BKI in standalone mode. I thought I suceed, but this was las saturday.
> 
> Do you regularly vacuum and analyze the database?
> 
> -- 
> Peter Eisentraut
> http://developer.postgresql.org/~petere/
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo <at> postgresql.org


Gmane