Jan Wieck | 1 Aug 2005 02:31
Picon
Favicon

Re: How to avoid or reduce LISTEN/NOTIFY use ?

On 7/29/2005 2:14 PM, Christopher Browne wrote:
> Vivek Khera wrote:
> 
>> On Jul 29, 2005, at 5:16 AM, Hannu Krosing wrote:
>>
>>> I'd actually like the choice of listen/notify or polling to be a
>>> configuration decision made by administrator.
>>>
>>
>> It is my experience that such settings are usually set incorrectly 
>> when left to the human...  I suppose the recommended setting would be 
>> "auto" so it could switch back and forth depending on load.
> 
> Right.
> 
> And it seems to me that the reason for a "dogmatic" position would be
> that someone might prefer to "never use NOTIFY" because of some
> perceived problem of the past, a perceived problem which a competent
> policy should transform into an incorrect perception.
> 
> The elimination of CONFIRM notifications should already cut pg_listener
> traffic in half; a reasonable "switchover" policy should virtually
> eliminate the traffic for heavily updated servers.

Don't forget that WAIT FOR EVENT relies on CONFIRM's to come in. If your 
make their delivery random on "something happens anyway", you might end 
up waiting forever.

Jan

(Continue reading)

Jacek Konieczny | 1 Aug 2005 11:14
Picon

Log shipping fails when the disk is full

I have been experimenting with Slony-I (1.1.0) log shipping features.
That would work for me, but there is one, big problem: when my
filesystem is full slon daemon writes empty log files instead of
stopping until normal logging may continue. I thought it is not supposed
to work this way, so I looked into the sources and found out, that most
of log write errors are ignored, so one the logs are not reliable. Is is
really supposed to be that way?

Greets,
	Jacek
Sun.betty | 1 Aug 2005 11:30
Picon
Favicon

About use perl admin tool

Hello!

              All!

I configure /u2/slave/pgsql/etc/slon_tools.conf

# $Id: slon_tools.conf-sample,v 1.5 2005/03/10 17:50:03 smsimms Exp $
# Author: Christopher Browne
# Copyright 2004 Afilias Canada
# Revised extensively by Steve Simms

# Keeping the following three lines for backwards compatibility in
# case this gets incorporated into a 1.0.6 release.
#
# TODO: The scripts should check for an environment variable
# containing the location of a configuration file.  That would
# simplify this configuration file and allow Slony-I tools to still work
# in situations where it doesn't exist.
#
if ($ENV{"SLONYNODES"}) {
    require $ENV{"SLONYNODES"};
} else {

    # The name of the replication cluster.  This will be used to
    # create a schema named _$CLUSTER_NAME in the database which will
    # contain Slony-related data.
    $CLUSTER_NAME = '_golf1';

    # The directory where Slony should record log messages.  This
    # directory will need to be writable by the user that invokes
    # Slony.
    $LOGDIR = '/u2/slave/slonyI/log';

    # (Optional) If you would like to use Apache's rotatelogs tool to
    # manage log output, uncomment the following line and ensure that
    # it points to the executable.
    #
    # $APACHE_ROTATOR = '/usr/local/apache/bin/rotatelogs';

    # Which node is the default master for all sets?
    $MASTERNODE = 1;

    # Include add_node lines for each node in the cluster.  Be sure to
    # use host names that will resolve properly on all nodes
    # (i.e. only use 'localhost' if all nodes are on the same host).
    # Also, note that the user must be a superuser account.

    add_node(node     => 1,
      host     => '10.10.10.67',
      dbname   => 'master',
      port     => 7543,
      user     => 'slave',
             password => '123');

    add_node(node     => 2,
      host     => '10.10.10.67',
      dbname   => 'slave',
      port     => 7543,
      user     => 'slave',
             password => '123');

#    add_node(node     => 3,
#      host     => 'server3',
#      dbname   => 'database',
#      port     => 5432,
#      user     => 'postgres',
#             password => '');

    # If the node should only receive event notifications from a
    # single node (e.g. if it can't access the other nodes), you can
    # specify a single parent.  The downside to this approach is that
    # if the parent goes down, your node becomes stranded.

#    add_node(node     => 4,
#      parent   => 3,
#      host     => 'server4',
#      dbname   => 'database',
#      port     => 5432,
#      user     => 'postgres',
#             password => '');

}

# The $SLONY_SETS variable contains information about all of the sets
# in your cluster.

$SLONY_SETS = {

    # A unique name for the set
    "set1" => {

 # The set_id, also unique
 "set_id" => 1,

 # Uncomment the following line to change the origin
 # (a.k.a. master) for the set.  The default is $MASTERNODE.
 #
  "origin" => 1,

 # The first ID to use for tables and sequences that are added
 # to the replication cluster.  This must be unique across the
 # cluster.
 #
 # TODO: This should be determined automatically, which can be
 # done fairly easily in most cases using psql.  create_set
 # should derive it, and give an option to override it with a
 # specific value.
 "table_id"    => 1,
 "sequence_id" => 1,

 # This array contains a list of tables that already have
 # primary keys.
        "pkeyedtables" => [
   't_account_basic_info',
            't_add_preorder',
      ],

 # For tables that have unique not null keys, but no primary
 # key, enter their names and indexes here.
 "keyedtables" => {
     't_player_score' => 'ak_t_player_score_t_player',
 },

 # If a table does not have a suitable key or set of keys that
 # can act as a primary key, Slony can add one.
 #
 # Note: The Slony development team does not recomment this
 # approach -- you should create your own primary keys instead.
 "serialtables" => ["t_coach_fee_setup" ,
                      "t_player_score" ,],

 # Sequences that need to be replicated should be entered here.
 "sequences" => ['seq_my_course_id',
         'seq_account_basic_info_account_id',
   ],
    },

#    "set2" => {
# "set_id"       => 2,
# "table_id"     => 6,
# "sequence_id"  => 3,
# "pkeyedtables" => ["table6"],
# "keyedtables"  => {},
# "serialtables" => [],
# "sequences"    => [],
#    },

};

# Keeping the following three lines for backwards compatibility in
# case this gets incorporated into a 1.0.6 release.
#
# TODO: The scripts should check for an environment variable
# containing the location of a configuration file.  That would
# simplify this configuration file and allow Slony tools to still work
# in situations where it doesn't exist.
#
if ($ENV{"SLONYSET"}) {
    require $ENV{"SLONYSET"};
}

# Please do not add or change anything below this point.
1;

then

cd /u2/slave/pgsql/admin/bin

$ ./slon_watchdog2 --help
Usage: ./slon_watchdog node sleep-time

$ slon_watchdog2 1 2
psql: warning: extra command-line argument "--tuples-only" ignored
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/tmp/.s.PGSQL.0"?
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/tmp/.s.PGSQL.0"?
/u2/slave/slonyI/log/slony1/node/*log: No such file or directory

or

$ ./slon_watchdog 1 2
sh: /u2/slave/pgsql/admin/bin/restart_node: not found
<stdin>:1: ERROR: syntax error at or near
Invoke slon for node 1 - /u2/slave/pgsql/bin/slon -s 1000 -d2 -g 80 _golf1 'host=10.10.10.67 dbname=master user=slave port=7543 password=123' 2>&1 > /u2/slave/slonyI/log/slony1/node1/master-2005-08-01_17:32:01.log &
sh: /u2/slave/pgsql/admin/bin/restart_node: not found
<stdin>:1: ERROR: syntax error at or near
Invoke slon for node 1 - /u2/slave/pgsql/bin/slon -s 1000 -d2 -g 80 _golf1 'host=10.10.10.67 dbname=master user=slave port=7543 password=123' 2>&1 > /u2/slave/slonyI/log/slony1/node1/master-2005-08-01_17:32:05.log &
sh: /u2/slave/pgsql/admin/bin/restart_node: not found
<stdin>:1: ERROR: syntax error at or near

if my usage is incorrect ?  how to use is correct ? who use this perl admin tool ? maybe know how to use correct . If you know Please give me some advise . Thanks very much !

__________________________________________________
赶快注册雅虎超大容量免费邮箱?
http://cn.mail.yahoo.com

_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://gborg.postgresql.org/mailman/listinfo/slony1-general
cbbrowne | 1 Aug 2005 16:55

Re: Log shipping fails when the disk is full

> I have been experimenting with Slony-I (1.1.0) log shipping features.
> That would work for me, but there is one, big problem: when my
> filesystem is full slon daemon writes empty log files instead of
> stopping until normal logging may continue. I thought it is not supposed
> to work this way, so I looked into the sources and found out, that most
> of log write errors are ignored, so one the logs are not reliable. Is is
> really supposed to be that way?

It is certainly supposed to fail when disk errors occur so that you don't
have empty logs.

If you can point out locations of insufficient error checking, that would
be helpful.

I have tried it out with a full disk and found it stopping; it sounds very
rather peculiar that that wouldn't happen for you.

In particular, every time a query is written out (e.g. -
INSERT/UPDATE/DELETE), the return codes are checked, and things roll back
if the writes do not succeed.

There is the possibility that maybe your platform is providing unexpected
return codes; can you detail operating system version, please?
Rod Taylor | 2 Aug 2005 22:41
Picon

Improper Replication of structure

I'm not sure how it happened, but I currently have a synchronization
error within the data in a structure between 2 databases.

Out of approx. 1 million entries, the discrepancy includes 8000 tuples.

        Standard Slony 1.0.5
        Source DB is PostgreSQL 8.0.3
        Destination DB is PostgreSQL 7.4.6

There have been no errors in the the Slony log files. Replication is not
lagging (0 events behind in both directions), nor are there more than a
few hundred entries in the sl_log_1 structure.

Any thoughts before I reset these structures?

--

-- 
Hannu Krosing | 2 Aug 2005 23:30
Picon

Re: Improper Replication of structure

On T, 2005-08-02 at 16:41 -0400, Rod Taylor wrote:
> I'm not sure how it happened, but I currently have a synchronization
> error within the data in a structure between 2 databases.
> 
> Out of approx. 1 million entries, the discrepancy includes 8000 tuples.
>         
>         Standard Slony 1.0.5
>         Source DB is PostgreSQL 8.0.3
>         Destination DB is PostgreSQL 7.4.6
> 
> There have been no errors in the the Slony log files. Replication is not
> lagging (0 events behind in both directions), nor are there more than a
> few hundred entries in the sl_log_1 structure.
> 
> Any thoughts before I reset these structures?

What do you mean by "structures" ? 

Slony replicates only data, not structure.

Do you have any tables with many triggers (before and after, changing
tables back and forth) in your replicated set.

We've had problems with complicated trigger paths at some point.

--

-- 
Hannu Krosing <hannu@...>
Rod Taylor | 2 Aug 2005 23:28
Picon

Re: Improper Replication of structure

On Wed, 2005-08-03 at 00:30 +0300, Hannu Krosing wrote:
> On T, 2005-08-02 at 16:41 -0400, Rod Taylor wrote:
> > I'm not sure how it happened, but I currently have a synchronization
> > error within the data in a structure between 2 databases.
> > 
> > Out of approx. 1 million entries, the discrepancy includes 8000 tuples.
> >         
> >         Standard Slony 1.0.5
> >         Source DB is PostgreSQL 8.0.3
> >         Destination DB is PostgreSQL 7.4.6
> > 
> > There have been no errors in the the Slony log files. Replication is not
> > lagging (0 events behind in both directions), nor are there more than a
> > few hundred entries in the sl_log_1 structure.
> > 
> > Any thoughts before I reset these structures?
> 
> What do you mean by "structures" ? 

I intend to completely fry the Slony installation and start from
scratch.

It's obviously corrupted in some way or another.

> Do you have any tables with many triggers (before and after, changing
> tables back and forth) in your replicated set.

The only triggers are internal to PostgreSQL for foreign keys.

--

-- 
Christian Storm | 3 Aug 2005 22:03

Large Update Not Completing

We performed one large transactional update (2.3 million rows) on our  
master db yesterday
that took roughly 15 minutes to execute.  Seventeen hours later it  
still hasn't cleared through to
the two slave subscribers.  The status for one of the subscribers is:

st_origin                       | 1
st_received                  | 3
st_last_event               | 579105
st_last_event_ts          | 2005-08-03 12:10:21.826932
st_last_received          | 536291
st_last_received_ts     | 2005-08-03 12:07:52.816493
st_last_received_event_ts | 2005-08-02 18:34:29.121862
st_lag_num_events    | 42814
st_lag_time                   | 17:35:54.432473

After pouring over the literature I haven't been able to determine  
how I can figure out where this big update stands on the slaves by
looking at the slony logs or sl_* tables.  I'm trying to figure out  
whether it is stalled or how much has been processed.  In short,  I'm  
trying to figure out whether we should wait (its almost done) or just  
start over and resync the subscribers.

Does anyone know how to figure this out?

We are using Slony 1.0.5 with Postgres 7.4.6.

Thanks ahead of time for any help.

Christian
Andrew Gold | 4 Aug 2005 01:23

Safe to delete from pg_namespace?

Hi everyone,

I've used the sample scripts to set up my first attempt at replication. 
I screwed up on one of the lines and forgot to comment out the line 
where it adds an index to the public.history table. The script quits as 
it should but cannot be rerun as it exists because the new namespace 
(_slony1, in my example) has been created.

I would like to rerun the script, but I need to clean up the partial 
initialization (init cluster). Can I safely delete the entry in 
pg_namespace, or are there side effects? Is there another way to clean 
up a failed initialization? I am adding a pre-existing table from a live 
database, so I need to be sure that removing the entry in pg_namespace 
doesn't cause problems, although I suppose I could just create the same 
scenario on a test machine ...

The table is a junk table which I created for testing, but the parent 
database is valid.
set add table (set id=1, origin=1, id=1, fully qualified name = 
'public.spam', comment='test table')

I'd like to know for sure.

Thanks,

Andrew Gold
Jan Wieck | 4 Aug 2005 03:19
Picon
Favicon

Re: Safe to delete from pg_namespace?

On 8/3/2005 7:23 PM, Andrew Gold wrote:
> Hi everyone,
> 
> I've used the sample scripts to set up my first attempt at replication. 
> I screwed up on one of the lines and forgot to comment out the line 
> where it adds an index to the public.history table. The script quits as 
> it should but cannot be rerun as it exists because the new namespace 
> (_slony1, in my example) has been created.
> 
> I would like to rerun the script, but I need to clean up the partial 
> initialization (init cluster). Can I safely delete the entry in 
> pg_namespace, or are there side effects? Is there another way to clean 
> up a failed initialization? I am adding a pre-existing table from a live 
> database, so I need to be sure that removing the entry in pg_namespace 
> doesn't cause problems, although I suppose I could just create the same 
> scenario on a test machine ...
> 
> The table is a junk table which I created for testing, but the parent 
> database is valid.
> set add table (set id=1, origin=1, id=1, fully qualified name = 
> 'public.spam', comment='test table')
> 
> I'd like to know for sure.

There's a special slonik command for that.

http://gborg.postgresql.org/project/slony1/genpage.php?slonik_commands#stmt_uninstall_node

Jan

> 
> Thanks,
> 
> Andrew Gold
> _______________________________________________
> Slony1-general mailing list
> Slony1-general@...
> http://gborg.postgresql.org/mailman/listinfo/slony1-general

Gmane