Mike James | 24 Aug 15:19 2015

Slony replication failed invalid memory alloc request size

We have a slony slave in AWS. During an Amazon maintenance, the instance was rebooted and now the daemon won't start. The log file has this error message:


2015-08-24 06:50:03 UTC INFO   remoteWorkerThread_1: syncing set 1 with 102 table(s) from provider 1
2015-08-24 06:50:33 UTC ERROR  remoteWorkerThread_1_1: error at end of COPY IN: ERROR:  invalid memory alloc request size 1970234207
CONTEXT:  COPY sl_log_1, line 97033:

I'm not sure whether this is a Postgres error or a slony error. We're running Postgres 9.3.5 and slony 2.2.1. Any help much appreciated.

Mike
 
 
  Mike James | Manager of Infrastructure
267.419.6400, ext 204 | mike.james-8f6iZ/AGwKbQT0dZR+AlfA@public.gmane.org
201 S Maple St. | Suite 250 | Ambler, PA 19002 
Clutch.com | Twitter | LinkedIn | YouTube | Clutch Support Center
 
The only end to end consumer management platform that empowers consumer-focused businesses to identify, target, message, and engage their best customers.
_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://lists.slony.info/mailman/listinfo/slony1-general
Dave Cramer | 23 Aug 23:56 2015
Picon

Dropping a node when the node is gone ?

Drop node always wants to connect to the node ??

Dave Cramer
_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://lists.slony.info/mailman/listinfo/slony1-general
Tory M Blue | 14 Aug 00:51 2015
Picon

Slony multiple locations, questions concerning node ID's


Been a bit, but I'm back with a question.

I'm looking at doing a migration from one site to another and eventually have a hot warm site.

Right now I have a 5 box slony cluster at site A

Site A: Node 1 and 2 can switch/fail, nodes 3-5 are partial and can't become master.

Node 1: MASTER
Node 2: SLAVE
Node 3: QUERYSLAVE1
Node 4: QUERYSLAVE2
Node 5: QUERYSLAVE3

I have a new 5 node cluster in Site B.

Site B:  This is our default turn up, so they have the same NODE ID as Site A..

Node 1: MASTER
Node 2: SLAVE
Node 3: QUERYSLAVE1
Node 4: QUERYSLAVE2
Node 5: QUERYSLAVE3

I was really interested in Node 2 of Site A replicating to Node 1 of Site B, and then just let the other boxen in site B take their information from Node 1 of the same site.

 I think there is going to be some confusion with NODE ID's. I believe I can tell Site A that the Master of Site B is actually node 10 <whatever> and get it replicating and be able to drop NODE 10 if I needed from site A. However if I end up switching to the Master in Site B,  It won't know how to tell SITE A to kiss off,  as the NODE's in it's config will be 1,2,3,4,5, which are the same as Site A, so there would be no real way for me to clean up and or remove SITE A devices, drop node using NODE ID would be no beuno.

So I guess basically, do I have to have unique NODE ID's (think I do, but if not??). It will just require some rework for every site I bring up,  this means I have a few different configurations and or scripts to use depending on what site the cluster is in.

Is this sort of clear, murky yet you have the general idea, or holy carp Tory what are you asking?

Thanks
Tory
_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://lists.slony.info/mailman/listinfo/slony1-general
Steve Singer | 27 Jul 04:03 2015

Slonik API

During the slony session at the PGCON unconference some people asked 
about an API so they could manipulate slony clusters without having to 
call slonik.

I've done a draft attempt at moving slonik functionality into a library.

You can view my draft API here
https://github.com/ssinger/slony1-engine/blob/libslonik/src/slonik/slonik_api.h

A sample program that uses it might look like

        struct SlonikContext * context;

         SlonikApi_NodeConnInfo n1;
         n1.no_id=1;
         n1.conninfo="host=localhost dbname=test1 port=5435";
         SlonikApi_NodeConnInfo ** n_list =malloc(sizeof( 
SlonikApi_NodeConnInfo *)*2);
         n_list[0]=&n1;
         n_list[1]=NULL;
         context = slonik_api_init_context("disorder_replica",n_list);
         slonik_api_sync(context,1);
         slonik_api_subscribe_set(context,1,1,2,0,0);

The idea is that you would setup a structure with the conninfo 
configuration and init your slony context.  You would then pass this 
context to the function call for the slonik command you want to call.

I'd like to get a sense of how people feel about this API.

If we like the API in general then we still need to figure out how to 
deal with error reporting.  The library still prints errors and messages 
to stdout which isn't ideal for an API.  We could either pass a file 
stream to the context or use callback functions

I haven't yet added support for try blocks but I think that would be 
straightforward to do. I also need to make slonik statically link with 
the library.

I have only done limited testing so far.  If we like this API I will 
then write wrappers to allow the Java JVM to call into libslonik and add 
some tests to the slony test framework.
David Fetter | 25 Jul 06:31 2015

Cloning an origin?

Folks,

While in the best of all possible worlds, we'd have planned out a
replication strategy before we get tables whose initial sync via
"SUBSCRIBE SET" will never finish, we aren't always given that much
ability to plan that soon.

CLONE is great when you want to light an Nth node for N > 2, but
that's just adjusting an extant cluster, not creating one in the first
place.

What stands between the state of the slony code and being able to
clone an origin node?

Cheers,
David.
--

-- 
David Fetter <david@...> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@...

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
Dave Cramer | 30 Jun 22:45 2015
Picon

how to remove an event

I managed to create a path which had a user which was wrong, then executed a subscribe set on that path.

I tried unsubscribe set but the event is still in the logs ?


Dave Cramer
_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://lists.slony.info/mailman/listinfo/slony1-general
Dave Cramer | 30 Jun 19:45 2015
Picon

using slony to go from 8.4 to 9.4

I'm trying to replicate from 8.4 to 9.4

Getting the following error on the store node

could not access file "$libdir/slony1_funcs.2.2.2" This is occurring when trying to install the functions into the 9.4 node

Dave Cramer
_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://lists.slony.info/mailman/listinfo/slony1-general
Mark Steben | 5 Jun 17:09 2015

bloat maintenance on slony internal tables

Good morning,

I track table bloat on all db tables on master and slave. Some of the slony tables, notably sl_apply_stats and sl_components (on the slave) can grow to 1- 2 thousand times their original size due to bloat.  This is a condition that has been prevalent for some time now.  Is there a utility that
I can run to clean this bloat out or do I need to resort to VACUUM FULL?

Thank you,

--
Mark Steben
 Database Administrator
<at> utoRevenue | Autobase 
  CRM division of Dominion Dealer Solutions 
95D Ashley Ave.
West Springfield, MA 01089

t: 413.327-3045
f: 413.383-9567

www.fb.com/DominionDealerSolutions
www.twitter.com/DominionDealer
 www.drivedominion.com





_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://lists.slony.info/mailman/listinfo/slony1-general
Picon

Slony error when insert data

Hello,

I am using Slony1-2.2.4, PostgreSQL 9.4.1. I set replication system by documentation. I have I replicated
table named 'test'. When I start replication, it seems all good. But if I insert some data in test at master
node, slon SYNC failed:

ERROR  remoteListenThread_2: "select ev_origin, ev_seqno, ev_timestamp,        ev_snapshot,       
"pg_catalog".txid_snapshot_xmin(ev_snapshot),       
"pg_catalog".txid_snapshot_xmax(ev_snapshot),        ev_type,        ev_data1, ev_data2,        ev_data3, ev_data4,       
ev_data5, ev_data6,        ev_data7, ev_data8 from "_slony_example".sl_event e where (e.ev_origin = '2' and
e.ev_seqno > '5000000004') order by e.ev_origin, e.ev_seqno limit 40" - server closed the connection unexpectedly

-- 
Best regards, Dmitry Voronin
_______________________________________________
Slony1-general mailing list
Slony1-general <at> lists.slony.info
http://lists.slony.info/mailman/listinfo/slony1-general
Soni M | 17 Apr 05:44 2015
Picon

long 'idle in transaction' from remote slon

Hello All,
2 nodes configured for slony 2.0.7, on RHEL 6.5 using postgres 9.1.14. Each slon manage local postgres.
Slony and RHEL installed from postgres yum repo.

On some occasion, on master db, the cleanupEvent last for long time, up to 5 minutes, normally it finish for a few seconds. The 'truncate sl_log_x' is waiting for a lock which takes most time. This make all write operation to postgres have to wait also, some get failed. As I inspected, what makes truncate wait is another slon transaction made by slon slave process, that is transaction which run 'fetch 500 from LOG'. This transaction left 'idle on transaction' for a long time on some occasion.

Why is this happen ? Is this due to network latency between nodes ? Is there any work around for this?

Many thanks, cheers...

--
Regards,

Soni Maula Harriz
_______________________________________________
Slony1-general mailing list
Slony1-general@...
http://lists.slony.info/mailman/listinfo/slony1-general
David Fetter | 15 Apr 00:56 2015

Multiple slons per node pair?

Folks,

This came up in the context of making slony k-safe for some k>0.

Naively, a simple way to do this would be to have >1 machine, each
running all the slons for a cluster, replacing any machines that fail.

Would Bad Things™ happen as a consequence?

Cheers,
David.
--

-- 
David Fetter <david <at> fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter <at> gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
_______________________________________________
Slony1-general mailing list
Slony1-general <at> lists.slony.info
http://lists.slony.info/mailman/listinfo/slony1-general

Gmane