After some substantial progress on the Slony-I engine development, I'd 
like to give a little status report and invite everyone who wants to 
participate in this project to join the mailing list and the development 
team.

The project homepage is here:

     http://gborg.postgresql.org/project/slony1/projdisplay.php

   ====================================================================

The current development status:

Slony-I configures on RedHat Linux and FreeBSD 4.9 against a PostgreSQL 
7.4. The engine is capable of hot install, hot subscribe with catch up 
and cascaded slaves. The config utilities are either not existent yet or 
ugly, incomplete shell scripts. The replication log trigger is the 
pototypes version in PL/Tcl.

What this odd collection is capable of doing is creating a master, slave 
and cascaded slave replication system with a pgbench database as master. 
The kick is, that the master database is originally just a plain, stock 
PG 7.4.1 with zero changes, and the pgbench application is running non 
stop through the whole process.

   ====================================================================

Next steps:

The log trigger function must be reimplemented in C. I will do this 
during the next couple of days.

Implement the functionality to change the data provider of a slave. With 
that a slave can be added as a cascaded slave, copy the data, catch up 
and then switch over to replicate against the master, or an existing 
slave can become a cascaded one to reduce the load on the master.

Implement the failover capability to replace a failed master with a slave.

Add backup and replication logging (sort of PITR based on replication 
information).

Add sequence replication.

Add script execution support for schema changes.

Both, the provider change and the failover need a much more complex 
configuration than the current shell scripts can setup. The problem is 
that the configuration and administration tools are not designed yet. So 
here is a huge field for others to step up and take a key role in this 
project.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #


Wed, 03 Mar 2004 16:57:28 -0500
Jan Wieck <JanWieck@Yahoo.com>
Slony-I makes progress
On Wed, Mar 03, 2004 at 04:57:28PM -0500, Jan Wieck wrote:
> After some substantial progress on the Slony-I engine development, I'd 
> like to give a little status report and invite everyone who wants to 
> participate in this project to join the mailing list and the development 
> team.

Jan, thank you so much for your hard work on this project.

> Both, the provider change and the failover need a much more complex 
> configuration than the current shell scripts can setup. The problem is 
> that the configuration and administration tools are not designed yet. So 
> here is a huge field for others to step up and take a key role in this 
> project.

So what are you looking for here? When I last built slony, things
mostly worked, but a few niggling details were broken. I was going to
submit a few patches, but when I talked to you, it seemed like you
weren't quite ready for patches. Is the tree stable enough that I could
do some work on it and expect it to remain relatively consistent over a
few hours or even a day or two?

Also, to get this out of the way (as it presently plagues erserver), do
you have a particular language in mind? I'd like to avoid the dogmatic
jihad by not submitting a perl tool if the eventual goal is to be
end-to-end C (or java or tcl or whatever).

Alex

--
alex@posixnap.net
Alex J. Avriette, Unix Systems Gladiator
"Something seems to have happened to the life support system , Dave."

Wed, 3 Mar 2004 19:19:50 -0500
"Alex J. Avriette" <alex@posixnap.net>
Re: Slony-I makes progress
Followup-To: Slony1-general ML

Alex J. Avriette wrote:

> On Wed, Mar 03, 2004 at 04:57:28PM -0500, Jan Wieck wrote:
>> After some substantial progress on the Slony-I engine development, I'd 
>> like to give a little status report and invite everyone who wants to 
>> participate in this project to join the mailing list and the development 
>> team.
> 
> Jan, thank you so much for your hard work on this project.
> 
>> Both, the provider change and the failover need a much more complex 
>> configuration than the current shell scripts can setup. The problem is 
>> that the configuration and administration tools are not designed yet. So 
>> here is a huge field for others to step up and take a key role in this 
>> project.
> 
> So what are you looking for here? When I last built slony, things
> mostly worked, but a few niggling details were broken. I was going to
> submit a few patches, but when I talked to you, it seemed like you
> weren't quite ready for patches. Is the tree stable enough that I could
> do some work on it and expect it to remain relatively consistent over a
> few hours or even a day or two?

What I am looking for is a super-comfortable GUI application that makes 
planning and configuring a master-cascaded-multislave replication system 
doable by everyone who can identify a clickable button.

Honestly, I personally can live with a sh+sed+m4 tool collection. But I 
guess only few would agree to that. So it's basically up to you and 
everyone else around here what the outcome of this is.

What is required to fit into the data-center is a batch utility that can 
be called in a script and that causes a currently failing cluster to 
change the configuration (change the origin of data sets, change 
providers, drop nodes ... that kind of stuff). The same utility would 
ideally be able to setup new nodes etc. so that it can be used as an 
interims solution until the GUI wizzard is ready for prime time.

The current CVS replicates fine and the test_?_pgbench scripts in the 
src/ducttape directory do it all at once. I have changed a couple of 
things in the autoconf stuff. The whole thing is now expected to be 
compiled and installed by the postgres user with --prefix pointing to 
the postgres home directory (the same as the --prefix for the PG 
installation from sources was). The problem here is, that if we ever 
want to create a single C function from a GUI tool on a remote box, its 
shared library better be in the PostgreSQL lib directory so it can be 
... AS '$libdir/objfile' no matter where that is and what extension 
shared objects on that architecture have.

> 
> Also, to get this out of the way (as it presently plagues erserver), do
> you have a particular language in mind? I'd like to avoid the dogmatic
> jihad by not submitting a perl tool if the eventual goal is to be
> end-to-end C (or java or tcl or whatever).

For the production batch commandline utility I think it is C.

Other than that ... I said the field is open.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #


Wed, 03 Mar 2004 23:17:53 -0500
Jan Wieck <JanWieck@Yahoo.com>
Re: Slony-I makes progress