Plone && Relstorage && Postgresql && Async Replication
So you don't have the cash to pay for ZRS? RubyRep does Master/Master replication. Well it will also do A ->B A->C A->D etc...
This will have 1 plone server and two database servers, both having a full copy of the plone data
Plone server
build a Ubuntu machine and deploy out a Plone site however you are comfortable. Now modify it as follows.
Relstorage
make sure
sudo apt-get install libpq-devFilesystem to DB BLOBS
To convert from Blobs on filestorage to Postgres:
it requires the blob information from your zope.conf
zodbconvert.cfg
<filestorage source>
path /opt/plone/zeocluster/var/filestorage.bkp/Data.fs
blob-dir /opt/plone/zeocluster/var/blobstorage.bkp
</filestorage>
<relstorage destination>
blob-dir /tmp/blobcache
shared-blob-dir false
<postgresql>
dsn dbname='plonedb1' user='plone' host='10.0.x.x' password='passwordhere'
</postgresql>
</relstorage>
Postgresql
build two ubuntu machines then upgrade ubuntu to 11.10 via do-release-upgrade
sudo su
apt-get install postgresql-9.1 postgresql-plperl postgresql-client ssl-cert libsys-hostname-long-perl libsys-syslog-perl libboolean-perl libdbix-safe-perl libdbd-pg-perl postgresql-contrib
make sure postgresql.conf is allowing TCP/IP connections by modifying
vim /etc/postgresql/9.1/main/pg_hba.conf
add to bottom:
host plonedb1 all 0.0.0.0 0.0.0.0 md5
RubyRep Installation
sudo apt-get install gawk unzip wget
wget http://rubyforge.org/frs/download.php/74408/rubyrep-1.2.0.zip
Put the following into file /opt/rubyrep/rubyrep.conf
PROXY=0
CONFIG="plonedbsync.conf"
RUBYREP="/opt/rubyrep/rubyrep"
mkdir /var/log/rubyrep
touch /var/log/rubyrep/replication.log
Put the following into the file /opt/rubyrep/plondbsync.conf
RR::Initializer::run do |config|
config.left = {
:adapter => 'postgresql',
:database => 'plonedb1',
:username => 'replicationuser',
:password => 'xxxxx',
:host => 'x.x.x.b',
:pg_params => '?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory',
}
config.right = {
:adapter => 'postgresql',
:database => 'plonedb1',
:username => 'replicationuser',
:password => 'xxxxx',
:host => 'x.x.x.a',
:pg_params => '?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory',
}
config.options[:sync_conflict_handling] = :left_wins
config.options[:replication_conflict_handling] = :left_wins
config.include_tables /./ # regexp matching all tables in the database
config.exclude_tables /commit_lock$/
Create the file /etc/init.d/rubyrep and add the following:
#!/bin/bash
#
# /etc/init.d/rubyrep
#
# Written by David Post
# Functions provided by this file
# * Start
# * Stop
# * Restart
# * Sync
# * Status
# Configuration for use with this script should be kept in /etc/rubyrep/rubyrep.conf
# The only configuration that I am writing right now is proxy, conf and rubyrep.
if [ -f /opt/rubyrep/rubyrep.conf ]
then
. /opt/rubyrep/rubyrep.conf
fi
if [ ! -n "$CONFIG" ]
then
echo "No config file specified in rubyrep.conf"
exit 1
fi
[ -r /opt/rubyrep/"$CONFIG" ] || { echo "File not found: $CONFIG"; exit 1; }
CONFIG=/opt/rubyrep/"$CONFIG"
# These are some functions to make this check if RubyRep is running before starting it.
# This checks if it is running or not
# RUN = is it running proxy or replicate, this may be different than the configuration file says. e.g.
# The just changed the config file
function test_running(){
PID=""
if (("$PROXY"))
then
PID=`ps -Af | grep rubyrep| grep proxy | grep java |gawk '{print $2}'`
if [ -n "$PID" ]
then
RUN=P
else
PID=`ps -Af | grep rubyrep| grep replicate | grep java |gawk '{print $2}'`
[ ! -n "$PID" ] || RUN=R
fi
else
PID=`ps -Af | grep rubyrep| grep replicate | grep java |gawk '{print $2}'`
if [ -n "$PID" ]
then
RUN=R
else
PID=`ps -Af | grep rubyrep| grep proxy | grep java |gawk '{print $2}'`
[ ! -n "$PID" ] || RUN=P
fi
fi
}
case "$1" in
start)
test_running
if [ -n "$PID" ]
then
echo "RubyRep already running."
exit 0
fi
if (("$PROXY"))
then
echo "Starting RubyRep proxy"
$RUBYREP proxy 2>&1 | gawk '{print strftime("%Y-%m-% %T",systime()), $0; fflush();}' >> /var/log/rubyrep/proxy.log &
else
echo "Starting rubyrep replication"
$RUBYREP replicate -c $CONFIG 2>&1 | gawk '{print strftime("%Y-%m-%d %T",systime()), $0; fflush();}' >> /var/log/rubyrep/replication.log &
fi
;;
stop)
test_running
case "$RUN" in
P)
echo "Stopping RubyRep proxy"
kill `ps -Af | grep rubyrep | grep proxy | grep java | gawk '{print $2}'` > /dev/null
;;
R)
echo "Stopping RubyRep replication"
kill `ps -Af | grep rubyrep | grep replicate | grep java | gawk '{print $2}'` > /dev/null
;;
*)
echo "RubyRep not running"
;;
esac
;;
restart)
test_running
if [ -n "$PID" ]
then
$0 stop
sleep 1
fi
$0 start
;;
sync)
echo "Syncing databases..."
/opt/rubyrep/rubyrep "sync -c $CONFIG"
;;
status)
if (("$PROXY"))
then
echo -e "RubyRep Proxy is \\c"
ps -Af | grep rubyrep| grep proxy | grep java >/dev/null || echo -e not \\c
else
echo -e "RubyRep Replication is \\c"
ps -Af | grep rubyrep| grep replicate | grep java >/dev/null || echo -e not \\c
fi
echo running
;;
test) # This ones sort of a secret that I used for testing if the function test_running() was working or not
test_running
if [ -n "$PID" ]
then
echo "RubyRep is running on PID $PID"
else
echo "RubyRep is not running"
fi
;;
*)
echo "Usage: $0 {start|stop|restart|sync|status}"
;;
esac
exit 0
---
java -Djruby.memory.max=1280m -Djruby.stack.max=2048k -Xmx1280m -Xms256m -Xss2048k -Djffi.boot.library.path=/opt/rubyrep/jruby/lib/native/i386-Linux:/opt/rubyrep/jruby/lib/native/ppc-Linux:/opt/rubyrep/jruby/lib/native/x86_64-Linux -Djava.awt.headless=true -Xbootclasspath/a:/opt/rubyrep/jruby/lib/jruby.jar -classpath : -Djruby.home=/opt/rubyrep/jruby -Djruby.lib=/opt/rubyrep/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /opt/rubyrep/jruby/bin/jgem install activerecord activerecord-jdbc-adapter
Make sure both databases that you are syncing have the schema loaded. This is easiest with a zodbconvert to both databases at the same time, but a schema export from one database without data will work as well.
