Chapter 11. Workbench High Availability

The VFS repositories (usually git repositories) stores all the assets (such as rules, decision tables, process definitions, forms, etc). If that VFS resides on each local server, then it must be kept in sync between all servers of a cluster.

Use Apache Zookeeper and Apache Helix to accomplish this. Zookeeper glues all the parts together. Helix is the cluster management component that registers all cluster details (nodes, resources and the cluster itself). Uberfire (on top of which Workbench is build) uses those 2 components to provide VFS clustering.

To create a VFS cluster:

Download Apache Zookeeper and Apache Helix.
Install both:
1. Unzip Zookeeper into a directory ($ZOOKEEPER_HOME).
2. In $ZOOKEEPER_HOME, copy zoo_sample.conf to zoo.conf
3. Edit zoo.conf. Adjust the settings if needed. Usually only these 2 properties are relevant:
```
# the directory where the snapshot is stored.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
```
4. Unzip Helix into a directory ($HELIX_HOME).
Configure the cluster in Zookeeper:
1. Go to its bin directory:
```
$ cd $ZOOKEEPER_HOME/bin
```
2. Start the Zookeeper server:
```
$ sudo ./zkServer.sh start
```
  If the server fails to start, verify that the dataDir (as specified in zoo.conf) is accessible.
3. To review Zookeeper's activities, open zookeeper.out:
```
$ cat $ZOOKEEPER_HOME/bin/zookeeper.out
```
Configure the cluster in Helix:
1. Go to its bin directory:
```
$ cd $HELIX_HOME/bin
```
2. Create the cluster:
```
$ ./helix-admin.sh --zkSvr localhost:2181 --addCluster kie-cluster
```
  The zkSvr value must match the used Zookeeper server. The cluster name (kie-cluster) can be changed as needed.
3. Add nodes to the cluster:
```
# Node 1
$ ./helix-admin.sh --zkSvr localhost:2181 --addNode kie-cluster nodeOne:12345
# Node 2
$ ./helix-admin.sh --zkSvr localhost:2181 --addNode kie-cluster nodeTwo:12346
...
```
  Usually the number of nodes a in cluster equal the number of application servers in the cluster. The node names (nodeOne:12345 , ...) can be changed as needed.
  Note
  nodeOne:12345 is the unique identifier of the node, which will be referenced later on when configuring application servers. It is not a host and port number, but instead it is used to uniquely identify the logical node.
4. Add resources to the cluster:
```
$ ./helix-admin.sh --zkSvr localhost:2181 --addResource kie-cluster vfs-repo 1 LeaderStandby AUTO_REBALANCE
```
  The resource name (vfs-repo) can be changed as needed.
5. Rebalance the cluster to initialize it:
```
$ ./helix-admin.sh --zkSvr localhost:2181 --rebalance kie-cluster vfs-repo 2
```
6. Start the Helix controller to manage the cluster:
```
$  ./run-helix-controller.sh --zkSvr localhost:2181 --cluster kie-cluster 2>&1 > /tmp/controller.log &
```
Configure the security domain correctly on the application server. For example on WildFly and JBoss EAP:
1. Edit the file $JBOSS_HOME/domain/configuration/domain.xml.
  For simplicity sake, presume we use the default domain configuration which uses the profile full that defines two server nodes as part of main-server-group.
2. Locate the profile full and add a new security domain by copying the other security domain already defined there by default:
```
<security-domain name="kie-ide" cache-type="default">
    <authentication>
         <login-module code="Remoting" flag="optional">
             <module-option name="password-stacking" value="useFirstPass"/>
         </login-module>
         <login-module code="RealmDirect" flag="required">
             <module-option name="password-stacking" value="useFirstPass"/>
         </login-module>
    </authentication>
</security-domain>
```
  Important
  The security-domain name is a magic value.

Configure the system properties for the cluster on the application server. For example on WildFly and JBoss EAP:

Edit the file $JBOSS_HOME/domain/configuration/host.xml.

Locate the XML elements server that belong to the main-server-group and add the necessary system property.

For example for nodeOne:

<system-properties>
  <property name="jboss.node.name" value="nodeOne" boot-time="false"/>
  <property name="org.uberfire.nio.git.dir" value="/tmp/kie/nodeone" boot-time="false"/>
  <property name="org.uberfire.metadata.index.dir" value="/tmp/kie/nodeone" boot-time="false"/>
  <property name="org.uberfire.cluster.id" value="kie-cluster" boot-time="false"/>
  <property name="org.uberfire.cluster.zk" value="localhost:2181" boot-time="false"/>
  <property name="org.uberfire.cluster.local.id" value="nodeOne_12345" boot-time="false"/>
  <property name="org.uberfire.cluster.vfs.lock" value="vfs-repo" boot-time="false"/>
  <!-- If you're running both nodes on the same machine: -->
  <property name="org.uberfire.nio.git.daemon.port" value="9418" boot-time="false"/>
</system-properties>

And for nodeTwo:

<system-properties>
  <property name="jboss.node.name" value="nodeTwo" boot-time="false"/>
  <property name="org.uberfire.nio.git.dir" value="/tmp/kie/nodetwo" boot-time="false"/>
  <property name="org.uberfire.metadata.index.dir" value="/tmp/kie/nodetwo" boot-time="false"/>
  <property name="org.uberfire.cluster.id" value="kie-cluster" boot-time="false"/>
  <property name="org.uberfire.cluster.zk" value="localhost:2181" boot-time="false"/>
  <property name="org.uberfire.cluster.local.id" value="nodeTwo_12346" boot-time="false"/>
  <property name="org.uberfire.cluster.vfs.lock" value="vfs-repo" boot-time="false"/>
  <!-- If you're running both nodes on the same machine: -->
  <property name="org.uberfire.nio.git.daemon.port" value="9419" boot-time="false"/>
</system-properties>

Make sure the cluster, node and resource names match those configured in Helix.

11.1.2. jBPM clustering

In addition to the information above, jBPM clustering requires additional configuration. See this blog post to configure the database etc correctly.

Chapter 11. Workbench High Availability

11.1.1. VFS clustering

Note

Important

11.1.2. jBPM clustering