Clustering

Since Camel 2.20

Experimental feature

In Camel 2.20 we have introduced the following cluster related SPI:

  • Cluster Service

    A regular Camel service that manages cluster resources such as views (see below)

  • Cluster View

    Represent a view of the cluster with its own set of isolated resources. As today views provide supports for:

    • Leader Election

    • Topology events like members joining/leaving the cluster)

  • Cluster Member

    Represent a member of the cluster.

Cluster SPI Set-Up

A Cluster Service is just like any other camel service so set it up you only need to register your implementations to the camel context:

MyClusterServiceImpl service = new MyClusterServiceImpl();

context.addService(service);

The configuration of the Cluster Service depends on the implementation you have choosed. Out of the box camel provides the following implementations:

Type Module Class

atomix

camel-atomix

org.apache.camel.component.atomix.cluster.AtomixClusterService

consul

camel-consul

org.apache.camel.component.consul.cluster.ConsulClusterService

file

camel-core

org.apache.camel.component.file.cluster.FileLockClusterService

kubernetes

camel-kubernetes

org.apache.camel.component.kubernetes.cluster.KubernetesClusterService

zookeeper

camel-zookeeper

org.apache.camel.component.zookeeper.cluster.ZooKeeperClusterService

On Spring/Blueprint all the Cluster Service instances are automatically added to the camel context.

Configuration examples:

  • Spring Boot

    camel.component.file.cluster.service.enabled = true
    camel.component.file.cluster.service.id = ${random.uuid}
    camel.component.file.cluster.service.root = ${java.io.tmpdir}
  • Spring XML

    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="
             http://www.springframework.org/schema/beans
             http://www.springframework.org/schema/beans/spring-beans.xsd
             http://camel.apache.org/schema/spring
             http://camel.apache.org/schema/spring/camel-spring.xsd">
    
    
      <bean id="zx" class="org.apache.camel.component.zookeeper.cluster.ZooKeeperClusterService">
        <property name="id" value="node-1"/>
        <property name="basePath" value="/camel/cluster"/>
        <property name="nodes" value="localhost:2181"/>
      </bean>
    
      <camelContext xmlns="http://camel.apache.org/schema/spring" autoStartup="false">
        ...
      </camelContext>
    
    </beans>

Cluster SPI Usage

The Cluster SPI is leveraged by the folowing new implementations:

  • ClusteredRoutePolicy

    This is an implementation of a RoutePolicy that starts the routes it is associated to when the Cluster View it uses takes the leadership

    context.addRoutes(new RouteBuilder {
        @Override
        public void configure() throws Exception {
            // Create the route policy
            RoutePolicy policy = ClusteredRoutePolicy.forNamespace("my-ns");
    
            // bind the policy to one or more routes
            from("timer:clustered?delay=1000&period=1000")
                .routePolicy(policy)
                .log("Route ${routeId} is running ...");
        }
    });

    To apply the same policy to all the rooutes a dedicated RoutePolicyFactory can be used:

    // add the clustered route policy factory to context
    context.addRoutePolicyFactory(ClusteredRoutePolicyFactory.forNamespace("my-ns"));
    
    context.addRoutes(new RouteBuilder {
        @Override
        public void configure() throws Exception {
            // bind the policy to one or more routes
            from("timer:clustered?delay=1000&period=1000")
                .log("Route ${routeId} is running ...");
        }
    });
  • ClusteredRouteController

    This is an implementation of the RouteController SPI that lets the camel context start then starts/stops the routes when the leadership is taken/lost. This is well integrated with spring-boot apps so assuming you have tour routes set-up like:

    @Bean
    public RouteBuilder routeBuilder() {
        return new RouteBuilder() {
            @Override
            public void configure() throws Exception {
                from("timer:heartbeat?period=10000")
                    .routeId("heartbeat")
                    .log("HeartBeat route (timer) ...");
                from("timer:clustered?period=5000")
                    .routeId("clustered")
                    .log("Clustered route (timer) ...");
            }
        };
    }

    You can then leverage spring-boot configuration to make them clustered:

    # enable the route controller
    camel.clustered.controller.enabled = true
    
    # define the default namespace for routes
    camel.clustered.controller.namespace = my-ns
    
    # exlude the route with id 'heartbeat' from the clustered ones
    camel.clustered.controller.routes[heartbeat].clustered = false
  • Master Component

    The master component is similar to a ClusteredRoutePolicy but it works on consumer level so it ensure the only a single endpoint in a cluster is consuming resources at any point in time. Set it up is very easy and all you need is to prefix singleton endpoints according to the master component syntax:

    master:namespace:delegateUri

    A concrete example:

    @Bean
    public RouteBuilder routeBuilder() {
        return new RouteBuilder() {
            @Override
            public void configure() throws Exception {
                from("timer:heartbeat?period=10000")
                    .routeId("heartbeat")
                    .log("HeartBeat route (timer) ...");
                from("master:my-ns:timer:clustered?period=5000")
                    .routeId("clustered")
                    .log("Clustered route (timer) ...");
            }
        };
    }