As more and more teams and companies adopt Apache Kafka, you can find yourself wanting to share data via replication of one or more topics from one cluster to another. While replication of an entire cluster with all of it’s topics as a means of failover can be achieved with tools such as Mirror Maker and Confluent Replicator, for replication of a single topic there are fewer examples. Even more so when the source topic is serialized with Avro, with the schema stored in Confluent Schema Registry.

In Clojure we can take or drop elements from a collection based on a predicate using the functions take-while and drop-while. With the function take-while we take elements as long as the predicate returns true. Once the predicate returns false the function stops returning elements. Using the function drop-while we skip elements in the collection if the predicate returns true. If the predicate returns false the remaining elements in the collection are returned.

The map data structure is used a lot in Clojure. When we want to use Java objects in our Clojure code we can convert the Java object to a map with the bean function. This function will use reflection to get all the properties of the Java object and converts each property with the property value to a key with value in the resulting map. The bean function will not recursively convert nested objects to a map, only the top-level properties are turned into key value pairs.

It is very easy to work with Java classes in Clojure. If we want to create a new object based on a Java class and invoke methods to initialize the object directly we can use the doto macro. The first argument is an expression to create a new object and the rest of the arguments are functions to invoke methods on the newly created object. The object returned from the first argument is passed as first argument to the method invocations. The doto function returns the object that is created with the first argument.

The clojure.string namespace contains a lot of useful functions to work with string values. The escape function can be used to replace characters in a string with another character. The function accepts as first argument the string value and the second argument is a map. The map has characters as key that need to be replaced followed by the value it is replaced with. For example the map {\a 1 \b 2} replaces the character a with 1 and the character b with 2.

When we use a function as argument for the map function that returns a collection we would get nested collections. If we want to turn the result into a single collection we can concatenate the elements from the collections by applying the concat function, but we can do this directly with the function mapcat. The function mapcat takes as first argument a function (that returns a collection) and one or more collections as next arguments.

When we are working with sets in Clojure we can use some useful functions from the clojure.set namespace. In a previous post we learned how we can get the difference of several sets. To get the union of several input sets we use the union function of the clojure.set namespace. The function returns a new set that is the union of unique elements from the input sets. A nil value is ignored by the union function.

If we want to get the values from a set that are not part of one or more other sets we must use the difference function in the clojure.set namespace. The function returns a set with all values from the first set that are different from values in other sets.

In Clojure functions are everywhere. In a previous post we learned that sets can be functions, but Clojure also makes keywords functions. A keyword is a symbol starting with a colon (:) and is mostly used in map entries as key symbol. The keyword as function accepts a map as single argument and returns the value for the key that equals the keyword in the map or nil if the keyword cannot be found.

One of the nice things in Clojure is that some data structures are also functions. For me this felt rather strange when learning Clojure (coming from Java), but it can be very powerful. A set in Clojure is also a function. The set as function accept a single argument and it return nil when the argument is not part of the set, otherwise the argument value is returned. This behaviour also makes a set as function a nice predicate to be used for example in collection functions.