Jackson Mixin To The Rescue

Many a times it is not possible to annotate classes with Jackson Annotations simply for serialization/deserialization needs. There could be many reasons, for example

  • Classes which needs to be serialized/deserialized are 3rd party classes.
  • You don’t want Jackson invade into your code base every where.
  • You want cleaner and modular design.

Jackson Mixin feature would help solve above problems easily. Lets consider an example :

Let’s say you want to serialize/deserialize following class (Note that it does not have getter/setter)

 public class Address {

	private String city;
	private String state;

	public Address(String city, String state) {
		this.city = city;
		this.state = state;;
	}

	@Override
	public String toString() {
		return "Address [city=" + city + ", state=" + state +  "]";
	}
}

If you try to serialize, you would get the following error

Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: No serializer found for class com.some.package.Address and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) )

To solve the above issue, you have to do the following

  1. Add a default constructor
  2. Add getter/setter for each property

However that is not possible for many cases. Here we can use Jackson Mixin to get around with this problem, to do that we have to create corresponding mixing class, as can be seen here, constructor should match as that of your source object and you have to use Jackson annotations (@JsonCreator, @JsonProperty etc.)

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;

public abstract class AddressMixin {

    @JsonCreator
    public AddressMixin(
            @JsonProperty("city") String city,
            @JsonProperty("state") String state) {
        System.out.println("Wont be called");
        
    }
}

Still you will get the following exception

Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: No serializer found for class com.some.package.Address and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) )

It turns out that, we have to tell jackson to use reflection and access the fields.

mapper.setVisibility(mapper.getSerializationConfig()
        	.getDefaultVisibilityChecker()
                .withFieldVisibility(JsonAutoDetect.Visibility.ANY)
                .withGetterVisibility(JsonAutoDetect.Visibility.NONE)
                .withSetterVisibility(JsonAutoDetect.Visibility.NONE)
                .withCreatorVisibility(JsonAutoDetect.Visibility.NONE));

Here is the teset code:

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.databind.ObjectMapper;

public class JacksonMixInTest {

	public static void main(String[] args) throws IOException {
   
	Address address = new Address("Hyderabad",  "Telangana");

        ObjectMapper mapper = buildMapper();

        final String json = mapper.writeValueAsString(address);
        System.out.println(json);

        mapper.addMixIn(Address.class, AddressMixin.class);

        final Address deserializedUser = mapper.readValue(json, Address.class);
        System.out.println(deserializedUser);
    }

	private static ObjectMapper buildMapper() {
		ObjectMapper mapper = new ObjectMapper();
                mapper.setVisibility(mapper.getSerializationConfig()
                .getDefaultVisibilityChecker()
                .withFieldVisibility(JsonAutoDetect.Visibility.ANY)
                .withGetterVisibility(JsonAutoDetect.Visibility.NONE)
                .withSetterVisibility(JsonAutoDetect.Visibility.NONE)
                .withCreatorVisibility(JsonAutoDetect.Visibility.NONE));
		return mapper;
	}
}

Here is the output:

{"city":"Hyderabad","state":"Telangana"}
Address [city=Hyderabad, state=Telangana]

References

 

zookeeper

Running Multiple Zookeeper Instances On Single Windows Machine

This is indeed correct with Zookeeper Runner

Running Zookeeper in windows in made so easy that even grandma can do it.

 Step 1: copy three instances of ZookeeperRunner on your local drive

I have copied it as instance1, instance2 and instance3

zookeeper-multi-instances

Step 2: copy required zookeeper jar files to [RUNNER_HOME]/lib

Note : you have to do it for all the instances.

zookeeper-runner-jars

Step 3: update the zoo config files

Note: you have to do it for all the instances.

client ports

instance1= 2181

instance2=2182

instance3=2183

zookeeper-runner-zoo-config

Step 4: update the wrapper config

zookeeper-runner-wrapper-config

Step 5 : update the myid files

zookeeper-runner-myid

Step 6 : start Zookeeper instances

Note : you have to do it for all the instances.

zookeeper-runner-start

Here are the running instances.

zookeeper-runner-instance1

zookeeper-runner-instance2

zookeeper-runner-instance3

 

Multi Node Distributed Execution Using Infinispan and Dexecutor

We will try to execute Dexecutor in a distributed mode using Infinispan. For the demo we would be setting up multiple infinispan nodes on single machine.

Refer Introducing Dexecutor, to get an introduction on Dexecutor  and to understand the problem we would solve in a distribute fashion. In short:

We would be distributing the execution of dexecutor tasks on Infinispan nodes in a single machine.

To do that one of the nodes would act as master and submit the tasks to DistributedExecutorService to be executed by other infinispan worker nodes.

Step 1: Add dexecutor-infinispan dependency

 


<dependency>
  <groupId>com.github.dexecutor</groupId>
  <artifactId>dexecutor-core</artifactId>
 <version>1.0.2</version>
</dependency>

 

Step 2: Add the default jgroups.xml

Step 3: Create the CacheManager

private DefaultCacheManager createCacheManagerProgrammatically(final String nodeName, final String cacheName) {
	DefaultCacheManager cacheManager = new DefaultCacheManager(globalConfiguration(nodeName), defaultConfiguration());
	cacheManager.defineConfiguration(cacheName, cacheConfiguration());
	return cacheManager;
}

private GlobalConfiguration globalConfiguration(String nodeName) {
	return GlobalConfigurationBuilder
				.defaultClusteredBuilder()
				.transport()
				.nodeName(nodeName)
				.addProperty("configurationFile", "jgroups.xml")
				.build();
}

private Configuration defaultConfiguration() {
	return new ConfigurationBuilder()
				.clustering()
				.cacheMode(CacheMode.REPL_SYNC)
				.build();
}

private Configuration cacheConfiguration() {
	return new ConfigurationBuilder()
				.clustering()
				.cacheMode(CacheMode.DIST_SYNC)
				.hash()
				.numOwners(2)
				.build();
}

Step 4 : Create Dexecutor instance using InfinispanExecutionEngine


EmbeddedCacheManager cacheManager = createCacheManagerProgrammatically(nodeName, cacheName);
final Cache<String, String> cache = cacheManager.getCache(cacheName);
DefaultExecutorService distributedExecutorService = new DefaultExecutorService(cache);
DefaultDependentTasksExecutor<Integer, Integer> dexecutor = newTaskExecutor(distributedExecutorService);

private DefaultDependentTasksExecutor<Integer, Integer> newTaskExecutor(final DistributedExecutorService executorService) {
	return new DefaultDependentTasksExecutor<Integer, Integer>(taskExecutorConfig(executorService));
}

private DependentTasksExecutorConfig<Integer, Integer> taskExecutorConfig(final DistributedExecutorService executorService) {
	return new DependentTasksExecutorConfig<Integer, Integer>(executionEngine(executorService), new SleepyTaskProvider());
}

private InfinispanExecutionEngine<Integer, Integer> executionEngine(final DistributedExecutorService executorService) {
	return new InfinispanExecutionEngine<Integer, Integer>(executorService);
}

Step 5: Only master should create tasks

if (isMaster) {
	DefaultExecutorService distributedExecutorService = new DefaultExecutorService(cache);
	DefaultDependentTasksExecutor<Integer, Integer> dexecutor = newTaskExecutor(distributedExecutorService);

	buildGraph(dexecutor);
	dexecutor.execute(ExecutionBehavior.TERMINATING);
}

Refer the full code here

Step 4: Run the Application

Terminal #1 : run as worker

mvn test-compile exec:java -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.infinispan.Node" -Dexec.classpathScope="test" -Dexec.args="s node-A"

Terminal #2: run as worker

mvn test-compile exec:java -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.infinispan.Node" -Dexec.classpathScope="test" -Dexec.args="s node-B"

 

Terminal #3 : run as master

mvn test-compile exec:java  -Dexec.classpathScope="test" -Djava.net.preferIPv4Stack=true -Dexec.mainClass="com.github.dexecutor.infinispan.Node" -Dexec.args="m node-C"

Here is the output:

dexecutor-multi-node-infinispan-single-machine

References

Introducing Dexecutor

From the Dexecutor Website

Executing dependent/Independent tasks in a reliable way, is made so easy that even grandma can do it.

That is indeed true with Dexecutor, specially considering the complexity involved writing error free programs involving dependent/Independent tasks. Without Dexecutor, you would end up writing tons of plumbing code rather than concentrating on the business.

With Dexecutor, you model your requirements in terms of Graph in an object oriented way, and rest would be taken care by the framework in a reliable way. Dynamically built graph, defines the executing order, what are all tasks that should run in parallel/sequential. For example if the graph built is the following

dexecutor_graph

Then it means, Task#1,Task#12 and Task#11 would run in parallel, once one of them finishes execution, its child nodes would begin execution. For example lets say if Task#1 finishes, then Task#2 and Task#3 would begin, similarly with Task#12 and Task#11, until all the tasks are executed or if a task end up in an error (If the execution behaviour is terminating)

That’s great indeed…. But how it is done ?

 DefaultDependentTasksExecutor<Integer, Integer> executor = newTaskExecutor();

//Build the graph
executor.addDependency(1, 2);
executor.addDependency(1, 3);
executor.addDependency(3, 4);
executor.addDependency(3, 5);
executor.addDependency(3, 6);
//executor.addDependency(10, 2); // cycle
executor.addDependency(2, 7);
executor.addDependency(2, 9);
executor.addDependency(2, 8);
executor.addDependency(9, 10);
executor.addDependency(12, 13);
executor.addDependency(13, 4);
executor.addDependency(13, 14);
executor.addIndependent(11);

// Execute
executor.execute(ExecutionBehavior.NON_TERMINATING);

Above code shows, Dexecutor expose two kind of APIs,

  • An API to construct the Graph (addDependency, addIndependent)
  • An API to Start execution (execute)

That’s simple indeed, You may be wondering, How the tasks are mapped?

TaskProvider to the rescue. TaskProviders, maps a graph node to a task, and is basically provided during Dexecutor instance creation. Refer to JavaDoc of Dexecutor Implementation, and an example of how to do it.

Well, the example is very simple, Can we get any real time example of how Dexecutor can be used?

Yes indeed, Dexecutor does have a sample application which provides a real time scenario, refer it for more details.

Finally I would like to quote the features of Dexecutor, Here is a snapshot from the website.

dexecutor-features

Refrences

Publish Your Artifacts To Maven Central

Step by step guide to deploying  arftifacts to maven central. For this blog we will use Github

Prerequisites

Step1: Make sure if you have installed JDK,Maven, Git etc.

Step 2: Create Github account (If not already done)

Refer https://help.github.com/articles/signing-up-for-a-new-github-account/

Step 3: Create a new Github repository

Refer https://help.github.com/articles/create-a-repo/

Step 4: Add a new SSH key to your Github account

https://help.github.com/articles/create-a-repo/

Step 5: Push the code to Github

https://help.github.com/articles/adding-an-existing-project-to-github-using-the-command-line/

Step 6: Sign up for Sonatype Jira account

https://issues.sonatype.org/secure/Signup!default.jspa

Step 7: Create A Jira issue for new project hosting

https://issues.sonatype.org/secure/CreateIssue.jspa?issuetype=21&pid=10134

Here is a sample request https://issues.sonatype.org/browse/OSSRH-24465

sonatype-new-project-jira-request

Step 8: Install GNU PG

Download from https://www.gnupg.org/download/ and install in your OS, verify as follows

Step 9: Generate the key pair

It would prompt you for phrase

gpg-passphrase-prompt

Publishing Steps

Step 1: Add distributed management section in your POM.

Add deploy plugin

Add distribution Management Section in your POM

Step 2: Add ossrh server detail into your settings.xml under M2_REPO home.

Id element of servers/server in settings.xml should be identical to id element of snapshotRepository and repository in your POM file.

Step 3:  Add SCM section in your POM

Step 4:  Add maven release plugin

Add GPG passphrase as profile in maven settings.xml, passphrase you have generated while generating the key

Add nexus staging maven plugin

Step 5 : Add source and javadoc plugin

Step 6:  Configure to Sign artificats while releasing

Configure to sign artifacts while releasing

Step 7: Publish GPG key pair

 

Distribute your key to GPG servers

gpg –keyserver [KEY_SERVER] –send-key [KEY_ID]

KEY_ID in the above case is 5694AA563793429557F1727835B3BD2A223A

Some of the key servers

  • pool.sks-keyservers.net
  • gnupg.net:11371
  • keys.pgp.net
  • surfnet.nl
  • mit.edu

Step 8: Do the release

mvn clean

mvn release:prepare

mvn release:perform

Step 9: push the tag and code to your remote repo

git push –tags

git push origin master

Step 10: Verify the sonatype repository

verify-sonatype-repo

Step 11: Update the Sonatype Jira Ticket

update-sonatpe-jira-about-release

Refer this example POM file for more detail

If something goes wrong

Step 1: Undo the release

git reset –hard HEAD~1(You may have to do it second time, depending upon when the error happened)

git reset –hard HEAD~1

Step 2: Delete the tag

git tag -d tagName

git push origin :refs/tags/tagName

References

  1. http://central.sonatype.org/pages/producers.html
  2. http://central.sonatype.org/pages/ossrh-guide.html
  3. http://central.sonatype.org/pages/apache-maven.html
  4. http://central.sonatype.org/pages/working-with-pgp-signatures.html
  5. https://github.com/dexecutor/dependent-tasks-executor
  6. http://stackoverflow.com/questions/5195859/push-a-tag-to-a-remote-repository-using-git
  7. https://oss.sonatype.org/
  8. https://fedoraproject.org/wiki/Creating_GPG_Keys
  9. https://ekaia.org/blog/2009/05/10/creating-new-gpgkey/
  10. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Step_by_Step_Guide/s1-gnupg-keypair.html
  11. https://wiki.debian.org/Keysigning